US Patent 10586369 Using dialog and contextual data of a virtual reality environment to create metadata to drive avatar animation

One or more services may generate audio data and animations of an avatar based on input text. A speech input ingestion (SII) service may identify tags of objects in a virtual environment and associate tags of those objects with words in the input text, which may be stored as metadata in speech markup data. This association may enable an animation service to generate gestures toward objects while animating an avatar, or may be used to create animations or effects of the object. The SII service may analyze input text to identify dialog including multiple speakers associated with the text. The SII service may create metadata to associate certain words with respective speakers (avatars) of those words, which may be processed by the animation service to animate multiple avatars speaking the dialog.