US Patent 11983462 Conversation guided augmented reality experience

Systems and methods are provided for providing an augmented reality experience. The systems and methods perform operations comprising: generating, for display by a messaging application, an image comprising one or more augmented reality elements, the one or more augmented reality elements being associated with a configurable entity; receiving, by the messaging application, speech input from a user; determining a schema associated with the one or more augmented reality elements; causing the speech input to be processed by a speech understanding model in accordance with the schema to determine one or more configurable state entity update values; updating the configurable entity associated with the one or more augmented reality elements based on the one or more configurable state entity update values; and modifying the one or more augmented reality elements in the image based on the updated configurable entity.