US Patent 10980612 Face tracking and reproduction with post-treatment smile

In an example embodiment, a processing device receives a 3D image of a face of a person. The processing device identifies a first plurality of visible landmarks and a second plurality of visible landmarks on the face from the 3D image, wherein the second plurality of visible landmarks are associated with soft facial tissues. The processing device determines a post-treatment arrangement of the first plurality of visible landmarks based on a first mapping between a current dentition of the person and a post-treatment dentition of the person. The processing device determines, based on applying the post-treatment arrangement of the first plurality of visible landmarks to a second mapping between the first plurality of visible landmarks and the second plurality of visible landmarks, a post-treatment shape of the soft facial tissues. The processing device generates a modified version of the 3D image, wherein the soft facial tissues have the post-treatment shape in the modified version of the 3D image.