US Patent 9762851 Shared experience with contextual augmentation

Techniques for augmenting video content to enhance context of the video content are described herein. In some instances, a video may be captured at a first location and transmitted to a second location, where the video is output in real-time. A context surrounding a user that is capturing the video and/or a user that is viewing the video may be used to augment the video with additional content. For example, the techniques may process speech or other input associated with either user, a gaze associated with either user, a previous conversation for either user, an area of interest identified by either user, a level of understanding of either user, an environmental condition, and so on. Based on the processing, the techniques may determine augmentation content. The augmentation content may be displayed with the video in an overlaid manner to enhance the experience of the user viewing the video.