US Patent 11282279 System, etc. for creating mixed-reality environment

A system that enables real-time recognition of a real-space environment in a mixed-reality environment includes a server, a portable display device including a display unit for displaying a virtual object to a user and a photographing unit that photographs a real space, and image acquisition devices that acquire images individually from a plurality of fixed points where it is possible to photograph a region in a predetermined real space, the system making it possible to render the virtual object in a superimposed fashion on a real space or a photographed image of the real space, viewed by the user via the display unit, the server or the display device including a three-dimensional-space-data storage part, a table storage part, a color-information determining part, a color-information updating part, a user-environment determining part, a virtual-illumination-information generating part, and a rendering part.