US Patent 10977149 Offline simulation system for optimizing content pages

A testing environment in which offline simulations can be run to identify policies and/or prediction models that result in more valuable content being included in content pages is described herein. For example, the offline simulations can be run in an application executed by an experiment device using data gathered by a production content delivery system. The simulation application can test any number of different policies and/or prediction models without impacting users that use a production content delivery system to request content.