US Patent 10360581 Automatic performance-triggered feature discovery

Automatic performance triggered model adjustment. A hierarchical feature tree is generated. Each child node's feature is more specific than its respective parent node's feature. A behavioral model comprising features of the feature tree is created and used in the operation of an advertising campaign. A degraded model feature is detected at the discovery system by comparing a performance metric of a model feature from two different time windows. The discovery system matches a node of the feature tree with the degraded feature and selects a prospective model feature from a same-level node or from a lower level node than the matching feature's node. An estimated performance metric for the prospective model feature is determined and the results are used to decide if the model should be updated to include the prospective model feature. The campaign can be operated with the automatically updated model.