US Patent 10572723 Activity detection by joint human and object detection and tracking

A computing device includes a communication interface, a memory, and processing circuitry. The processing circuitry is coupled to the communication interface and to the memory and is configured to execute the operational instructions to perform various functions. The computing device is configured to process a video frame of a video segment on a per-frame basis and based on joint human-object interactive activity (HOIA) to generate a per-frame pairwise human-object interactive (HOI) feature based on a plurality of candidate HOI pairs. The computing device is also configured to process the per-frame pairwise HOI feature to identify a valid HOI pair among the plurality of candidate HOI pairs and to track the valid HOI pair through subsequent frames of the video segment to generate a contextual spatial-temporal feature for the valid HOI pair to be used in activity detection.