US Patent 11335096 Method, system and electronic device for processing audio-visual data

A method, a system and an electronic device for processing audio-visual data. In the method, a first dataset is obtained, where the first dataset includes several data pairs, and each of the data pairs in the first dataset includes a video frame and an audio clip that match each other. A multi-channel feature extraction network model is established to extract the visual features of each video frame and the auditory features of each audio clip in the first dataset. A contrastive loss function model is established using the extracted visual features and the auditory features to train the multi-channel feature extraction network. A classifier is established to determine whether an input audio-visual data pair is matched.