Patent attributes
A method and a system for generating an augmented scene graph of an image and for training an explainable knowledge based (KB) visual question answering (VQA) machine learning (ML) model are provided. A scene graph encoding spatial and semantic features of objects and relations between objects in the image is obtained. An augmented scene graph is generated by embedding a knowledge graph to enhance the scene graph. An embedded set of questions and associated answers related to the image are obtained. The KB VQA ML model is trained to provide an answer to a given question related to the image based on the augmented scene graph and the embedded set of questions and associated answers. The KB VQA ML model is trained to retrieve a subgraph linking the question and the associated answer as a potential explanation for the answer.
