US Patent 11462325 Multimodal machine learning based clinical predictor

Methods and systems for performing a clinical prediction are provided. In one example, the method comprises: receiving first molecular data of a patient, the first molecular data including at least gene expressions of the patient; receiving first biopsy image data of the patient; processing, using a machine learning model, the first molecular data and the first biopsy image data to perform a clinical prediction of the patient's response to a treatment, wherein the machine learning model is generated or updated based on second molecular data including at least gene expressions and second biopsy image data of a plurality of patients; and generating an output of the clinical prediction.