US Patent 11252260 Efficient peer-to-peer architecture for distributed machine learning

A computer in a distributed peer-to-peer system is disclosed. The distributed system includes a plurality of computers configured to run a distributed machine learning (ML) program represented as an expression of a target loss function with a model parameter matrix. The computer includes: a parser module configured to convert a loss function in the distributed program into an expression graph and then one or more multiplication trees; a parameter replica module in communication with the parser module, the parameter replica module configured to maintain the model parameter matrix of the ML program; a compressor module in communication with the parameter replica module, the compressor module configured to extract sufficient factors from the expression graph for updating the model matrix; and a communication module in communication with the compressor module, the communication module configured to send the sufficient factors for updating model matrix to other machines in the distributed system.