Patent attributes
Matrix factorization based gradient compression may be applied to an allreduce operation to improve efficiency including the elimination of unnecessary meta data while maintaining accuracy in training of deep learning (DL) of Artificial Intelligence. This compression may include generating a predetermined matrix and a degree of data compression k as a dimension of the predetermined matrix for a plurality of computing nodes. Each computing node may receive a corresponding matrix of matrices to be allreduced, and each corresponding matrix may be decomposed into a plurality of non-fixed matrices and the predetermined matrix. The plurality of non-fixed matrices may be summed to provide an optimized matrix, which may be multiplied by the predetermined matrix to provide a result matrix. The optimized matrix may be designated as a predetermined matrix. These operations may be repeated until all of the matrices received by the computing nodes have been allreduced.
