US Patent 11789792 Message passing interface (MPI) collectives using multi-allgather

Systems, methods, and devices for performing computing operations are provided. In one example, a system is described to include an endpoint belonging to a collective that is organized as a hierarchical tree. The collective includes one or more application groups that are connected to leaf nodes of the hierarchical tree, where each application-level process in the one or more application groups initiate processing of data based on an order of arrival known for at least one other application-level process joining the one or more application groups.