Provided are methods for agent prioritization, which can include determining a primary agent set and generating, based on the primary agent set, a trajectory for the autonomous vehicle. Some methods described also include determining an interaction parameter of agents in the environment. Systems and computer program products are also provided.