Various arrangements are presented for determining a state of a traffic light. A first analysis of a scene may be performed that includes the traffic light. The first analysis may result in the output of a first classification result and a first confidence score. A second, different analysis of the scene may be performed that also outputs a classification result and a confidence score. A fusion process may be performed to fuse the classification results into a fused classification result based on the confidence scores and one or more weighting metrics. A vehicle may then be driven at least partially based on the fused classification result.