A method compares a background image to input images to determine a similarity scores λ for each input image. Then, the background image is updated only if the similarity score for a particular image is less than a predetermined threshold. Presumably, any pixel whose color does not change is part of a static background, and any pixel that does change is part of a moving object. The similarity score controls when input images are scored and the manner the background image is updated.