A method and system for inspecting railway components. The method includes receiving a stream of images containing railway components, detecting a railway component in each image, generating a plurality of feature vectors for each railway component image, measuring the dissimilarity between the railway component and a set of railway components detected in preceding images, in a sliding window, based on the feature vectors.