Patent attributes
Systems and methods for data quality monitoring are provided. Various embodiments include a data monitoring system that integrates into a data pipeline. The data monitoring system may receive a call from the data pipeline to analyze data inputs entering the data pipeline. The monitoring system can generate metadata describing the data inputs and compare the generated metadata with previously generated metadata to determine if the data inputs are historically consistent. The data monitoring system may return a consistency measure to the data pipeline. In further embodiments, the data monitoring system can generate metadata describing data outputs from the data pipeline and compare the output metadata to previously generated output metadata. In further embodiments, the data monitoring system may operate as a read only entity in a database. The monitoring system may monitor for changes in the database and determine when adverse changes occur in the database.
