Validating data in Data Lake

With Data Fabric Ledger you can identify if and when there is a misalignment between a system of record and Data Lake in terms of the data sent and the data received.

This issue can be temporary or permanently inconsistent data between the two application platforms. To identify when data became inconsistent, administrators can compare a summary of activities generated by a system of record with what was received in Data Lake.

The Data Fabric Ledger is a platform that helps data administrators to identify disparities between data being replicated by systems-of-record and Data Lake. Often, a symptom of delayed or deferred replications results in inaccurate query results. These effects can be temporary, because Data Fabric is an eventually-consistent data platform. Accurately identifying the scale and time of a delivery disparity is addressed by the Ledger.

A dashboard and reporting interface surfaces all ledger activities that are published and reconciled with Data Lake within the last 30 days of activity. Each ledger activity is a unique combination of a data object name and a period for which statistics and measurements were captured and reconciled.

Data objects that are supported by Compass and can be queried by Compass are supported for Ledger-based reconciliation reporting.

See Data Objects and Views panel.