Reconciliation of data sent in batches

A successfully completed reconciliation process publishes a transaction card to the Ledger user interface.

When an application publishes a transaction to the Ledger service, the Ledger begins a reconciliation process approximately 5 minutes after receipt from an application. This buffer permits any in-flight content and delivery retries to conclude before measurements are taken for accuracy and effectiveness of reporting. After a card is published to the Ledger, the card can be expanded to explore additional details. These details can be used to determine whether there is a misalignment between any of the information provided and then reconciled within Data Lake.

This table shows header attributes on each card:

Attribute Description

Interval start

The timestamp from which an application has begun a measurement window capturing delivery statistics and counters for replicated data objects.

Interval end

The timestamp from which an application has concluded a measurement window for delivery statistics and counters for replicated data.

Application

The application, or system-of-record, associated with the Ledger transaction and replicated data objects.

Data object

The name of the data object being reconciled.

Time in transit

The average time during which data objects included in this reconciliation are in transit from a source application to Data Lake.

Classification

A conclusion resulting in a Match or Mismatch identification for the card. This is determined by whether all of the measurements between an application and Data Lake are in alignment. It depends if differences between statistics from application and Data Lake are equal to 0 or not.

Cards can be expanded to explore additional details identifying how and what measurements were reported during the reconciliation process.

This table shows what details are included on a card:

Attribute Descpription

Data object Sent

The number of objects with specific name sent within defined interval from application to Data Lake.

Data object Ingested

The number of objects with specific name and source_publication_date between start and end of defined interval, ingested to Data Lake.

Instance Count Application

The number of records for all the objects sent by application included in reconciliation.

Instance Count Data Lake

The number of records for all the objects ingested in Data Lake and included in reconciliation.

Row Count Application

The number of unique records for specific table and time.

Row Count Compass

The number of unique records for specific data object and timestamp.