Data versioning

Data Lake provides low-cost storage and retention of enterprise data with a coupled compute layer in the Compass SQL platform to interrogate and analyze the contents stored within the object store. To provide an effective and meaningful experience when processing data, a metadata-driven framework is used by Data Lake. It derives how to process and present data to an application, service, or client in consultation with Data Lake.

With this metadata-driven approach, clients can surface queries that present operational states of tables as compared with the system of record. They can also query across historical states of what data previously looked like.

To achieve this, data replicated to Data Lake from Infor CloudSuites includes a versioning mechanism called a variation ID. A variation ID, a special indicator in Data Catalog, is typically represented as an increasing integer or time stamp. It provides a sorting path for Compass SQL to identify what data must be returned to the calling client based on their query construction. Clients can query Data Lake to retrieve the current state of a transactional order table and historical changes that have occurred in the past. They can also analyze data that has since been removed from the system of record.

For more information see Compass SQL Reference.