Data replication

Data replication is used to populate Data Lake. The replication process uses a multi-threaded batch task, which takes snapshots of data that has changed. Each table that supports replication has a column to track when the record was created and when the record was later updated. Record deletions are tracked using a delete history table. For deleted records, only the ID of the record that was deleted is stored.

For a list of WFM tables that support data replication, see Tables supporting data replication.

Replication occurs by sending JSON documents to IMS containing one JSON object for each record that has been inserted, updated, or deleted since the last time data was sent. In Workforce Management, the JSON documents are listed in Maintenance > System Administration > ION > IMS Document Name Configuration with a Document Type of DATA REPLICATION.

The employee time zone is used to calculate the UTC time for some date fields, for example, work detail start and end times. To accurately calculate these UTC times, all employee must have their time zone field set to the correct value.

To replicate data to Data Lake, the Data Replication task must be set up to run periodically in Workforce Management. If your deployment already has WFM data when data replication is enabled, the initial data replication can take several hours or longer, depending on how much data needs to be replicated. After the initial data replication, this task checks which records have changed since the task last ran and replicates the changes to Data Lake using IMS.

Note: Each change made to a specific record does not trigger an individual replication of the change. The specific changes that were made and who made the changes are also not tracked. Therefore, data replicated to Data Lake is not suitable for auditing changes.