Checking for Data Lake table status

When retrieving records from a Data Lake source, you can check the status of the retrieval operation. This is especially helpful when dealing with large sets of records.

This topic presents the procedure to check the status of the retrieval operation.

With the Data Lake Data Populator form open, click New records in Data Lake.
The system retrieves the number of records in the Data Lake that have been inserted since the date/time displayed in the Last inserted DateTime field. This number displays in the Number of new records in Data Lake field.
To start record retrieval, click AES insert
With the retrieval operation still in progress, click Status.
This opens the Data Lake Table Status form.
Click Check Logs.
This opens the Data Lake Logs form.
Execute Filter-In-Place (click Actions > Execute in Place).
The Data Lake Logs form displays the current status of the data retrieval operation. To update the display, re-execute Filter-In-Place. You can do this as many times as you want, till you reach the end of the data retrieval. You will know that the retrieval operation is complete when you seen an entry similar to this: 75/75 INSERTED - TableName
If there are log entries from a previous operation, you can use the filter options to select only the ones you want to display.
Optionally, you can perform these additional actions:
- Filter the log display by table name, Data Lake (unbound) source, localized (bound) table, or primary key.
- Clear all or some of the log entries.

When data retrieval is complete, you can take additional actions as desired. For example, you can use the IDO Data Viewer form to verify that all expected records have been retrieved.