Configuring SCP for exporting plan data to Data Lake (MT Cloud)

You must configure SCP and publish the metadata (schema) to export plan data to Data Lake.

These macros can be used for publishing the metadata to Data Catalog:

  • SCV – DL Publish Metadata All
  • SCV – DL Publish Metadata Inbox
  • SCV – DL Publish Metadata Outbox

The metadata related to the SCV Inbox and SCV Outbox tables can now be published to Data Catalog. This process allows users to import planning data from Data Lake to inbox tables and export plan results from outbox tables to Data Lake.

In addition to the columns available in the SCV table, published metadata includes these special ION Data Catalog properties:

  • IdentifierPath: Reference to the primary key columns of the table. The list of columns varies for each table.
  • VariationPath: Reference to the column that contains the latest variation (revision) number for each record that has been exported. The variation_number column is used for all tables.
  • DeleteIndicator: Reference to the column that contains the deletion flag for each record that has been exported. The deleted_flag column is used for all tables.
  • TimestampPath: Reference to the column holding the creation or last update timestamp of each record that has been exported. The modification_time column is used for all tables.