Validation and execution

These conditions are applicable during the execution process:

If Include Delete is set to No (or not populated) and Export Mode is set to Full (or not populated), the entire contents of the target table is exported.
If Table Name is specified, only the data from the specified table is exported.
If Table Group Name is specified, Include User-Defined = Yes and User-Defined Only = Empty or No, the data from all standard and user defined tables included in the specified SCV table group are exported.
If Table Group Name is specified, Include User-Defined = Yes and User-Defined Only = Yes, the data from all user-defined tables included in the specified SCV table group are exported.
If Table Group Name is specified and Include User-Defined = Empty or No, the data from all standard tables included in the specified SCV table group are exported.
If Table Area is specified, Include User-Defined = Yes and User-Defined Only = Empty or No, the data from all standard and user-defined tables included in the specified table area of the SCV Table Registry are exported.
If Table Area is specified, Include User-Defined = Yes and User-Defined Only = Yes, the data from all user-defined tables included in the specified table area of the SCV Table Registry are exported.
If Table Area is specified and Include User-Defined = Empty or No, the data from the standard tables included in the specified table area of the SCV Table Registry are exported.

These steps are executed to complete the export process:

Retrieve the <Next Variation Number> from the database sequence or exporting to Data Lake for each <Table> in the scope.
Determine the object name to use in the Data Catalog:
1. If Schema Prefix is not specified or Empty, use SCP_<Table> as the object name in the Data Catalog. For example, SCP_SO_DEMAND_PLAN_AGGR.
2. If Schema Prefix is specified, use SCP_<Schema Prefix>_<Table> as the object name in the Data Catalog. For example, SCP_UD_SO_DEMAND_PLAN_AGGR).
Determine the corresponding <Entity> and <Table Area> for the <Table>.
Retrieve the list of columns from the database for the <Table>.
Determine the list of business columns for the <Entity> using SCV Entities.
Determine the list of internal columns for the <Table> using SCV Column Registry in its corresponding <Table Area>.
Add each <Column> retrieved from the database for exporting:
1. If <Column> also exists as property in its corresponding <Entity>, or
2. If <Column> also exists in SCV Column Registry in its corresponding <Table Area>.
Note: The 'scp_validated_flag' Column is available only when Table Area = Inbox.
Find the special column for SCP instance using SCV Column Registry where ion_data_catalog_metadata = "SCPInstance":
1. If the SCPInstance column in SCV Column Registry does not exist in <Table>, add the column to the export, populate the column with the Instance value of the current SCP instance:
  - When multi-instance is not used or if the current SCP instance = 00 (master instance), populate the column with Null.
  - In all other cases, populate the column with the current SCP Instance. For example, 01, 02 and so on.
Find the column for Data Lake property "VariationPath" using SCV Column Registry where ion_data_catalog_metadata = "VariationPath":
1. If the VariationPath column in SCV Column Registry does not exist in <Table>, add the column to the export, populate the column with the <Next Variation Number> value.
Find the column for Data Lake property "TimestampPath" using SCV Column Registry where ion_data_catalog_metadata = "TimestampPath":
1. If the TimestampPath column in SCV Column Registry does not exist in <Table>, add the column for exporting and populate the column with current timestamp in UTC.

Find the column for Data Lake property "DeleteIndicator" using SCV Column Registry where ion_data_catalog_metadata = "DeleteIndicator". The value is determined by "DeleteIndicator" column specified in SCV Column Registry or scp_export_action_code using the following logic:

If the <Table> has DeleteIndicator column specified in SCV Column Registry, 
    and it has a value (not null or not empty), 
       then populate it using this value
else -- check the action code
    If the <Table> has scp_export_action_code column 
       and it has a value (not null and not empty)
        then 
            if the value is Deleted, then set it 'Y'
            Otherwise,                    set it 'N'
    else -- no value exists or specified in neither DeleteIndicator nor scp_export_action_code  
        use 'N'

If <Include Deleted> = "No", exclude the deleted records (as determined above) from the export process.
Determine whether <Export Mode> = 'Full' or 'Incremental':
- If <Export Mode> = "Full", select all records where the TimestampPath column in SCV Column Registry >= value of SCV Configuration Parameter SCP_DEFAULT_START_DATE.
- If <Export Mode> = "Incremental", only the records where the TimestampPath column in SCV Column Registry >= <snapshot window end timestamp for the current table> value and <Table>.scp_export_action_code = Create/Update/Delete.

The export process is performed using the following logic:

Consider only those records where <Table>.export_status = Open.
Mark those records to be sent as <Table>.export_status = Started.
Export records with <Table>.export_status = Started to Data Lake.
If the records are sent successfully, update the export status <Table>.export_status = Exported. Else, update the export status <Table>.export_status = Error.
If the <Export Mode> = "Full" and <Include Deleted> = Yes, the program sends the deleted records first. Later, the variation number is incremented and the other records are sent . This is done to avoid ambiguity when the same export contains both the Delete and Create at the same time.
If the <Export Mode> = "Full" and <Include Deleted> = No, the program sends only the records that are not deleted. ‘Create’ is used as the default value for all records. This is required to maintain backward-compatibility.