Workflow to extract and process

With LN Analytics, multiple extract and process workflows are delivered:

LN Foundation Extract And Process
Run this workflow if you only use the Foundation product.
LN Program Cost Ledger Extract And Process
Run this workflow if you only use the Program Cost Ledger product.
LN Quality Management Extract And Process
Run this workflow if you only use the Quality Management product.
LN Analytics Extract And Process
Run this workflow if you use the Foundation, the Program Cost Ledger, and the Quality Management products. The combined workflow is more efficient than separately running workflows for each product.
Note: 
  • To extract and process the LN Analytics spaces, the workflows must be used. Using Publishing All for the Extraction, Finance, Job Shop, Procurement, and Sales spaces, leads to corrupt and duplicate data.
  • If customers add security filters to LN Analytics, data-level security might be applied during the processing of spaces. Consequently, data can be processed partially, which is standard Birst behavior. To bypass this situation, see Extending contentI.

All delivered extract and process workflows have this identical setup:

  • Extraction of data from Data Lake in space LN-Extraction. First, data for the functional area spaces, such as the 'LN Foundaton', 'LN Program Cost Ledger', and 'LN Quality Management' extraction groups are extracted, followed by the 'LN Conformed Dimensions' extraction group. This sequence reduces incidences of late arriving dimensions, where fact data refers to dimension data, but dimension data is not yet available.
  • Processing of data sources. First, data sources in space LN-Extraction are processed, followed by space LN-Model-ConformedDimension and finally the functional area spaces LN-Model-JobShop, LN-Model-Sales, LN-Model-Finance, LN-Model-Procurement, LN-Model Service, LN-Model-Project, and LN-Model-Quality.
  • Setting variables. LN_usr_ExtractWorkflow, LN_usr_ExtractDomain are set and used when workflow log data sources are processed.
    • LN_usr_ExtractWorkflow is set to the name of the workflow.
    • LN_usr_ExtractDomain is set to ‘Conformed Dimensions’, ‘Foundation’, 'Program Cost Ledger', or 'Quality Management'.
    • If you use different “non-standard” LN_usr_ExtractDomain values in a customized workflow, the extraction logic interprets this as a different domain and will start the first extraction from LN_usr_incremental_LoadUTCStartTime.
Note: 
  • Prior to extracting data with an extraction group, the appropriate extraction period must be determined for the domain. This can be done by setting the related LN_usr_ExtractDomain value and executing the LOG_START processing group.
  • After a domain has been extracted and processed successfully, the correct LN_usr_ExtractDomain value is set prior to executing the LOG_END processing group. The next time the workflow is executed, only new records will be extracted during an incremental load.