Configuration parameters for running ETL [run_etl.bat/sh]

These parameters are specified on the client side[run_etl.bat/sh] and as background process parameters for XMDBToolBkg.ETL BKG process:

  • etlChunk: It is used to chunk the full etl file and get the etl data file as number of small individual files. You need to set it to true for chunking the files. By default, its false. it is used only on the client side.
  • etlChunkSize: By default 10000 lines. This will suggest the number of lines in each chunked file. It is used only on the client side.
  • deploy_tables: It must be set to Y. This is required parameter along with the mapping_classes parameter for running full etl. By default it is null.
  • deploy_specified_tables: It is used to specify the tables where the etl data needs to be deployed. Multiple tables can be specified separated by a colon. When this parameter is used only specified tables are mapped. For example; deploy_specified_tables EXPENSE_REPORT:EXPENSE_REPORT_LINE_ITEM.
  • mapping_classes: It must be set to USER_HIERARCHY. This is required parameter for running full etl. By default it is null.
  • fulletlrerun: It is used to run the full etl again in case of any errors. The beauty is, the original full etl is still in place (since it is deleted after the last chunk file has been downloaded), so no etl is generated again. The old file is chunked again and sent out. This parameter needs to be set to true if the rerun is required.
  • incremental: Set this parameter to true to run incremental etl. Default value is null.
  • manual_lock_rdb: It is used to unlock database. Set to N to unlock the database.
  • startDate: The start date that must be specified for running etl in a date range or in full etl with date range. Format must be MM-DD-YYYYY HH:MM:SS. When dates are specified in the command line, add a '%' between the data and the time part. For example, MM-DD-YYYYY%HH:MM:SS.
  • endDate: The end date that must be specified for running etl in a date range or in full etl with date range. Format must be MM-DD-YYYYY HH:MM:SS. When dates are specified in the command line, add a '%' between the data and the time part. For example, MM-DD-YYYYY%HH:MM:SS. Default date for full etl is current date.
  • db_type: It is used to specify the database type. By default it is mssql.
  • sql_end_delimiter: It is used to specify the delimiter. By default it is GO.
  • to_file_begin_tx: It is used to specify the beginning of the transaction.
  • to_file_commit_cmd: It is used to specify the commit.
  • toFile: It is used to specify the etl data file location. On Client, the full location is required and on the server only filename is required. The etl run's db-to-db if toFile is not specified.
  • splitETLFile: It is set to true, if we need the etl file to be generated as individual files (each file with the table name). By default it is false. It is used only on the server side.
  • etl_sqlTrace: If its set to true, the etl_sqlTrace file is generated in the log folder in the client side. Remember that this file is generated on when ETL DB - to - DB is run. Can be used on the client side and as bkg params to the XMDBToolBKG.ETL bkg process.