Data Transformation Using ETL

In some cases, raw data available for Birst to analyze cannot be directly mapped into a desired logical dimensional model. Either the structure of the source data is too different from the ultimate star model or various complex data manipulations (or complex filtering/augmentation) must be done to ensure it maps correctly. Birst offers ETL Services for this purpose. ETL provides a powerful and flexible mechanism to transform and manipulate data into Birst so that it can be loaded into the dimensional analysis environment.

Note: Birst does not trim any whitespace when importing data. If you want to remove whitespaces, you must trim the whitespace at the source level using an ETL script. If a source contains multiple line breaks, this may also skew data post-publishing.

Birst employs parallel script execution to improve the performance of script execution. When an ETL script is executed, it is executed multi-threaded so long as it does not have a WRITERECORD command in the COMPLETE block or does not make use of an ORDER BY statement in the SELECT statement.

Scripts are executed at the time that a data source is processed. Dependencies are analyzed such that if a script depends on another data source, that data source is loaded first. ETL Services will also identify loops and will not execute if a script refers to itself or one of its dependencies refers to itself.

Note: You should not execute a script if called by another enablement.