Data Flow and Procedural Scripting for Transformation

ETL Services uses two powerful models for data manipulation to provide significant flexibility in how data can be manipulated. The data flow model allows data to be progressively run through a series of different data manipulation steps until an output is finally created. This allows you to sequence pieces of transformations together into powerful chains. Procedural transformations allow more of a programming-style mechanism to create individual transformations. Procedural languages take advantage of the fact that they can use global memory and complex looping and conditional logic to do powerful manipulations.

A data flow model allows you to create a transformation process composed of many steps. In each step, data is sourced either from the original data, or from the results of another step. By stringing many steps together, manipulations can be layered to create powerful transformations.

A basic data flow example is:

Raw Source Data > Combine with lookups and custom logic > Load into a Target Database