Starting an initial load

To start the initial load, the pipeline must be running.

  1. Open a pipeline.
  2. Ensure that the pipeline is running.
  3. Click Initial load details. The right panel is expanded where you can define the initial load details.
  4. Select one of these options:
    Option Description
    All data Select this option to process all data objects that are in Data Lake when you start the initial load.
    Filter by timestamp Select a date and time in the From and To fields.
    Note: The timestamp in the To field is specified by default and its default value is the timestamp of when the pipeline was last started.
  5. Click Run.
    If the initial load is started successfully, its status changes to Running.
    Note: Data that you get from an initial load from the stream pipeline can be data that you have already processed in the live stream. This depends on the initial load details that you select. Processing large initial load extractions, regardless of which initial load details you select, can take a significant time. This is because of the data volume and number of data objects that exist in Data Lake.