When to use each method

Select the technology or technologies that best align for your product's solution needs. For example, when your application is BOD-capable or you are building an integration footprint into your solution. Your application should adopt ION as the integration method to send data to Data Lake.

To benefit from direct ingestion capabilities, your application may choose to adopt the Data Lake's bulk ingestion API. It allows for direct transfer of larger file sizes.

This table shows the use cases to help you choose the correct method of ingestion:

Method Use case

Bulk Ingestion

Use this method in these cases:

  • Your data source is an application that can use REST API to send data objects
  • The data to send is created or read in a batch-like fashion, such as:
    • Publishing data on request.
    • Publishing an initial load.
    • Publishing batch-based business workstreams, for example MRP.

Data Lake Flows (in ION Desk)

Use this method in these cases:

  • Your data source is a connection point in ION.
  • Your data must be transformed before ingesting to Data Lake, for example with ION Scripting or enrichment activities within Data Flows.
  • Multiple application subscribers are interested in receiving published replication data in addition to Data Lake.
Data Loader Use this method when migrating on-premises data stored within database tables, views, and materialized views to Data Lake.
Note: Do not send the same data objects through both the bulk ingestion method and Data Lake flows in parallel as it can lead to data duplication in Data Lake.