Choosing appropriate methods of ingestion
To ensure that your product's solution requirements are met with the appropriate ingestion method, consider these options:
- Adopt the Batch Ingestion API for data integration use cases in which the direct Data Fabric ingestion capabilities are required. The Batch Ingestion API enables the direct transfer of large file sizes and archived data. Alternatively, you can use the Streaming Ingestion method, which allows for publishing real-time data from the source.
- If, for example, your application is BOD-capable, or you are building an integration footprint into your solution, adopt ION as the integration method for sending data to Data Lake
This table shows the use cases to help you choose the appropriate method of ingestion:
Method | Use case |
---|---|
Batch Ingestion |
Use this method in these cases:
|
Streaming Ingestion | Use this method in these cases:
|
ION Data Lake Flows |
Use this method in these cases:
|
ION Data Loader | Use this method when migrating on-premises data that is stored within database tables, views, and materialized views to Data Lake. |
Atlas Upload widget | Use this method in these cases:
|
Caution:
Do not send the same data through
different ingestion methods in parallel. This can cause data duplication in Data Lake.