Integration tools

Data Publishing Management - Initial load through Data Lake Batch Ingestion API

The LN Data Publishing Management has adopted a new API for publishing records from the Initial Load: the Data Lake Batch Ingestion API. This API is provided by Data Fabric and makes it possible to send the messages directly to Data Lake. If you switch from the current IMS interface to Batch Ingestion API, ION can be skipped in the process. This provides multiple advantages:

  • Faster performance
  • No temporary ION bottlenecks caused by thousands of messages from Initial Load

We strongly recommend that you switch to Batch Ingestion. You can seamlessly make this change in the Data Publishing Parameters (ttdpm5130m000) session under Publication Method. No additional configuration is required.

If you are publishing not only to Data Lake and you require LN data also in other applications, you can still switch to Batch Ingestion. The usual Data Flow -is LN -> ION -> Data Lake + Application. Instead of this usual Data Flow, you can use ION Data Flow to retrieve messages from Data Lake and then route them to your application. The new Data Flow is LN -> Data Lake -> ION -> Application. This is now the preferred routing.

If you do not use Data Lake and only send LN messages to other applications, you should keep the current Data Flow using IMS.

Because the Batch Ingestion API currently does not offer any ping endpoint, the Check Parameters button has been renamed to Check IMS Connection . If you click this button while Batch Ingestion is the selected Initial Load Publication Method, the only publishing capabilities to Data Catalog and IMS are checked. Publishing capabilities to Batch Ingestion API are not checked. You can verify whether the Batch Ingestion publication is correctly set up. To do so, publish, for example, one record from one table using the Publish Data (ttdpm5205m000) session.