Set up to adopt Data Fabric Ingestion APIs for Data Lake

Caution: Do not adopt both Data Fabric ingestion APIs for Data Lake and implement integration from on-premises to MTC Data Lake. This duplicates the replication of data to the Data Lake.

Overview

This topic describes how to configure Mongoose to use the Data Fabric Batch Ingestion API to send data to the Infor Data Lake. This is the recommended approach for new Data Lake integrations, replacing the legacy ION Data Lake Flows method.

The Data Fabric Batch Ingestion API provides a direct ingestion path that does not require ION Desk flow configuration. The data is sent from the Mongoose Replicator service directly to the Data Lake through the Data Fabric private API.

Note: The legacy ION Data Lake Flows method, using IMS and ION Desk, is still supported for backward compatibility. The existing configurations using that method continue to function without changes.

Who should use this

  • System administrators configuring new Data Lake integrations form Mongoose-based applications.
  • Cloud operations teams provisioning Mongoose tenants that require Data Lake connectivity.
  • On-premises administrator integrating with a multi-tenant cloud (MTC) Data Lake environment.

Prerequisites

Before starting, verify these:
  • Infor OS Integration is available with Data Fabric and ION components enabled.
  • Mongoose 2026.10 or later, includes Data Fabric transfer type.
  • Administrative access to Mongoose Configuration Manager, Mongoose Intranets, Sites, and Replication forms.
  • Data Fabric URLs obtained from your Infor OS provisioning
    • Data Fabric Batch Ingestion API base URL
    • Data Catalog URL
  • OAuth credentials, Consumer Key and Consumer Secret, for Data Fabric API authentication.
  • TenantID for your Infor OS environment.

Key benefits of adopting Data Fabric Ingestion APIs

Implementing Data Fabric Batch Ingestion provides several advantages for Data Lake integrations:

  • Optimized for large-scale data archival - The batch ingestion API is designed for high-volume data transfer scenarios, making it well suited for archive and historical data use cases. This provides a scalable and supported method for loading large datasets into the Data Lake.
  • Modernizes Data Lake integration architecture - Data Fabric Batch Ingestion replaces legacy ION Data Lake Flow-based integrations, helping you align with Infor's current Data Lake ingestion strategy and reducing dependence on deprecated technology.
  • Simplifies the ingestion path - By sending data directly from the Mongoose Replicator service to the Data Lake, the solution eliminates dependency on the ION-IMS connector and related flow configuration. This reduces integration complexity and lowers administrative overhead.
  • Reduces configuration and maintenance effort - Fewer integration components and configuration steps make the Data Lake deployment easier to implement, troubleshoot, and maintain over time.
  • Improves long-term supportability - Adopting the recommended ingestion framework helps ensure compatibility with future platform enhancements and reduces the risk associated with legacy integration methods.

Step 1: Configuring an Intranet for Data Fabric

On the Intranets form, create or update an intranet to use the Data Fabric transfer type.
  1. Open the Intranets form
  2. Create new intranet, or edit an existing Data Lake intranet
  3. On the General tab, configure these fields:
    External
    Set the value to Selected
    Transport
    Set the value to ESB
    Transfer Type
    Set the value to Data Fabric.
    Note: This drop down field, formerly labeled as ION Transfer Type, includes these options.
    • ION Messaging value (1) - Legacy IMS-based ingestion.
    • ION Data Lake Flows value (2) - Legacy flow-based ingestion.
    • Data Fabric value (3) - Recommended method.
    Data Catalog URL
    Specify the Data Catalog URL from your Infor OS provisioning
    URL
    Enter the Data Fabric Batch Ingestion API base URL
  4. Save the intranet.

Step 2: Configuring security credentials

A new Security Token Service entry named datafabricoauth is used to hold the OAuth credentials for Data Fabric API communication.

  1. Open the Configuration Manager
  2. Locate the security token configuration for your site
  3. Ensure the Consumer Key and Consumer Secret are configured for the datafabricoauth service.
  4. Click Save
Note: All Data Fabric API communication uses HTTPS with OAuth 1.0 signature-based authentication. The Consumer Key and Consumer Secret are stored in the existing site configuration, no new credential storage mechanism is introduced.

Step 3: Creating a site for the Data Lake

On the Sites form, create a site entry that uses the Data Fabric intranet:

  1. Open the Sites form.
  2. Create a new site entry for the Data Lake.
  3. Select the intranet created in Step 1: Configuring an intranet for Data Fabric.
  4. On the System Info tab, populate the TenantID field.
    Note:  This information can be found in the Infor CloudSuite Portal (CSP) > Multi-Tenant > Customer Environments > Details tab.
  5. On the Site User Map tab, create a map entry using sa or another suitable administrative user account.
  6. Specify a Message Bus Logical ID for your Mongoose-based application. For example: lid://infor.mongoose.datalake.
  7. Click Save.

Step 4: Configuring Replication Categories and Rules

Creating a Replication Category

  1. Open the Replication Categories form.
  2. Create a category with these settings:
    Replication Transfer Type
    Set the value to Data Lake.
    Table Or Function
    Set the value to the name of the table to replicate.
    Object Type
    Set the value to Table.

Creating a Replication Rule

  1. Open the Replication Rules form.
  2. Create a rule with these settings:
    Source Site
    The site from which the data is replicated.
    Target Site
    The Data Lake site created in Step 3: Create Site for the Data Lake.
    Category
    The category created from Creating a Replication Category.
    Interval Type
    Set the value to Immediate.
    Disable Replication
    Cleared (unchecked).
    Update All Columns
    Cleared (unchecked)

Regenerating triggers

  1. Open the Replication Management form.
  2. Regenerate the replication triggers

Step 5: Configuring the Replicator Service

  1. Open the Service Configuration Manager.
  2. On the Replication tab, add the configuration that corresponds to the site being replicated.
    Note:  In the Edit Replication Configuration dialog box do not select the Master Site option.

Step 6: Generating the Data Lake Schema

  1. In your Mongoose application, open the Generate Data Lake Schema form.
  2. For the Site, select the Data Lake site created in Step 3: Creating a site for a Data Lake.
  3. Click Generate Schema.

This creates the schema definition that the Data Fabric API uses to ingest data correctly.

Step 7: Starting the Replicator Service

After all the configuration is complete:
  1. Start or restart the Replicator service.
  2. Verify in the service logs that the Data Fabric transfer type is being used.

Step 8: Testing the integration

  1. Open a form associated with the table being replicated.
  2. Change some data on the form and save.
  3. Verify that the data appears in the Data Lake, typically within minutes.
  4. Check the Replication Messaging Errors to reflect its broader scope covering all transfer types.
    Note: The form formerly named ION Messaging Errors has been renamed to Replication Messaging Errors to reflect its broader scope covering all transfer types.

Error handling and retry behavior

The Data Fabric transfer type uses the same retry and error handling pattern as the existing IMS-based sending.

Behavior Detail
Immediate retry One (1) retry with a five-second delay on failure.
Site cooldown On persistent failure, the site is skipped for a configurable period. Default value is 5 minutes, controlled by DataFabricSkipPinMinutes.
Unaccepted documents Documents rejected by the API are skipped for 15 minutes before retry.
Error logging Errors are recorded in the Replication Messaging Errors form.
Batch reset Failed messages are freed and re-queued for retry, the same as the IMS logic.

Compatibility notes

  • Existing ION Messaging and ION Data Lake Flow configurations are unaffected by this feature. No behavioral changes occur for sites using transfer types 1 or 2.
  • Multiple transfer types per table - if a table has replication rules pointing to sites with different transfer types; for example: one using ION Messaging and another using Data Fabric, both are processed independently. To use only one, disable the unwanted replication rule.
  • No priority between transfer types - the system does not prefer on transfer type over another. All active rules are processed.
  • Data Fabric Streaming API is not supported - only Batch Ingestion is used due to the current architecture design.

Differences from legacy ION Data Lake Flows Setup

Aspect ION Data Lake Flows (Legacy) Data Fabric Batch Ingestion
Requires ION Desk flow configuration Yes No
Requires ION Desk connection point Yes No
Requires service account CSV upload Yes No
Transfer Type setting ION Messaging or ION Data Lake Flows Data Fabric
Schema generation Through ION Desk document refresh or Mongoose form Through Mongoose Generate Data Lake Schema form
Error logging ION Messaging Errors form Replication Messaging Errors form
Authentication ION API OAuth + Service Account OAuth 1.0 through datafabricoauth token service
Data Catalog URL field Not applicable Not required