Data Lake
Objects stored in the Data Lake are called data objects. For example:
- Sales order BOD
- JSON message sent from IoT or another application
- CSV file from a third-party integration
- Log file
- PDF file
These are the core aspects of Data Lake:
-
The actual storage of data objects.
-
Indexing, to make the contents accessible.
-
User interface to configure Data Lake and to manage the contents.
-
APIs or other interfaces to add, retrieve, or purge data.
-
Security, to ensure that you can only add or retrieve if you have sufficient permissions.
Additionally, Data Lake provides services to retrieve, massage, cleanse, join, or aggregate data.
These are the most important services:
-
Interfaces to ingest data, such as documents flowing through ION, an ingestion API, or a queue to upload data.
-
Interfaces to retrieve data, such as an API or a JDBC protocol you can use to query Data Lake.
Data Lake uses an object storage architecture to provide long-term persistence of data in its raw, original format. CloudSuite applications and 3rd party applications, tools, and services can replicate transactional and master data for eventual processing and query analysis.
Any type of content can be sent and stored in Data Lake, including this content:
- BODs (Business Object Documents).
- JSON.
- Delimited file formats such as CSV, TSV, and PSV.
- Binary file formats using the Data Catalog’s Any data format.
To ensure data security, all data objects that are stored in Data Lake are encrypted with AES-256 bit encryption. Data in-transit is secured by TLS version 1.2.
For data acquisition and ingestion that is facilitated by ION there are additional enrichment services. These include lifecycle management of data flows, scripting transformation capabilities with Python, and an end-to-end troubleshooting framework with ION OneView. With ION’s suite of connectivity and ingestion features, difficult-to-reach data and applications with limited data interfaces are integrated to ingest data with a variety of technology connectors.
For more details on these services, see the Infor ION Desk User Guide and Infor ION Technology Connectors Administration Guide.
Objects can be retrieved from Data Lake by Birst for Analytics strategy and development or with API methods exposed in the ION API Gateway.