Send OneView events to Data Fabric

You can choose to send both (live/archived) connect/process data toData Fabric . See OneView data archiving. This data can be sent to Data Fabric:

  • All events from OneView, including those in the EventGroup.DOCUMENT_TRACKING, as well as process-related events from EventGroup.WORKFLOW and EventGroup.PULSE, can be sent to Data Fabric.
  • If you enable sending data to Data Fabric, payloads are not sent to Data Fabric. If needed, you can create a Data Lake flow to include the payloads in Data Fabric.
Note: 
  • No process-related events have been archived so far; as process-related events have been present in the Elasticsearch cluster since the beginning. So, if you enable sending Process Data to Data Fabric, all the events that are present in the cluster are sent to Data Fabric.
  • For non-production users, when a user has not opted in, data is purged when the retention period is completed. For users who have opted in, it is also ensured that data is purged only after it is sent to Data Fabric.
  • For production users who opted in, data is sent to the Data Fabric, and the current archiving process will not happen. If the user has not opted to send to the Data Fabric, then data is archived after the retention period is completed.

Ingesting Data into Data Fabric

The Batch Ingestion API is utilized to send live and archived OneView data to Data Fabric. This feature for sending OneView data to Data Fabric is exclusively available for AWS Commercial, as the Batch Ingestion API is only available in that environment.

Schema Maintenance

These are some important points:

  • Data Catalog schemas are created once, and users must maintain them.
  • If schemas are deleted unknowingly, the Batch Ingestion API and archiving process will not complain or be aware. The archiving process will continue pushing data to Data Fabric, but users cannot search that data in Compass until the schema is recreated. Data is not lost, but search functionality is impaired until schemas are recreated.

Dedicated Object Schemas

The ION_OneViewEvent and ION_OneViewEventProperties object schemas must be dedicated to OneView Data Archiving. If a user has their schema with these names, the archiving data will not map to that schema, resulting in improper search results in the Compass interface.