Analytical mode

In analytical mode, record-level deduplication is performed during data conversion. Newly-ingested records are evaluated to determine whether they represent new records or updates to existing records that have already been converted into Compass-formatted tables.

This evaluation is based on Data Catalog object metadata, including identifier paths and variation attributes. Deduplication logic ensures that query results present a consolidated view of data based on the defined object schema.

Conversion performance in analytical mode depends on the characteristics of incoming data. When ingestion consists primarily of new records, for example, during initial loads or through large batch ingestions between query runs, conversion is efficient. This is because processing is optimized for bulk data handling and can outperform transactional mode.

When ingestion consists primarily of frequent updates to existing records, incoming records are repeatedly compared against existing converted data. The deduplication checks increase conversion overhead, particularly when queries are run frequently.

Analytical mode is most effective when deduplication is required and large volumes of new data are ingested between query runs.

Tip: Before you select analytical mode, we recommend that you consider data characteristics and test analytical mode on representative objects.