Configuring Compass query processing modes

On the Mode Configuration panel, you can control how to process Data Lake objects during query run by selecting either the transactional or analytical mode. The selected mode determines how data is converted and prepared in Compass before a query runs.

For details on the effect of each mode on data conversion, deduplication, and query behavior, see Query processing.

Typically, you enable the analytical mode when record-level deduplication is required or when large volumes of new data are ingested in between query runs. In the transactional mode, minimal processing is performed during conversion and no deduplication is applied.

When you enable the analytical mode, deduplication is applied during data conversion in Compass based on the object schema definition in Data Catalog, including the primary key and variation ID. For each primary key, only one record per variation ID is returned in the query results. If required, multiple records for the primary key can be returned through SQL expressions or functions.

Mode Configuration panel

The Mode Configuration panel contains these columns:

  • Object Name

    A list of objects that are registered in Data Catalog and are available for querying in Compass.

  • Use Analytical Mode

    Controls which query processing mode is used for an object. When the switch is off, the transactional mode is used. When the switch is on, the analytical mode is used.

  • Active Mode

    Shows the mode that is currently used for each object.

  • Status

    Shows the migration status for each object.

Note: To configure the mode, you must have a Data Fabric security role that grants access to the Mode Configuration feature.

Enabling the analytical mode

  1. Select Compass.
  2. Click the Settings icon in the query editor toolbar.
  3. In the Mode Configuration panel, enable the Use Analytical Mode option for one or more objects.
  4. Click Save.

    After you click Save, the selected objects are migrated to the analytical mode. You can monitor the migration progress in the Status column. Click the Refresh icon to view the most up-to-date information.

    When the migration is completed, the Active Mode column is updated accordingly.

Disabling the analytical mode

If required, you can migrate objects from the analytical mode back to the transactional mode. The Active Mode column indicates the mode that is currently applied to run queries for each object.

  1. Select Compass.
  2. Click the Settings icon in the query editor toolbar.
  3. In the Mode Configuration panel, disable the Use Analytical Mode option for one or more objects.
  4. Click Save.