Using Compass query APIs

Use Compass APIs to query and retrieve data that is stored in Data Lake.

Overview

Based on your requirements, select one of these Compass APIs:

  • Asynchronous API: Used to submit queries as jobs and retrieve results after query completion. This API is recommended for most Data Lake query and extraction scenarios, including large result sets, long-running queries, scheduled integrations, and data pipelines.
  • Synchronous API: Used to submit a query and return results directly in the HTTP response. This API is suitable for applications that require immediate results and when asynchronous processing, polling, and result retrieval add unnecessary complexity.

This table shows the comparison between asynchronous and synchronous APIs for different aspects:

Aspect Asynchronous API Synchronous API
Interaction Three-step interaction: POST → poll STATUS → GET RESULT Single request → response
Endpoint /compass/v2/jobs /compass/v2/statement/{category}
Content-type text/plain application/json
Result delivery Polls until query completion, then retrieves results. Streamed directly in the HTTP response body.
Timeout Managed by Status/Polling/Expiry Client-defined. 120 seconds by default. If exceeded, a query is terminated by the server.
Pagination Offset-based on RESULT call. None. Full result streamed.
Best-fit scenarios Data Lake extractions, large result sets, long-running queries, scheduled integrations, orchestrated pipelines Applications that require immediate responses, request-response integrations, scenarios in which implementing asynchronous processing adds unnecessary complexity
Note: The synchronous API is a simplified consumption model and doesn't replace the asynchronous API.

Versions

Compass APIs are available through the Infor Data Fabric API Suite.

The legacy Compass APIs from the ION API Suite are deprecated. Use the Compass APIs from the Infor Data Fabric API Suite for new implementations and integration updates.

Compass APIs are not available in AWS GovCloud.

For information on how to use API Gateway and how to interact with Swagger documentation for the API methods, see ION documentation.

For more details on Data Lake Compass and querying data objects, see Querying the Data Lake.

Asynchronous API

The content-type HTTP header is required for the client software that uses Compass V2 APIs. The content-type must be set to text/plain.

To run a query, you can use these methods:

  • Post a Compass job to receive a unique Query ID for the job.
  • Use the Compass status method and Query ID for the job to retrieve its status.
  • Use the Compass results method and the Query ID for the job to retrieve the results of a finished job. The result API has these parameters for pagination:
    • Pagination
    • Limit
    • Offset

Managing the retrieval of result sets

Select an optimal size to download query results based on hardware constraints and network timeouts. This allows also the re-retrieval of a portion of the result in the event of a connection failure. The result of a query is static. The result does not change when data is added or updated in Data Lake. The page size is based on rows, with a maximum limit of 100,000 rows with a 10 MB size limit.

Offset and Limit parameters are used for retrieved result sets. Query results are available for roughly 20 hours after receiving the FINISHED status. The query is associated with the method used to run the query.

You cannot run a query in the Compass user interface and then check the status or retrieve the results through the API. The query is available only to the tenant and principal ID that submitted the job. If you run a query, someone else cannot query for the status or retrieve results.

This table shows the imposed quotas per method per tenant:

Method Calls
/compass/v2/jobs Up to 100 calls per minute
/compass/v2/status Up to 1000 calls per minute
/compass/v2/result Up to 10000 calls per minute
/compass/v2/cancel Up to 1000 calls per minute

Interface and consumption methods are exposed through the Compass API Service registered within the API Gateway suite for Infor Data Fabric.

For more information on how to use API Gateway and how to interact with Swagger documentation for the API methods, see API Gateway documentation.

Synchronous API

These high-level steps describe how the synchronous API processes a query and returns the result set:

  1. Send a single POST request to /DATAFABRIC/compass/v2/statement/{category} with the SQL statement and optional parameters in the JSON body.
    Note: Specify dataLake as the category. The category is provided to support future enhancements.
  2. The request is authenticated with OAuth 2.0, validated, routed to the appropriate data store, and the query is processed.
  3. The result set is returned in the same HTTP response.
  4. Receive the complete result set in the format that depends on one of these Accept headers:
    • JSON
    • NDJSON
    • CSV

If a query exceeds the timeout (120 seconds by default), the query processing is terminated and an error response is returned. No partial results are delivered.

Idempotency

Query results are transient. Each query submission is processed as a new query, including identical queries.

Request body

Only application/json is supported. You must provide all inputs in a structured JSON payload.

text/plain input is not supported for the synchronous API.

Output formats (Accept header)

These output formats are supported:

  • application/json
  • application/x-ndjson
  • text/csv

JSON output can produce very large result sets, depending on the query.

Use NDJSON or CSV formats when large JSON files cannot be processed efficiently.

Metadata in responses

Metadata is available only in application/json responses. Metadata and result data are returned in a single JSON document.

Metadata is not supported in NDJSON and CSV responses.

Detailed column metadata is available only in application/json responses. CSV and NDJSON responses include column names but not detailed column metadata. In CSV responses, column names are included in headers.

Result set metadata includes query IDs and error details.

For Data Lake queries, successful responses include a query ID but not the row count.

Error responses include a query ID and exception message for Data Lake queries.

Compression / Content negotiation

These Accept-Encoding encodings are supported:

  • gzip
  • identity: Default fallback when no or invalid encoding is specified.
  • deflate: Only the standard variant is supported, not the Microsoft variant.

For details, see the Swagger documentation.