Marking or clearing data as corrupt

Marking a data object as corrupt is to prevent further processing of the data object in Compass and the dataobjects API.

When an object is set as corrupt:

  • Compass will not query the data object’s data.
  • The /dataobjects/byfilter API method does not return the corrupted data objects, even if they match the filter query. This default behavior can be overridden with the request parameters.

To mark or clear data objects as corrupt:

  1. Go to Data Lake > Atlas.
  2. In the left panel, select an object from the list.
  3. In the data grid, select the check box for one or more data objects to show the data grid toolbar.
  4. Click the Flag icon to mark the selected data object as corrupted.
    When the selected data objects are already marked as corrupted this action clears the mark.
  5. Click Yes when the confirmation dialog box is displayed with the selection details.
    Data objects that are marked as corrupted are indicated with an icon in the Corrupt column.

    Alternatively,you can also mark a data object as corrupt while viewing its details in Atlas.

    1. In Atlas, click the Details icon for an object in the data grid.
    2. When the panel on the right opens, click Mark Corrupt.
      If the object is already marked as corrupt, then the button indicates Mark Uncorrupt. When clicking Mark Uncorrupt the corrupt flag for the data object is removed.

    The Data Lake endpoint also provides a method to mark or clear a data object as corrupt.

    For more information on how to use this API, see the Swagger documentation for the /dataob jects/markcorrupt method in the Data Lake API endpoint.