Custom objects of type DSV

When you register a DSV object schema in the Data Catalog, the schema is assigned a subtype.

Based on the separator value that is provided in the schema, one of these subtypes is assigned:

  • CSV: Comma-separated values
  • TSV: Tab-separated values
  • PSV: Pipe-separated values
  • Other: Any separator that is not a comma, tab, or pipe.

A custom object of type DSV is defined by its name and a schema file.

To create an object of type DSV, prepare this folder structure for the import:

FOLDER: DSV
+--FOLDER: <object name>
+----FILE: <object name>.schema.json
+----FILE: <object name>.properties.json

These validations are performed when importing:

  • Object names must be unique and must comply to the object naming conventions.
  • If an object with an identical name is already registered with another type, the import fails.
  • If an object with an identical name is already registered with type DSV, the imported schema overwrites the existing schema.
  • The <object name>.schema.json schema file is validated in this way:
    • Only one DSV schema can be used for a given object name.
    • The schema file must use UTF-8 encoding.
    • The schema file must contain a dialect property. This property defines the format of the delimited data object.

      The dialect property must contain a separator property, which defines the character that separates the values in a row within a data object.

    • The properties element in the schema file lists the fields, in order, that are expected to be in a DSV data object. This section of the schema must be a valid JSON according to the schema definition, version draft-06.
    • The anyOf keyword is not allowed.
    • The oneOf keyword is supported only to describe a type that can be null.

The <object name>.properties.json file is optional and can contain additional metadata properties for an object.

For information on how to define the <object name>.properties.json file, see Defining additional object metadata properties and Additional properties file.

This code is an example of a DSV schema definition:

{
   "title": "myDelimitedFile",
   "description": "DSV Schema for myDelimitedFile",
   "dialect": {
      "separator": ",",
      "skipLines": 1,
      “headerLine”: 1,
      "enclosingCharacter": "\""
   },
   "properties": {
      "Code": {
         "description": "Customer code",
         "type": "string",
         "maxLength": 10
      },
      "Customer": {
         "description": "Customer name",
         "type": "string",
         "maxLength": 100
      },
      "PubDatetime": {
         "description": "Publication timestamp",
         "type": "string",
         "format": "date"
      }
    }
  }