Changing Source Properties

From Source Properties, you can define the source's properties:

Source Data Change Behavior

This section affects whether and how Birst reacts to changes in the source data during subsequent processing. Select one of these options from the list menu:

Do Not Detect Source Data Changes on Upload

Any data source metadata changes (datatypes, widths etc.) are not detected. This option is only recommended when it is known that the data source structure does not change. It is expected that the data source column data type and length must be configured to correctly reflect the column data type and length as in source system. Additionally, other source data change options such as locking are not compatible with this property selected.

Automatic

Birst automatically makes changes when there have been changes in the source data since the last time you uploaded the data. Birst adds and removes columns and makes other changes as necessary when changes exist in subsequent loads of the source data. This setting is the default.

Locked

Locks the source data properties after the first upload so additional changes cannot be made. If Locked is selected and the source data changes in a subsequent upload (such as new columns, missing columns, data types), an error is displayed when the data is processed.

Custom

You can choose how to handle specific types of changes in the source data. When you select Custom, the fields below become active so you can choose which types of changes to allow. If you select Custom, choose these settings:

Allow column additions

When Source Data Change Behavior is set to Custom, by default it allows column additions. If a new column appears in the source, uploading the file with the new addition will succeed. If you do not want Birst to automatically add new columns, deselect this option. By default, the newly added column will be targeted to the most common target, that is, the hierarchy that is most commonly used for that source. To change targeting behavior for new columns click Advanced Options in the Properties tab and select an option for Newly Detected Source Columns.

Allow type upcasting

Select if you want Birst to automatically convert columns to a different data type (e.g., Integer, Varchar, DateTime, Float) based on the kind of data in the column in subsequent uploads of the source data. If this box is not checked, you will receive an error if a column contains data that does not match its data type when you process the data.

Note: The data type can be locked for individual columns by selecting Lock Type in the Column Properties.

Additionally, you can select:

Maintain placeholder columns for missing pre-existing columns

Select if you would like to keep placeholders for columns that used to be present in the source but are missing from a subsequent upload.

Remove missing pre-existing columns

Select if you would like to remove pre-existing columns that do not have values.

Publication Directives

This section determines how Birst manages the publication process:

Time Zone

Select the desired processing time zone for publishing the source data.

Replacement

Click inside this field to open a dialog to specify replacement values.

Bad Record Behavior

Select Replace with Null or select Skip Record in the event that the record is not valid.

Do not publish when used by another script

Select this check box to indicate that the source data is not to be published when it is used by another script.