Automatically appending files to S3 data sources

The Auto Append feature allows you to automatically append files based on search criteria. When the data is extracted in to Birst, the files are downloaded and any new files or folders matching the criteria are appended.

Auto Append is useful when you expect new files to be periodically dropped into your S3 bucket and would like to keep combining them in your Birst data source.

For example, your organization creates a daily sales report: "Sales_10292019.csv", "Sales_10302019.csv", and so on. in the StoresSales bucket. In Birst, you could create a "DailySales" source that effectively combines all .csv files in the "StoresSales" bucket that start with "Sales_".

  1. From the Edit objects in S3 Connector page, click Append Files.
  2. Select a bucket from the list of buckets that you have access to in S3.
  3. Specify the name in the Append Files field.
    Note: This name cannot be changed after downloading.
  4. Enable Auto Append.


  5. Specify your search criteria. This appends all files of a specified file type whose file name contains the specified criteria text. If results include more than one valid file type, you can specify your file type using the CSV or TXT buttons.


  6. After specifying your auto append criteria, select Download.