Extracting Data using JSON
Prior to setting up data extraction using JSON familiarize yourself with the latest implementation notes at JSON Connector.
To extract data from JSON
| 1. | On the Home page select a space or create a new one. This is the space to which the data will be extracted. |
| 2. | Go to Admin - Define Sources - Application Connectors. Alternatively, click the Use Extractors link on the Admin Navigation page. |
| 3. | Click New in the Connections panel. |
| 4. | In Connection Details name the connection. |
| 5. | Select JSON as the Connector Type. |
| 6. | If you will upload a ZIP file containing multiple .json files, enter their common keyword in the JSON File Filter field. |
| 7. | Upload the
JSON file or ZIP file. |
| 8. | Optionally, specify a Source File Prefix. This alphanumeric string will be prepended to the data source names so that you can easily recognize them and differentiate objects that may have similar names from different data sources. For example, use the string "JSON" to easily identify JSON objects. |
| 9. | Optionally, select the Action on Failure. By default if there are errors the connector will stop processing. You can change it to Retry once, or to Ignore errors and continue processing. |
| 10. | Click Save
Connection. The Data Sources panel appears under the Connections panel.
This panel will list the extraction objects you define in the next
steps. Extraction objects help you organize what data is extracted for the connection. |
| 11. | Click Add Object. The dialog prompts you for information needed to create a new Data Sources extraction object. |
| 12. | Enter the Object
Label to use as the name in Birst.
Important: Do not use the same name as an existing saved object from this or any other connector. It must be unique within a Birst space. Using a source file prefix can help to differentiate similarly named objects. If you edit an object and change the object label, a new object will be created with this name. |
| 13. | Click Get
Columns. Birst populates the Columns list and Mapping field. The Columns list contains the objects in the source represented as a JSON hierarchy. The names are referred to as the raw name or "technical name". The Mapping field contains the name of each object in the source, an equal sign, then the name it will be mapped to in Birst. By default the Mapping field is called the display name or "friendly name". |
| 14. | Edit the Columns list to check only the objects you need for reporting. |
| 15. | Edit the Mapping field list to delete the objects you don't need. It should contains only the mappings for the objects checked in the Columns list. |
| 16. | Optionally,
change the name of the column in Birst
by changing the friendly name after the equal sign. Important: Do not modify the =true or =false texts. Tip: After the first time that Birst extracts the column and lists it in the Columns tab of the Manage Sources page, do not change the column name in the Mapping field again. If you do so, a new result will be created and the old one will remain listed as a placeholder. If your intention is to change the name for display purposes, instead you should use the Manage Sources - Columns tab. |
| 17. | If you are using multiple .json files, Select the Action on Failure. If there is a problem with a file, you can set whether the connector will ignore the failure and move on to the next file, retry once and then stop if the retry does not succeed, or just stop. |
| 18. | Click Validate to validate the syntax in Birst. |
| 19. | Click Save.
Birst adds it to the
list of objects to be extracted in the Data Sources table, ready for extraction. Tip: To edit the object, click on the hyperlinked name. |
| 20. | Optionally,
select the row and enter the name of one or more extract groups, separated
by commas, in the Extract Group Name(s) field. Extract groups make it
possible to extract specific objects and process them separately
from other objects. See Creating
Extract Groups. Important: Use the same extract group for Issue and its sub-objects. Click Save after adding extract group names. |
| 21. | Click Extract
to upload the selected objects to Birst.
The Extract Groups dialog prompts you to select the extract group or groups
to use. If you did not define extract groups it defaults to All. Selecting All in the Extract Group dialog box will extract all selected objects, including those that do not belong to extract groups. |
| 22. | Click Continue.
Birst displays an indicator
to show that the extraction is in progress. Depending on the size of the data objects, the extraction may take a few minutes. When the extraction completes Birst lists the date and time of the extraction in the Data Sources table of the Show Selected Objects view. |
Tips:
You can safely log out of Birst while the extraction takes place and log back in later.
To cancel the extraction, click the Cancel Extract link under the progress bar. If there are multiple files, the connector will finish processing the current file and then stop.
Click the Last Extract Log link to view the logs from the data extraction and check for errors.
To remove an object from the Data Sources list, select the row and click Remove on the right.
Next Steps
Look at the raw data in Manage Sources page. Iterate the extraction process, if needed, to get the data you expect.
Process the data in the Process Data - Process New Data page. See Processing Data.
Set up scheduling in the Process Data - Schedule Data Processing page. See Scheduling Extraction and Data Processing for Cloud Applications.