Creating a workflow

You can create a workflow to update the model.

Before creating a workflow, make sure that you have service account credentials that will be applied in the context of task executions.

To acquire the service account, go to User management > Manage > Service accounts and download the service account credentials.

  1. In ION, select Monitors & Workflows > Workflows.
  2. On the Overview page, click Add.
  3. Specify a name.
  4. Optionally, specify a description.
  5. Add a Coleman Task element to the workflow diagram at the point where a task should be executed. You can apply one or multiple Coleman Tasks into the flow.
  6. Configure the Coleman Task parameters by selecting one of these task actions from the menu:
    Option Description
    Reload Datasets This action reloads data from Data Lake.
    Retrain Model This action runs the training quest.
    Update Production Quest This action updates the model from training to production and runs the production quest.
    Redeploy Endpoint This action deploys the endpoint with the latest model.
  7. Select the quest by name from the menu or specify a workflow parameter that contains the quest name.
  8. Specify the API Gateway service account for the task execution by clicking Import and selecting the service account credentials file.
  9. Click OK.
  10. Click the Test tab to execute a test which validates the configuration for the Coleman Task, the quest name, and the service account.
  11. Click the Settings tab and specify how to handle errors.
    Option Description
    The workflow continues Allows the workflow to continue to the next step in case of an execution error.
    The workflow fails Allows the workflow instance to fail in case of an execution error. This option is the one that makes sense for Coleman AI Platform.
  12. For each Coleman Task, you must add a new step to the workflow. Repeat the process as necessary for every Coleman Task.
    To have a complete orchestration of automatic model retraining, use all four task types for the same quest, in the same order as they are listed:
    • Reload Datasets to acquire the latest data from Data Lake.
    • Apply Retrain Model to rerun the training quest including the latest data.
    • Update the production quest.
    • Redeploy the endpoint.

    However, keep in mind the workflow modeler allows you to use any number of Coleman Tasks. It does not validate the sequence of processes, so be attentive with step ordering and configuration. Make sure you apply the logic that is required for your use case.

  13. Save and activate the workflow.