Using Agent Trajectory in Evaluations

You can use agent trajectory as a scoring metric in Evaluations.

  1. Select Factory > Evaluation > Add.
  2. Select GenAI Assistant as the evaluation type.
  3. In the Evaluation parameters window, enable Agent trajectory for comparison against expected behavior.
  4. Click Generate evaluation trace to run the evaluation trace with agent trajectory.

    The system returns the Evaluation list.

    The evaluation status updates to In progress.

    After processing, the status updates to Ready for review or Failed.

  5. Select the evaluation and click Agent trajectory trace in Evaluation parameters.
  6. Click Save to finalize the evaluation.
    The status updates to Final.
  7. Run the evaluation job to compare actual and reference agent trajectories.
  8. Review the results after the evaluation job finishes.
    Note: The system runs only the evaluations that have a Final status.