Creating evaluations

An evaluation can have multiple owners. When you create an evaluation, the system automatically assigns you as the first owner. You can add additional owners to run evaluations, create or manage evaluation schedules, and view sensitive execution data such as Ground Truth, Model Generated Output, and Agent Trajectory data.

Note: Non-owners can view general evaluation information but cannot run evaluations or access sensitive details.

Select Infor OS > GenAI > Factory > Evaluation.
Click + Add.
Specify a name for the evaluation.
Select the endpoint where the evaluation applies:
- GenAI Assistant
- Embedded Experience
Under Evaluation Parameters, click + Add and specify these:
- Input: The user prompt to evaluate.
- Ground Truth: The expected response from the GenAI Assistant.
- Judge Model: Model selector for defining model used for evaluation job.
- Agent Trajectory Ground Truth: Toggle for enabling Agent and Tool efficiency metric.
Enable any relevant Scoring toggles:
- Answer Correctness: The factual accuracy of the generated output, independent of the defined ground truth.
- Answer Similarity: The degree of alignment between the generated output and the defined ground truth.
- Answer Relevance: The measure of how appropriate and useful the generated output is with respect to the user’s intent.
- Agent Trajectory: The measure of how efficiently an Agent or Tool was used.
Click Add at the bottom of the Add Evaluation Parameter window to save.
If you enabled Agent Trajectory, click the Generate evaluation trace button to generate the trajectory.

After you click the Generate evaluation trace button, the overview screen is displayed. The value in the Status field shows the state of trajectory generation:
- In progress: the trajectory is being generated.
- Ready for review: the trajectory can be reviewed and edited.
- Final: the reviewer approved the trajectory.
Note: If you did not select Agent trajectory ground truth, the status of the evaluation is Final after you save it.