Passthrough LLM API endpoints

This section describes how to use the Prompt Passthrough endpoints in the GenAI application.

As part of the GenAI service, the Passthrough LLM API provides direct access to selected large language models. Before accessing the Passthrough endpoints, ensure you are assigned the following security role:

IONAPI-Admin
GenAI-PROMPTENGENEER

To access the Passthrough LLM API, navigate to API Gateway > APIs > GenAI Suite > GENAI/chatsvc Endpoint .

These are the most important endpoints for direct interaction with LLMs which we refer to as Passthrough LLM API Endpoints:

/api/v1/prompt for a single prompt to be submitted to the LLM
/api/v1/messages for submitting multiple messages with different outputs
/api/v1/prompt/stream for a single prompt to be submitted to the LLM for getting the response as a stream
/api/v1/messages/stream for submitting multiple messages with different outputs and getting the response as a stream

To use these APIs, you need to get the list of the available models, using this endpoint:

/api/v1/models for a list that provides details on all supported LLMs in the GenAI Assistant