Document support for LLM passthrough endpoints

Users can upload a document in a prompt to enable a Large Language Model (LLM) to respond based on the document’s content.

Supported file types include: .doc, .docx, .csv, .xls, .xlsx, .html, .txt, .pdf, and .md.

1. To include a document in a prompt, convert the local file to a Base64 string.

2. Then, provide the input within the LLM Passthrough API endpoint payloads:

   • For api/v1/messages and /api/va/messages/stream endpoints, include these fields under the data if the type is "document":

     • "format": Refers to the document extension.
     • "name": Refers to the document name: Use only alphanumeric characters, whitespace, hyphens, parentheses, and square brackets.
     • "data": Refers to document data. Provide the Base64-encoded string.
     Note: The file size limit is 4.5 MB.
     For example:

     "messages": [
          {
             "role": "user",
     	 "content": [
     	   {
     		"type": "document",
     		"data": {
     		   "format": "pdf",
     		   "name": "FileNameNoExtension",
     		   "data": "base64encoded string of document"
     	    }
     
     

   • For api/v1/prompt endpoint, include these fields as part of the "document" object:

     • "format": Refers to document format. Use only these file types: doc, docx, csv, xls, xlsx, html, txt, pdf, md.

     • "name": Refers to document name. Use only alphanumeric characters, whitespace, hyphens, parentheses, and square brackets..

     • "data": Refers to document data. Provide the Base64-encoded string.

     For example:

     "document": {
             "format": "pdf",
             "name": "FileNameNoExtension",
             "data": "base64 encoded string of document"
       }
     }
     

     Note: In the prompt, set a clear goal for how the model should use the document.
     Note: The file size limit is 4.5 MB.

{
  "system": "You are a math professor who helps answer math questions",
  "model": "CLAUDE",
  "version": "claude-3-7-sonnet-20250219-v1:0",
  "config": {
    "max_response": 8192,
    "reasoning": {
	"enabled": true,
	"budget_tokens": 4096
    }
  },
  "messages": [
     {
        "role": "user",
	 "content": [
	   {
		"type": "document",
		"data": {
		   "format": "pdf",
		   "name": "FileNameNoExtension",
		   "data": "base64encoded string of document"
	    }
	},
	{
	   "type": "text",
	   "data": "Do something with this document"
	}
      ]
    }
  ]
}

{
  "model": "CLAUDE",
  "version": "claude-3-7-sonnet-20250219-v1:0",
  "useConverseApi": true,
  "config": {
    "max_response": 16000,
    "reasoning": {
        "enabled": true,
        "budget_tokens": 8000
    }
  },  
  "prompt": "Summarize the document",
  "document": {
        "format": "pdf",
        "name": "FileNameNoExtension",
        "data": "base64 encoded string of document"
  }
}