Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.coralos.ai/llms.txt

Use this file to discover all available pages before exploring further.

A definition for an LLM proxy provider. This type is only used in the llm-proxy.providers server config.
Configuration of LLM proxy providers can be skipped by providing a Coral Cloud API key.
Any agent that has defined LLM proxy requests requires the server to have a suitable LLM proxy provider configured for every request.

​
name

required The name of the LLM proxy provider. This must be unique. Application developers can request the usage of a specific LLM proxy provider by specifying this name in the proxies map.

​
format.type

required 
[[llm-proxy.providers]]
name = "My OpenAI Provider"
format.type = "OpenAI"
# ...
The LLM provider format. Current supported types are:
  • OpenAI
  • Anthropic

​
models

optional 
[[llm-proxy.providers]]
name = "My OpenAI Provider"
format.type = "OpenAI"
models = ["gpt-4.1-mini", "gpt-5-mini"]
allow_any_model = true
# ...
An array of models that this provider supports. This may be omitted if allow_any_model is set to true. Note that if this is empty, the default value of allow_any_model is true.

​
api_key

required 
[[llm-proxy.providers]]
name = "My OpenAI Provider"
format.type = "OpenAI"
api_key = "sk-..."
# ...
The API key to use for this provider.

​
base_url

required 
[[llm-proxy.providers]]
name = "My OpenAI Provider"
format.type = "OpenAI"
base_url = "https://api.openai.com/v1"
# ...
The base URL to use for this provider. Requests will be made to this URL with the path of the request appended. Example base URLs
ProviderBase URL
AI21 Labshttps://api.ai21.com/studio/v1
Aleph Alphahttps://api.aleph-alpha.com
Anthropichttps://api.anthropic.com/v1
Coherehttps://api.cohere.ai/v1
DeepSeekhttps://api.deepseek.com/v1
Fireworks AIhttps://api.fireworks.ai/inference/v1
Google Geminihttps://generativelanguage.googleapis.com/v1beta
Groqhttps://api.groq.com/openai/v1
Hugging Face Inferencehttps://api-inference.huggingface.co/models
Meta (Llama via API)https://api.llama.com/v1
Mistral AIhttps://api.mistral.ai/v1
Nvidia NIMhttps://integrate.api.nvidia.com/v1
OpenAIhttps://api.openai.com/v1
OpenRouterhttps://openrouter.ai/api/v1
Perplexity AIhttps://api.perplexity.ai
Replicatehttps://api.replicate.com/v1
Together AIhttps://api.together.xyz/v1
Voyage AI (embeddings)https://api.voyageai.com/v1
xAI (Grok)https://api.x.ai/v1

​
timeout

default 10s
optional 
[[llm-proxy.providers]]
name = "My OpenAI Provider"
format.type = "OpenAI"
timeout = "30s"
# ...
See here for more information on the duration format. A timeout for requests made to this provider. Slower providers may require a longer timeout.

​
allow_any_model

optional 
[[llm-proxy.providers]]
name = "My OpenAI Provider"
format.type = "OpenAI"
allow_any_model = true
# ...
If true, this provider will match a request for any model that matches the specified format. This is useful for defining fallback providers that can handle many models, e.g., OpenRouter. Resolution of AgentLlmProxyRequest will prioritize providers that have explicitly specified a model over providers that allow any model. The default value is false if models contains at least one model and true otherwise.