Table of Contents

πŸ‡«πŸ‡· Version franΓ§aise

LLM Providers

The providers on this page serve the Generation category. Embedding, reranking and decision providers (TypeSafe, Liquid, Kev) are listed in Model providers; a decision model judges an LLM call through a decision profile (Decision models).

HttpLlmProviderBase (Orkeon.Infrastructure.LLMs.Base) provides the abstract base for all LLM providers. It integrates HTTP handling (IHttpClientFactory), the Polly retry policies, and JSON serialization.

Retry budget. Llm:MaxRetries (10 by default) covers the failures that come back in seconds β€” request errors, 5xx, 429 (except an exhausted balance, below). A buffered call that hits Llm:TimeoutSeconds is retried once (ResilienceDefaults.LlmTimeoutRetries: every attempt costs the whole timeout), then the provider answers with a failure naming the setting and the two ways out (a longer timeout, thinking off); a caller's cancellation is never retried. A failed call travels as LlmResponse.Error and, through the chat-client adapter, as an exception β€” it is never mistaken for an empty answer, so the agent loop fails the task at once with the provider's reason instead of retrying without tools.

Rate-limit window. A 429, or a 529 that carries a reset header, is not retried on the backoff ladder: the call waits until the window resets, on the buffered path and on the streaming connect alike (RateLimitResetHeaders). It reads Retry-After (seconds or an HTTP date), retry-after-ms (milliseconds, sent by OpenAI and TypeSafe), anthropic-ratelimit-requests-reset and anthropic-ratelimit-tokens-reset (RFC 3339) and x-ratelimit-reset-requests and x-ratelimit-reset-tokens (OpenAI durations such as 6m0s). retry-after-ms, when sent, is the more precise of the two Retry-After headers and replaces it; the longest of that value and the reset headers then wins. A finite value beyond the window fails the call with LlmRateLimitException, never with an arithmetic overflow. The wait never exceeds Llm:MaxRateLimitWait (5 minutes by default, separate from the 30-second cap of the 5xx ladder): a reset beyond it fails the call at once with the reason (LlmRateLimitException) instead of parking an unattended run. Each wait is announced to ILlmRetryObserver with the reason rate-limit window and the delay. A 529 without a header keeps the ladder.

Exhausted balance. A 429 whose body names a spent balance or quota β€” OpenAI's insufficient_quota, Kimi's exceeded_current_quota_error, Z.AI's Insufficient balance β€” is not a rate limit: no wait clears it. It is not retried, neither on the ladder nor on a window, on the buffered path and on the streaming connect alike (ResiliencePolicies, LlmFailureClassifier.IsBalanceExhausted), and the error line adds (account balance exhausted: ...) after the vendor's words. It still classes as rate_limited, so a profile's fallback chain moves on to the next provider, and the decision model reports it as not transient. When the body cannot be read, the 429 is treated as a rate limit. Out of scope: a balance refused with another status (Anthropic's 400 "credit balance is too low", a 402 from DeepSeek, MiniMax or OpenRouter) is not retried either, but classes as refused and does not fail over; the code-graph embedding provider (Orkeon.Analysis) has its own retry loop and still retries insufficient_quota (issue #328).

A streamed call is bounded the same way. A streamed request is sent with ResponseHeadersRead, and HttpClient.Timeout stops counting once the headers are in, so every streaming path (the OpenAI dialect, Azure, Anthropic, Ollama β€” chat and text streams) runs under a StreamReadBudget: Llm:TimeoutSeconds bounds the whole call, headers and body, and the optional Llm:StreamIdleSeconds bounds the silence between two lines of the body (unset by default). An elapsed bound ends the chat stream with a Completed event whose response carries error (the sentence naming the setting) and error_type (TaskCanceledException for the whole-call bound, StreamIdleTimeout for the idle one) over whatever content had arrived, with no RawResponseBody; the text stream throws an HttpRequestException with the same sentence. A stream the provider closes without its finish marker (finish_reason or [DONE], message_stop, done: true) is a failed call too when no answer text and no tool call arrived (error_type StreamTruncated, the sentence saying how much reasoning was streamed); with some answer it is served as it came, flagged stream_truncated in its metadata. The chat-client adapter fails a streamed turn on any error, partial content or not; the headers phase re-sends an elapsed HttpClient.Timeout once (LlmTimeoutRetries), like the buffered path. Every provider records the vendor's own stop reason, raw, under finish_reason in the response metadata (length for the OpenAI dialect and Ollama's done_reason, max_tokens for Anthropic's stop_reason), buffered or streamed; Azure's string-only stream aside. A script reads it to tell an exhausted output budget from an empty answer.

Thinking is replayed unchanged (Anthropic). The assistant turn of a tool call carries its thinking and redacted_thinking blocks (text, signature, data) as the provider sent them, buffered or streamed, in LlmMessage.RawThinkingBlocks; the next request re-emits them first, before the turn's text and tool_use blocks, which the API requires for extended thinking with tools. Other providers ignore them. A tool call without id, name or an object arguments is refused with an error that names its index, its name and the field.

Implemented providers:

Provider Class Namespace
OpenAI OpenAILlmProvider (via OpenAICompatibleProviderBase) Orkeon.Infrastructure.LLMs
Anthropic AnthropicLlmProvider Orkeon.Infrastructure.LLMs
Azure OpenAI AzureOpenAILlmProvider Orkeon.Infrastructure.LLMs
Ollama OllamaLlmProvider Orkeon.Infrastructure.LLMs
Together AI TogetherAiLlmProvider Orkeon.Infrastructure.LLMs
DeepSeek DeepSeekLlmProvider Orkeon.Infrastructure.LLMs
HuggingFace HuggingFaceLlmProvider Orkeon.Infrastructure.LLMs
Kimi KimiLlmProvider Orkeon.Infrastructure.LLMs
Qwen QwenLlmProvider Orkeon.Infrastructure.LLMs
Mistral AI MistralLlmProvider Orkeon.Infrastructure.LLMs
Z.AI (Zhipu GLM) ZaiLlmProvider Orkeon.Infrastructure.LLMs
Google Gemini GeminiLlmProvider Orkeon.Infrastructure.LLMs
Grok (x.AI) GrokLlmProvider Orkeon.Infrastructure.LLMs
MiniMax MiniMaxLlmProvider Orkeon.Infrastructure.LLMs
OpenRouter (aggregator) OpenRouterLlmProvider Orkeon.Infrastructure.LLMs
Mammouth AI (aggregator) MammouthLlmProvider Orkeon.Infrastructure.LLMs

Fourteen of them extend OpenAICompatibleProviderBase (itself an HttpLlmProviderBase); Anthropic and Ollama extend HttpLlmProviderBase directly, because their APIs are not OpenAI-shaped. Docker Model Runner and any other OpenAI-compatible server have no class of their own: they are driven by OpenAILlmProvider through their base URL. Endpoints, default models and provider keys are shared constants in Orkeon.Constants.Llm (LlmProviderEndpoints, LlmProviderDefaultModels, LlmProviderKeys, LlmModelOutputLimits).

Generic adapters (ChatClientToLlmProviderAdapter, LlmProviderToChatClientAdapter, ChatClientToBasicLlmProviderAdapter) are available in Orkeon.Infrastructure.LLMs.Adapters to integrate other providers compatible with the IChatClient interface. A Microsoft Agent Framework agent can also serve as a model (AIAgentLlmProvider, ADR-010).

Provider resolution

LlmProviderFactory (Orkeon.Infrastructure.LLMs) has two entry points:

  • Create(providerType, config) takes an explicit provider key (LlmProviderKeys): openai, anthropic, ollama, azure-openai / azure, together / togetherai, qwen, deepseek, kimi / moonshot, mistral, huggingface / hf, gemini / google, grok / xai, minimax, zai / glm / zhipu, openrouter, mammouth, and custom / docker-model-runner, which build the OpenAI dialect on the configured base URL. An unknown key throws NotSupportedException. This is the path of orkeon llm probe --provider.
  • Create(config) infers the provider β€” this is what a run does, since the Llm section of the settings file carries no provider key. The first rule that matches wins:
    1. With a base URL, the address alone decides (LlmEndpointReading, the one reading the run and the CLI share; the model and the key are not consulted):

      Address Provider
      a known vendor host β€” azure / .cognitiveservices., together.xyz, the DashScope hosts, deepseek.com, moonshot.cn / moonshot.ai, mistral.ai, generativelanguage.googleapis.com, api.x.ai, api.minimax.io / api.minimaxi.com, huggingface.co / hf.co, api.z.ai / bigmodel.cn, openrouter.ai, mammouth.ai, openai.com, anthropic.com that vendor
      /engines/ in the path, model-runner.docker.internal or port 12434 Docker Model Runner (OpenAI dialect)
      port 11434 Ollama, at its native root (a trailing /v1 or /api is dropped)
      anthropic in the host or in the path Anthropic
      any other address OpenAI-compatible server (custom), at the path written
      a vendor host under an /anthropic path, or an address that is not an absolute http(s) URL refused at start, with the setting's path and the fix

      A server is Ollama only on port 11434, wherever it runs. A custom address with no path (http://host:8000) is read as an OpenAI-compatible server answering at its root and is reported once per section (LlmEndpointNotice.Silent, a warning on stderr and in the log): add the path of its routes, often /v1.

    2. Model prefix (only without an address) β€” gpt, claude, llama / codellama (Ollama), mistral / ministral (the bare mistral and mistral:<tag> go to Ollama), qwen, deepseek, moonshot, glm, gemini, grok, minimax, openrouter/. Mammouth is never inferred from a model name: its identifiers are the vendors' own.

    3. API key prefix β€” hf_ β†’ HuggingFace, xai- β†’ Grok.

    4. Otherwise OpenAI.

A section that names no model skips rule 2. The model a call runs on is then resolved by the provider, the same way for all sixteen (HttpLlmProviderBase.ResolveModel): the call's own when its configuration names one, else the model the provider is configured with β€” its profile's β€”, else its DefaultModel (its LlmProviderDefaultModels entry; Azure has none of its own and falls back to OpenAI's as a deployment name, so name yours). A configuration that names no model (LlmConfig.OnProfile(), an empty Model) never reaches the wire empty, and never carries another vendor's model.

The configuration a call runs on follows the same rule, field by field (LlmConfig.InheritFrom, applied by HttpLlmProviderBase.EffectiveConfig in the sixteen providers): a configuration passed with a call completes the one the provider was built with, it does not replace it. Every field the call leaves unset β€” null, an empty or blank string, no stop sequence β€” is the provider's: the key, BaseUrl, TimeoutSeconds (nullable: unset everywhere, 30 s), Azure's ApiVersion, Anthropic's WorkspaceId, Thinking, MaxTokens, Temperature and TopP (nullable too), Seed, SystemMessage, ResponseFormat, Cache, Tools, the grammar and the model; custom parameters merge, the call's keys winning. Every field the call sets wins. The settings that cannot be unset β€” the penalties (0, every vendor's default) and the tool mode β€” are the caller's, and MaxRetries and Grammar are read from the provider's configuration when it is built. A call cannot unset what its provider sets; it overrides it (Thinking = { Enabled = false }, LlmResponseFormat.Text()). The chat client adapter registered without a base configuration starts from the provider's own (ILlmProvider.BaseConfig).

Sampling: what is set is sent, what is not set is not. Temperature and TopP are null when nothing sets them β€” not the call, not the agent, not the task, not the profile β€” and then nothing reaches the wire: the model applies its own default (often 1; the Modelfile's on Ollama). Set, they are sent whatever their value. A default model may refuse a sampling value it does not support (a pinned 0.7 on gpt-5.6-sol or claude-sonnet-5), which is why nothing is sent unless something sets it. One measured exception: Mistral's dialect writes top_p: 1 when nothing sets one (AlwaysEmitTopP). The penalties, the seed and the stop sequences of an agent's configuration travel through the chat client too; on the wire, Ollama writes top_p, seed and stop in its options, and a dialect that has no field for a penalty or a seed says so with the structured warning below, never in silence. A temperature, top_p or penalty is sent as written, up to six significant digits (the float of ChatOptions holds no more): 0.7, never 0.699999988079071, under any current culture.

Every provider the factory builds is wrapped in CallModeLlmProvider, then, by LlmProviderEntrance, in MeteredLlmProvider (always; it reports to the host's ILlmUsageSink when one is registered), TracedLlmProvider and, when the host registered an ILlmRateLimiter, RateLimitedLlmProvider (see Decorators and registration), and returned behind an LlmProviderAdapter.

Declared capabilities

Every provider declares a LlmProviderCapabilities value object (Domain, exposed on ILlmProvider; a provider overrides HttpLlmProviderBase.DeclaredCapabilities, which defaults to LlmProviderCapabilities.Unknown): ResponseFormat (None/JsonObject/JsonSchema), Thinking (None/EffortOnly/Toggle/Budget), EffortLevels (the effort levels the field knows: low, medium, high by default, up to xhigh for OpenAI and Grok, and the five for Z.ai, Anthropic, OpenRouter, DeepSeek, Kimi, Qwen, Together AI, Hugging Face, MiniMax and Mammouth AI; the per-model table below narrows them model by model): a level asked that is not declared is sent as the highest declared level below it, with a warning in the provider's log; a value outside the five, such as OpenAI's none, is no level and is sent unchanged for the vendor to judge), Vision, ExplicitPromptCaching, RequiresJsonKeywordInPrompt, ReplaysReasoningContent, RunsOwnTools (declared by the Microsoft Agent Framework bridge alone: it is an agent that runs its own tools). One capability is not the vendor's to declare: GbnfGrammar, which no vendor API documents, is switched on by the configuration the provider is built with (Llm:Grammar, LlmConfig.GrammarEnabled) for a llama.cpp-compatible server behind the OpenAI-compatible providers or Ollama β€” ILlmProvider.Capabilities is the declaration plus that switch. And one is no vendor's at all: ReplaysPrompt, declared by the echo provider alone (UndefinedLlmProvider, the model of a host without an Llm section), says that what it returns is its prompt, never a model's answer β€” the crew's planner then skips its call with a warning instead of reading the prompt back as a plan. The decorators a provider is wrapped in (MeteredLlmProvider, RateLimitedLlmProvider) pass the capabilities through. OpenAICompatibleProviderBase translates the declaration into the OpenAI dialect once (vision payloads, response_format, thinking, the CapabilityMismatchHint diagnostics); Anthropic and Ollama write their own dialects, and Qwen overrides the hook for DashScope's thinking fields. An option a provider cannot honour produces a structured warning (event id 110, Option '…' was declared but … does not support it β€” it was not sent) β€” never a silent drop; that includes frequency_penalty and presence_penalty on every dialect, and seed wherever the dialect does not write it (all but Ollama). CapabilityMismatchHint (OpenAI-compatible providers and Ollama) covers the per-model side: when a vendor refuses a capability the provider declares (a text-only model sent an image, a model without thinking or tools), the error says whose assumption was wrong. The mirror-image case β€” a constraint only the server can state β€” has its own seam: OpenAICompatibleProviderBase.TryAdaptRejectedPayload gives a provider one chance to adapt a payload the API rejected with a 4xx and re-send it once (generate and chat paths; streaming never retries). Kimi uses it for Moonshot's invalid temperature: only 1 is allowed for this model β€” the mandated value is read from the rejection itself (which models mandate it is decided server-side, a hard-coded list would drift) and the substitution is logged as a structured warning; only a temperature that was set and sent is replaced, since one nothing sets is not sent. Two more dialect seams serve the aggregators: ReasoningFieldName names the vendor field the reasoning trace is read from (reasoning_content by default, reasoning on OpenRouter β€” the Orkeon metadata key stays reasoning_content), and usage.cost becomes the cost metadata wherever a vendor bills in the response β€” buffered or streamed β€” with cost_currency beside it when the provider states its vendor's billing currency (CostCurrency: USD on OpenRouter, whose credits are dollars; no other provider states one). From there the charge travels as billed: the chat client adapter carries it onto the ChatResponse (AdditionalProperties, also on its streaming path), the agent loop reports it on the usage event (CostUsageEvent.Cost, null when the vendor billed nothing β€” a free model's 0 stays 0), the scripting facade does the same for ctx.llm.*, and the run's cost.updated relays it with costSource: "vendor" (the run event bus). CostBudgetManager prices from its registry only a call nobody priced, for its own budgets; that estimate never reaches the wire. A chunk carrying a root-level error after the HTTP 200 ends a stream the way a pre-stream refusal does β€” error metadata on the chat stream, an HttpRequestException on the token stream β€” never as a clean completion. All 16 providers are IStreamingLlmProviders. The chat client adapter's streaming path (GetStreamingResponseAsync) is built like its buffered one β€” the same messages, roles, tools and options β€” and reads the provider's chat stream (ChatStreamingAsync): the text as it arrives, then one last update carrying the tool calls (native or of the text protocol), the provider's usage, the finish reason, the model, the vendor's cost and the reasoning to replay. Folded, the updates are the response the buffered call returns, so a streamed agent turn calls its tools and is metered exactly, never estimated; a provider that does not stream answers it buffered. A tool call is replayed to the provider with every property it carried, buffered or streamed, each call with its own: the adapter keeps the provider's tool_calls[i] element on its FunctionCallContent and replays it whole, and the merge of streamed fragments keeps what a fragment adds beyond index, id, type and function β€” Gemini's thought signature (extra_content.google.thought_signature), which Gemini requires back, travels that way; a call that carried nothing else is replayed as id, type and function. A stream says what the buffered answer says: Anthropic assembles the tool_use blocks it streams; MiniMax splits its leading <think> block out of the content deltas (LeadingReasoningTag), so the text a stream carries is the answer its final response keeps; and on the native protocol, the OpenAI dialect and Anthropic give a streamed answer the body their buffered answer carries, in which the text-protocol fallback reads a call the model wrote as text. The per-provider matrix lives in the provider comparison.

Tool-call deltas. The chat stream tells its caller about a tool call while the model builds it: LlmStreamEventKind.ToolCallDelta carries ToolCallDelta(Index, Id, Name, ArgumentsDelta), and the terminal event still holds the whole call, unchanged. The OpenAI dialect (the fourteen providers that derive from OpenAICompatibleProviderBase, Azure OpenAI included) emits one delta per delta.tool_calls fragment; Anthropic one when a tool_use block opens (id and name) and one per input_json_delta; Ollama one complete delta per call, since it sends calls whole. None of the 16 providers is mute: a test checks that the three families cover them, so a provider added later must join one of them or say that it cannot (a structured warning, never a silent drop). The chat client adapter passes the fragments on as updates that carry no content, so the fold of the turn is the one it was; ILlmDeltaSink.OnToolCallDelta receives them (empty by default) and they ride the llm.delta event (deltaKind: "tool_call"). Reasoning deltas are not forwarded to the sinks. With a DLP interceptor of the external-output channel registered (AddOrkeonDlp), the arguments of each call are held by index until the stream ends, scanned whole and emitted as one delta, so a secret cut between two fragments is redacted; without one they stream as they arrive. A fallback chain commits to a provider at its first event, a tool-call delta included: the partial deltas of a provider that failed before any event never reach the caller, and one that fails after reaching it keeps the call, its failure ending the stream.

Dialect seams of the OpenAI-compatible base

What differs between the compatible vendors is expressed through a handful of protected members, not through copies of the payload builder:

Seam Default Overridden by
ApplyProviderSpecificOptions judges the thinking options and writes response_format from the declared capabilities Together AI (adds context_length_exceeded_behavior: truncate); Kimi (removes temperature and top_p on the kimi-k3 family, with a structured warning)
WriteReasoningFields writes the judged thinking controls (switch, effort, budget); the default writes thinking: {type} and reasoning_effort Qwen (enable_thinking, thinking_budget), OpenRouter (the reasoning request object), Together AI (reasoning: {enabled}), Hugging Face (chat_template_kwargs.enable_thinking), MiniMax (thinking: disabled only)
MaxTokensFieldName max_tokens OpenAI (max_completion_tokens)
AlwaysEmitTopP top_p written only when set Mistral (top_p: 1 when nothing sets one)
SplitReasoningFromContent / EnrichAssistantMessage nothing split; reasoning_content replayed when ReplaysReasoningContent is declared MiniMax (inline <think> block split out, re-inlined on replay)
ReasoningFieldName reasoning_content OpenRouter (reasoning)
CostCurrency none OpenRouter (USD)
TryAdaptRejectedPayload re-sends once without the output cap when the catalogue's cap was refused Kimi (the mandated temperature, then the base rule)
BuildEndpoint {baseUrl}/chat/completions Azure OpenAI (dated deployment URL, or the v1 surface with api_version: v1)

Prompt-cache tokens and TTL

LlmResponse carries three cache counts: CacheHitTokens (read from the cache), CacheWriteTokens (written to it β€” Anthropic's cache_creation_input_tokens) and CacheMissTokens (input not served from the cache). PromptTokens already includes the written and read tokens: a write is never counted twice. MeteredLlmProvider records them once per call in the CostUsageEvent; CostBudgetManager keeps per model the prompt, completion, cache-write, cache-read and reasoning tokens and the cost (CostReport.ModelUsageByModel), and session_stats returns them in by_model_tokens[] beside the unchanged by_model cost map. A cache or reasoning count that no call reported is null in ModelUsage, never 0 (a call that reported none adds nothing to the others: null + 5 = 5), and its key is absent from the by_model_tokens[] entry; 0 means the provider measured nothing. A model whose ModelPricing declares CacheWritePricePerMillion / CacheReadPricePerMillion is priced as (prompt - write - read) x prompt rate + write x write rate + read x read rate + completion x completion rate; without those rates the cost is unchanged. Every built-in Anthropic entry declares them (#257): a write at 1.25x the prompt rate, a read at 0.1x unless the vendor states another figure (Opus 5.5 and Sonnet 5.5 read at 0.20 $/M, Fable 5.1 at 0.25 $/M). The write rate is the 5-minute TTL's, 1.25x; the 1-hour share of a write is costed at 2x the prompt rate: Anthropic reports it in usage.cache_creation.ephemeral_1h_input_tokens, which AnthropicLlmProvider reads (buffered and streamed) into LlmResponse.CacheWrite1hTokens, and MeteredLlmProvider hands to the CostUsageEvent as CacheWrite1hTokens, a subset of CacheWriteTokens. claude-opus-4-5 (and its dated ids) is priced at the official 5 / 25 $/M, not at the claude-opus-4 prefix's. Haiku 5.5 is priced at its rate for a prompt up to 100K tokens; a longer prompt bills more, which the registry does not model. A ModelPricing row may name a Provider (null: any provider; CustomPricings:<i>:Provider in the settings). CostBudgetManager prices a call with IModelPricingRegistry.TryGetPricing(provider, model) / CalculateCost(provider, model, …): a custom row beats a built-in one, and within each tier the provider's own rows come first (see configuration).

The cache TTL is LlmCacheConfig.Ttl (5m or 1h). A call can carry it as the option LlmChatOptionsKeys.CacheTtl (orkeon:cache_ttl), which ChatOptionsComposer fills from the agent's llm.cache.ttl; the chat client adapter folds it into the call's Cache.Ttl and Anthropic emits it in cache_control. A call TTL with no cache configuration that places a breakpoint is itself the request: the adapter marks the system prompt with that lifetime (LlmCacheConfig.SystemPrompt()), since a lifetime on nothing would cache nothing. The OpenAI-compatible providers and Ollama cannot honour it and log the structured unhonoured-option warning (event 110, cache_ttl) instead of dropping it silently.

The request is opt-in, from YAML (llm.cache: { system, tools, ttl }) or from a script: agentBuilder().cache({ systemPrompt, tools, ttl }) and the cache key of LlmConfig.with({...}) map onto the same LlmCacheConfig (LlmCacheConfig.Request). The TTL is 5m or 1h, anything else is refused in both, and a script's ttl without systemPrompt or tools is refused as caching nothing. An object that marks nothing (cache: {}, cache({})) is an explicit Off (LlmCacheConfig.Off): it turns an inherited cache off, the host default included, where a cache left out inherits it.

A host sets a default for every agent with Llm:Cache (SystemPrompt, Tools, Ttl), and a profile with Llm:Profiles:<name>:Cache. It is validated as the YAML block is (LlmCacheConfig.Request): a Ttl other than 5m or 1h, or one with neither SystemPrompt nor Tools, refuses the load and names the key. It applies only to a provider whose cache is explicit (LlmProviderCapabilities.ExplicitPromptCaching, Anthropic): elsewhere the provider's BaseConfig drops it, so it neither reaches the call nor warns. An agent's or task's own cache, an explicit Off included, always wins. The procedural ctx.llm calls of a script agent run with the cache and the effort its builder set (.cache(...), .effort(...), or the LlmConfig given to .llm(...)), a call's own options winning.

Profile fallback

A profile that fails over names its replacements: Llm:Profiles:<name>:Fallback: [b, c] (a list of profile names, or one comma-separated value; default names the host's own profile). A call that fails on <name> in a way a retry cannot cure goes to b, then to c; a profile registered in code takes the same list with services.AddOrkeonLlmProfile(name, provider, baseConfig, ["b", "c"]).

Failure Class (reason) Moves on
connection refused, reset, DNS, a stream closed before any answer transport yes
a 5xx server_error yes
a 429, once Llm:MaxRetries is spent, or a rate-limit window longer than MaxRateLimitWait rate_limited yes
the HTTP timeout, a 408, a stream idle past StreamIdleSeconds timeout yes
every profile of a nested chain failed exhausted yes
any other 4xx (400, 401, 403, 422...), a blocked content (the decision stages), a failure nothing is known about β€” no: it is the answer
the caller's own cancellation, thrown or reported in-band by the provider β€” no: the call is cancelled, nothing is switched or tried

LlmFailureClassifier is the one classification: ResiliencePolicies takes its list of retryable statuses from it (5xx, 408, 429), so what a retry may cure and what a chain takes over are the same set.

How the chain is built (LlmProfileRegistry): FallbackLlmProvider is the outermost link, above profiles that are already assembled. Each profile keeps its own decorators (RateLimited β†’ Traced β†’ Metered β†’ CallMode β†’ provider), so the meter and the trace name the provider that answered, a failed call is traced as the failure it is, and each profile keeps its own call mode. The chat client of the chain reads tool calls with the format auto-detection (Anthropic or OpenAI shape), not a per-profile parser. The chain's Capabilities (and the structured-output choice the chat client makes from them) are the first profile's only; a later profile with fewer capabilities is reported by the warning below, not accounted for in the chain's own.

Each profile uses its own settings. The chat client starts every call from the first profile's configuration and lays the caller's options over it; the chain gives a later profile the call with every field that still equals the first profile's replaced by that profile's own value (model, key, endpoint, MaxTokens, Temperature, Thinking, CustomParameters, SystemMessage, StopSequences, GrammarGbnf, timeouts, all of them). A field the caller set on the call (an explicit MaxTokens, a temperature in the chat options) differs from the first profile's and stays for every profile. A caller that sets a field to exactly the first profile's value cannot be told from the profile: the later profile's value applies.

  • Streaming: a stream moves on only before its first delta (a throw, or a completion that is a failure, with nothing yielded). Once a token reached the caller the call belongs to its profile and a later failure is the caller's. A profile that does not stream answers buffered.
  • Budget: Llm:Profiles:<name>:FallbackBudgetSeconds (absent: none) is the longest the chain spends, from its first call, before it stops trying the next profile. It is checked between two profiles and never cuts a call in flight, so without it a chain costs, at worst, each profile's TimeoutSeconds times its MaxRetries.
  • Exhausted chain: a buffered call answers with an error response whose ErrorType is FallbackExhausted and whose text names every profile and its failure; a call that failed by exception throws one HttpRequestException carrying the same text, the last exception as its cause.
  • Validation (when the registry is built, not in CrewFactory, which sees only the profiles crews name): a name no profile defines, a profile that names itself, and a cycle (a -> b -> a) refuse the host's start. A fallback may be a profile hidden from crews by Orkeon:Host:LlmProfiles.
  • Lesser fallback: a fallback whose LlmProviderCapabilities are below the first profile's (vision, response format, thinking, GBNF grammar) logs one warning naming what it lacks when the chain is built; an option that needs it is then handled by the provider that receives it, as for any provider.
  • Event: each switch writes llm.fallback on the --events stream (the run event bus) and a warning in the log. A host that registers an ILlmFallbackObserver is told too. Only orkeon run (with --events) registers one: orkeon-host, the REPL and a library host fail over all the same but write no llm.fallback, the log warning being the only trace unless the host registers its own observer.
  • No stickiness: a chain starts again from its first profile on every call. With a primary that is down, each turn of a tool loop pays the primary's TimeoutSeconds times its MaxRetries before it reaches the fallback; FallbackBudgetSeconds does not shorten that wait (it is checked between two profiles), so lower the primary's own TimeoutSeconds and MaxRetries for a profile that has a fallback.
  • A revoked key blocks the chain: a 401 and a 403 never switch (they are a configuration error, not an outage), so a primary whose key was revoked answers its error to every call and its fallbacks are never tried.

What a model accepts, and audio as input

LlmProviderCapabilities is declared once per provider, so it cannot tell two models of one vendor apart: gpt-4o takes no reasoning effort, o3-mini takes three, and both ride the same provider. The per-model side is a table, IModelCapabilityRegistry (ORKEXP005, experimental), built like ModelPricingRegistry: a model id prefix maps to ModelCapabilities (AudioInput, AudioOutput, Vision, Thinking, ReasoningEfforts), the longest prefix of the model asked wins, case-insensitive. A prefix matches at a boundary only: the end of the id or a separator (-, ., :, @, /) right after it, so glm-5.3 covers glm-5.3-flash and never glm-5.30. An aggregator's vendor/model id (OpenRouter, Together AI, Hugging Face) is looked up whole first, then by the part after its last /. The built-in table is in code and covers the OpenAI chat, audio and reasoning families, Gemini, Claude, Grok, Mistral, and of the other vendors the models the provider campaign measured a restriction on: glm-5.3, kimi-k2.7-code, MiniMax-M3.1-Flash and qwen3.7-plus. A model with no entry of its own (kimi-k3, kimi-k3.1-pro, the qwen3.8 models, deepseek-flash) takes the levels its provider declares. The models that do not reason are entered with Thinking: None: gpt-3.5, gpt-4, gpt-4-turbo, gpt-4o, gpt-4.1, the audio models (gpt-4o-audio, gpt-4o-mini-audio, gpt-audio), gpt-5-chat, gemini-1.5 and gemini-2.0.

An entry can also name the provider that serves the model, <provider>/<model> (the provider's ILlmProvider.Name), for a gateway that maps a model's values its own way: the call looks up <provider>/<model> first, the whole id, then the model's own entry as above. The built-in ones come from campaign M18 (2026-11-13): mammouth/gemini-3.7, mammouth/gemini-3.8, openrouter/google/gemini-3.7 and openrouter/google/gemini-3.8 always reason and take minimal to max (Google's own API takes none to high), and together/zai-org/glm-5.3 is unrestricted, both switches included (Z.AI's own API refuses the off switch and most efforts). A model it does not know is added, or a built-in entry replaced, from the configuration, under either form of key (Llm:ModelCapabilities:<model>:... or Llm:ModelCapabilities:<provider>/<model>:...):

Llm:
  ModelCapabilities:
    acme-voice:                 # a prefix: acme-voice-2 matches
      AudioInput: true
      Thinking: EffortOnly          # None, EffortOnly (always reasons), Toggle or Budget
      ReasoningEfforts: [low, high]   # left out: no restriction; written empty: the model takes none
    together/zai-org/glm-5.3:   # this model as Together AI serves it, read before glm-5.3
      Thinking: Toggle

An entry merges with the built-in value of the same id field by field: a field left out keeps the built-in value, so an entry that sets AudioInput alone keeps the model's built-in effort restriction (for a model the table does not know, a switch left out is false and the efforts stay unrestricted). A ReasoningEfforts written empty says the model takes none. Thinking takes one name: a comma-separated value (EffortOnly,Toggle) refuses the load. A registry the host registers itself (IModelCapabilityRegistry, before AddOrkeonLlmProfiles) wins over the configured one. A model id containing : (Ollama's llama3:8b) cannot be a key: .NET configuration has no escape for the path separator :. Key its prefix instead (llama3 matches llama3:8b and every other tag). An entry without a model id ("", " ", or a key such as :Vision) is refused, whatever it holds (configuration).

  • Audio in. An AudioContentPart reaches a model that takes it as an input_audio part of the Chat Completions API (raw bytes, audio/wav or audio/mpeg; the API takes no audio address). The OpenAI-compatible family does it through ContentConverter.ToOpenAIContentParts(content, model, registry), so OpenAI and Gemini (through its compatible layer) are covered; ToAIContents judges the same way for the chat-client path. A model the table does not credit with audio, or does not know, is refused with a NotSupportedException that names the model and the Llm:ModelCapabilities:<id>:AudioInput key that declares it: the audio is never dropped in silence. Anthropic and Ollama stay refusals (their APIs take no audio); a provider that declares no vision keeps its structured warning "message attachments (audio)" and sends the text only. Azure OpenAI is named by deployment: declare the deployment with an override. Audio output is out of scope: the table records it, nothing sends or reads it.
  • Thinking and effort, judged against the model (event id 110, 113). Every provider, Anthropic and Ollama included, judges LlmConfig.Thinking the same way before writing it. The thinking support that applies is the lower of the provider's and the model's: on a model that always reasons (EffortOnly) "on" sends no switch and "off" is dropped with a structured warning; on a model with Thinking: None nothing is sent, with a warning. An effort is lowered to the highest level below it that both the provider's field and the model take, with a warning naming the model and its efforts; a model whose list is written empty gets no effort. A vendor value outside the five levels (none, minimal) is sent when the model's list holds it or the model is unknown; efforts compare case-insensitively. A token budget is judged against the provider's declared support alone: a provider that declares Budget sends it even to a model that always reasons, since the gateway converts it (OpenRouter on Gemini 3.7, campaign M18). Ollama's think holds one value, so "off" with an effort sends think: false and warns that the effort is not sent. A model the table does not know gets the options as asked, with a warning that nothing was checked, once per provider instance and model (Llm:ModelCapabilities:<id> silences it); a model whose entry leaves ReasoningEfforts out is not restricted.
  • Anthropic's switch is per model. "On" sends thinking: {type: adaptive, display: summarized}, so the trace comes back as a summary. "Off" sends the model's own value: between_tools on Sonnet 5.5, which takes it up to effort high, disabled up to xhigh on Sonnet 5, and disabled up to high on every other Claude model; above that effort the switch is left out with a warning, since the API refuses the pair. A buffered answer carries the thinking blocks in the reasoning_content metadata, as the stream does.
  • What comes back. The reasoning tokens (completion_tokens_details.reasoning_tokens) are read for every OpenAI-compatible dialect, buffered and streamed, and the chat-client adapter maps the provider's finish reason onto ChatResponse.FinishReason (stop, length, tool_calls, content_filter, or the vendor's own value): Gemini's MODEL_CONTEXT_WINDOW_EXCEEDED reads as length, its RECITATION and PROHIBITED_CONTENT as content_filter.
  • Fallback chains (WarnAboutLesserFallbacks) read the same table: a fallback whose model takes no audio, has a lower effective thinking support (a model that cannot reason, or one that cannot be switched off) or fewer efforts than the primary's is warned about once, next to vision, response format and grammar. Each link is read with its provider's name, so a <provider>/<model> entry applies; a vendor value (none, minimal) is not reported missing on a fallback model the table does not list, since a call would send it unchecked.
  • Asking the provider (#359, #360, ORKEXP005). ILlmProvider.DescribeReasoning(model) answers what a model takes with this provider: LlmReasoningCapabilities (the model, whether the table knows it, the effective ThinkingSupport, and the levels among low to max, in order, the provider's EffortLevels narrowed by the model's ReasoningEfforts: empty when the model takes no effort or does not reason). A null or blank model reads the provider's own. ILlmProvider.ReportEffort(config) applies the same judgement as the call to the configuration merged with the host's (so Llm:Thinking:Effort counts) and answers LlmEffortReport: the effort asked, the effort sent (null when none goes out, Ollama's "off" included) and LlmEffortNotice Lowered, Dropped or Unchecked (a model the table does not know), null when the effort went as asked. It is null when no effort is asked. Both default to null on the interface; HttpLlmProviderBase answers them, and every decorator forwards them to its inner provider. A fallback chain answers for its first profile and a decision provider for its target: a call that fails over, or that a verdict routes to another profile, is judged by the provider that serves it. The scripting DSL reads them as LlmConfig.capabilities, ctx.llm.capabilities(opts), ctx.llm.reasoning(opts) and the reasoning field of LlmConfig, ActResult, ChatResponse and LlmStream (DSL reference).

Embedding providers

Orkeon:Embeddings:Provider is resolved by DefaultEmbeddingProviderResolver, after a local or code-analysis provider and before the fail-fast stub. Five names are accepted; another one refuses the start.

Provider Class Endpoint Default model (dimensions)
openai OpenAIEmbeddingProvider any M.E.AI embedding generator the host registers text-embedding-3-small (1536)
ollama OllamaEmbeddingProvider /api/embed of the Ollama host the configured Model
mistral MistralEmbeddingProvider https://api.mistral.ai/v1/embeddings, bearer key mistral-embed (1024)
azure-openai AzureOpenAIEmbeddingProvider {Endpoint}/openai/deployments/{Model}/embeddings?api-version=, api-key header the deployment named by Model
gemini GeminiEmbeddingProvider https://generativelanguage.googleapis.com/v1beta/openai/embeddings, bearer key gemini-embedding-001 (3072)

The last three share OpenAICompatibleEmbeddingProviderBase: one request shape (model and input), answers re-ordered by index, batches cut at BatchSize, and a vector whose length is not the declared Dimension refused. When Model and Dimension keep the global defaults (text-embedding-3-small, 1536), mistral and gemini use their own model and dimensions; azure-openai always takes Model as the deployment name. A 4xx or 5xx answer raises an HttpRequestException that names the provider, the status and the start of the body, never the key. Embedding calls do not retry: a 429 or a 5xx reaches the caller as that exception on the first answer, and Llm:MaxRetries applies to chat only. mistral-embed (1024) and gemini-embedding-001 (3072) have a fixed size and these providers send no output size: a Dimension other than that size (the global default counts as unset) is refused when the provider is built, naming the size, before any call. The key is ApiKey or the variable named by ApiKeyEnvVar; with neither, the first call fails with a message naming both. Model ids of the embedding providers are not in the capability table of What a model accepts: it covers chat input and reasoning effort.

Changing the embedding model invalidates stored vectors. At start, a guard (run by RunnerHost.Build for orkeon run, the other commands and the daemon, by the REPL's start, and by the hosted service of a host that starts its services) compares the provider, model and dimensions with the fingerprint recorded at Orkeon:Embeddings:FingerprintPath (default /workspace/.orkeon/embedding-fingerprint.json, in the virtual file system). The first start records it; a different one stops the host with an error that names both models. Nothing is migrated: restore the previous settings, or re-embed and clear the stored vectors, then delete the fingerprint file. The guard acts only where a mount covers FingerprintPath. A host whose embedding provider is configured, with or without an Orkeon:Embeddings section, is guarded there: a host that embedded under the default provider and then writes a section (a switch to Mistral or Gemini) finds the default's fingerprint and is refused. A host with no embedding provider (none registered, no section) writes nothing. A host whose FingerprintPath lies under no mount (a run with no /workspace) skips the guard at Debug, with no warning and no "Access denied" line; on a mount that grants no Write or Create on the file and no Create on its folder (overrides included; a read-only /workspace) the guard still reads and compares a recorded fingerprint and still refuses a changed model, but writes nothing, and when no fingerprint is recorded yet logs one Debug line that names Orkeon:Embeddings:FingerprintPath, never a Warning; a provider that cannot report its dimensions at start (a custom ModelPath that is missing) makes the guard log a Warning with the provider's own message, and the next call asks again; and an empty FingerprintPath turns the guard off, with no log. Every warning about reading or writing the fingerprint names Orkeon:Embeddings:FingerprintPath; the warning for a provider that cannot report its dimensions carries that provider's own message instead.

/workspace is not always writable, so the two commands that cannot rely on it guard where it means something. orkeon forge defaults FingerprintPath to /forge/embedding-fingerprint.json, recorded per session: the guard there protects one session and nothing across sessions. That default sits at the lowest precedence: a FingerprintPath set explicitly in the settings wins. orkeon rag uses no host fingerprint (its default FingerprintPath is empty): the manifest of each collection records the provider, model and dimensions it was indexed with and refuses a change on ingest without --reindex, on search, on eval rag --no-ingest -c and on eval rag -c when its ingest pre-pass runs without --reindex, with a line that names the collection, both models and orkeon rag ingest --reindex, which rebuilds the collection and passes. A first write that a read-only mount denies is logged at Debug; any other failure to write still warns.

Cohere, Voyage and pgvector are not shipped.

Decorators and registration

  • MeteredLlmProvider β€” the one place LLM usage is measured: it reports every call of the provider it wraps to the host's ILlmUsageSink, with the vendor's own cost when the answer carries one and an estimate flagged as such otherwise. LlmProviderFactory wraps every provider it builds; a provider the factory does not build (the echo provider of a host without an Llm section, a Microsoft Agent Framework agent, a test double) is registered with services.AddOrkeonLlmProvider(sp => …, baseConfig), which exposes it as ILlmProvider, IBasicLlmProvider and IChatClient over one metered instance. AddOrkeonInfrastructure() registers no model of its own: orkeon run, orkeon-host and orkeon-repl register theirs from the Llm section β€” or the echo provider when there is none β€” and a container that registers none fails at its first LLM resolution, naming the missing service.
  • RateLimitedLlmProvider β€” the host's RateLimiting at the entrance of each provider: every call takes one lease of the ILlmRateLimiter (global, per provider, concurrency) for the time of the call, a stream for its whole enumeration. It is applied with the meter, around it, at the points that meter a provider β€” the factory, AddOrkeonLlmProvider and AddOrkeonLlmProfile, the profile registry's ForProvider, the manager's resolver, the planner of a C# crew (LlmProviderEntrance) β€” so every model call of the host is limited once, the scripts' ctx.llm.* included. A provider already limited is left as it is; a call made inside another limited call takes no lease (under MaxConcurrentRequests: 1 it would wait on itself); a provider that runs its own tools (RunsOwnTools, the Microsoft Agent Framework bridge) is not limited itself β€” what it calls of Orkeon's model is. A refusal is retried five times on its RetryAfter, then the call fails with the limiter's reason.
  • Call mode β€” CallModeLlmProvider decides how each call of a provider built from a setting reaches the model, for every caller (agent loop, manager, planner, judges, RAG). The setting's CallMode key (Llm:CallMode, Llm:Profiles:<name>:CallMode) is Auto (absent or empty), Buffer or Stream, case ignored; another value fails the load. Buffer answers whole; Stream and Auto stream on the wire when the provider streams natively and fold the events into the same LlmResponse (text, tool calls, finish reason, usage), so the meter and the limiter still see one call. In Auto, a clear refusal of streaming before any event (status 404, 405, 415 or 501, or 400/422 whose body names the stream, or a provider that declares no streaming) falls back to a buffered call for the rest of the run (LlmCallModeScope, opened per crew run), with one warning; a 401, 403, 429, a timeout or a stream cut after its first event stays a failure. In Stream, a refusal fails the call naming Auto and Buffer. It sits inside the meter and the limiter, applied by LlmProviderFactory.
  • LLM exchange logging β€” LlmLoggingDelegatingHandler (Orkeon.Infrastructure.Logging) captures every HTTP exchange (headers and payload, sanitized by LogSanitizer: credential headers redacted by name, secrets in bodies by pattern) as JSON Lines plus a structured log summary. services.AddLlmExchangeLogging(logDirectory, options) injects it into every IHttpClientFactory client; the IHttpClientBuilder overload targets one named client. LlmLoggingOptions: MaxBodyLengthChars (0 = no truncation), LogStreamingExchanges (true; a stream passes through to the caller as it arrives and is logged once it ends, as its rebuilt answer, or the raw text received when it was cut), FullEmbeddingLog (true; false shortens embedding arrays to a preview). The runners switch it on with --llm-log / --llm-log-path and read the options from the LlmLogging settings section; the files go to the internal /llm-logs mount, never visible to agents. It is a debugging facility: the capture holds full prompts.

Validating a provider against its real API

Every unit test in this area speaks to a mocked HTTP handler, which proves Orkeon sends what we believe it sends β€” not that the vendor accepts it. The second proof is produced by the campaign kit in llmproviders-test/:

llmproviders-test/run-campaign.sh --provider ollama --model llama3.2   # zero-cost first run
llmproviders-test/run-campaign.sh --all --config providers.local.json --dry-run

It drives orkeon llm probe over modes M1–M10, M12, M13 and M15–M18 of the provider test protocol (maintainers' internal matrix) and archives one Markdown report per campaign. M15–M17 send Orkeon's thinking configuration through the provider and record, per step, the reasoning fields that really went on the wire. M18 goes the other way: with --raw-reasoning, a handler rewrites the reasoning fields of the request just before it leaves (the provider still builds auth, URL, headers and dialect), so it bypasses Orkeon's capability judgement and measures what the vendor itself accepts or refuses. Each step also reads the vendor's answer on the wire before the provider parses it (trace, reasoning tokens, finish reason), so a gap between what the vendor returned and what the provider surfaced shows in the report. Once every mode ran, each M15–M17 step is checked against M18: a value Orkeon sent that the vendor refused (400, 422) means the model capability table misses a restriction, a value Orkeon lowered or left out that the vendor takes means the table is too narrow; either fails the mode. A value that switches reasoning off counts as taken only when it stopped the reasoning, and a vendor that keeps reasoning at effort:none proves nothing by accepting an effort. orkeon llm models -p <provider> --filter 'gpt-5.6-*' lists what a provider currently serves, so a campaign never depends on a hand-maintained model list.


See also: Provider comparison Β· LLM response format Β· Multi-modal content Β· Local models Β· Memory system Β· Security Β· Back to index