Table of Contents

πŸ‡«πŸ‡· Version franΓ§aise

LLM Response Format

Force JSON output at the provider boundary instead of patching prompts. One value object, LlmResponseFormat, is translated by every provider according to what its API declares it can honour.

Why

Most LLM APIs can constrain the shape of a response at the API layer. Ask for response_format: {"type": "json_object"} on DeepSeek and the API guarantees valid JSON β€” no broken {/} brackets, no leading prose; ask OpenAI, Gemini or Anthropic for a JSON Schema and the vendor validates the answer against it server-side. This is a second barrier complementing the GBNF grammar (llama.cpp-compatible servers, behind Llm:Grammar) and the downstream StructuredOutputResolver.

For Orkeon, this matters most for:

  • structured_output deliverables that must parse cleanly
  • Hierarchical manager agents returning JSON decisions
  • TypeScript-scripted workflows expecting typed objects

The three formats

Type Built with Meaning
text LlmResponseFormat.Text() The provider default. Maps to nothing on the wire β€” writing text and writing nothing are equivalent.
json_object LlmResponseFormat.JsonObject() Well-formed JSON, no schema.
json_schema LlmResponseFormat.JsonSchema(name, schema, strict = true) JSON validated against a JSON Schema (LlmJsonSchema: the schema travels as a JSON string; strict asks the vendor to reject any deviation, ignored where no strict mode exists).

Type is an open string: any other value is forwarded as-is (the YAML mapper logs a warning, event id 101), so a new vendor value works without a framework release.

Cascade β€” two layers, four surfaces

The base config and the task override are merged once per turn: by ChatOptionsComposer on the chat-client loop, by LlmConfigResolver.Resolve(baseConfig, taskOverride) on the fallback loops. Priority highest-first: task override β†’ base config. Each field is resolved independently (Profile, ResponseFormat, Temperature, MaxTokens, TopP, Thinking, Cache), and a null field on an override never erases an inherited value.

Surface Lands in Where it is written
1. Crew default base config crew YAML llm: β€” merged field by field into every agent's llm:
2. Agent base config agent YAML llm:, agentBuilder().withResponseFormat(...) / .withResponseSchema(...), the agent's LlmConfig in C#
3. Task task override (LlmConfigOverride) task YAML llm_override:, taskBuilder().withResponseFormat(...) / .withResponseSchema(...), CrewTaskBuilder.WithResponseFormat(...) / .WithLlmOverride(...)
4. Script-time base config llmConfig.with({ responseFormat }) on the config handed to agentBuilder().llm(...)

A call, from C# or from a script (ctx.llm.complete/chat/stream(..., { responseFormat })), passes its own LlmConfig, outside the cascade.

YAML surfaces

Crew default

llm:
  model: deepseek-flash
  response_format: json_object   # every agent of this crew replies in JSON

Agent override (wins over crew)

agents:
  extractor:
    role: "Invoice extractor"
    llm:
      model: deepseek-flash
      response_format: json_object   # only this agent forces JSON

Task override (wins over agent β€” the surgical one)

tasks:
  extract_invoice:
    description: "Extract fields. Reply as a json object."
    expected_output: "JSON object with all invoice fields"
    llm_override:
      response_format: json_object
      temperature: 0.0

The llm_override: block accepts profile (the task runs on that host LLM profile instead of its agent's), response_format, response_schema, temperature, max_tokens, top_p, thinking and cache β€” the agent-level llm: fields minus model. A task's cache: block (same shape as an agent's) replaces the agent's whole; a block with no breakpoint turns the agent's cache off for that task.

JSON Schema

response_schema: accompanies response_format: json_schema, at agent, crew or task level. schema is the JSON Schema as a JSON string (a block scalar keeps it readable); name defaults to response, strict to true:

llm:
  model: gpt-5.6-sol
  response_format: json_schema
  response_schema:
    name: invoice
    strict: true
    schema: |
      {"type": "object",
       "properties": {"number": {"type": "string"}, "total": {"type": "number"}},
       "required": ["number", "total"], "additionalProperties": false}

A response_schema: on its own implies json_schema. json_schema without a schema falls back to json_object with a warning (event id 102).

TypeScript / .ork.ts surfaces

// Builder β€” agent (the host's configured provider, here DeepSeek)
const extractor = agentBuilder()
    .name("extractor")
    .role("Invoice extractor")
    .llm(llm.default_.with({ model: "deepseek-flash" }))
    .withResponseFormat("json_object")
    .build();

// Builder β€” task (overrides agent on this task only)
const task = taskBuilder()
    .description("Return invoice fields as JSON")
    .expectedOutput("JSON")
    .agent(extractor)
    .withResponseSchema("invoice", {
        type: "object",
        properties: { number: { type: "string" }, total: { type: "number" } },
        required: ["number", "total"],
    })
    .build();

// A call passes its own LlmConfig, outside the cascade
const res = await ctx.llm.complete(
    "Return the answer as a json object",
    { responseFormat: "json_object" }
);

.llm(...) takes an LlmConfig β€” llm.default_, llm.model("…"), llm.profile("…") or .with({...}) on any of them β€” and refuses a string or a plain object literal. The provider is the host's, or the one of the host profile llm.profile(...) names; .llm(...) sets the model. A response format is checked against the provider the agent actually runs on. withResponseSchema(name, schema, strict?) takes an object literal or a JSON string and implies json_schema. ctx.llm.extract(prompt, schema) sends the schema as a json_schema constraint (name extract, strict: false, since a hand-written schema rarely carries the additionalProperties: false and full required a strict one needs) when the provider validates one (ResponseFormatSupport.JsonSchema), and asks for json_object otherwise; an explicit responseFormat on the call wins, "text" included.

C# fluent surface

var task = new CrewTaskBuilder()
    .Description("Extract as JSON")
    .ExpectedOutput("JSON")
    .WithResponseFormat(LlmResponseFormat.JsonSchema("invoice", invoiceSchemaJson))
    .Build();

// A call passes its own LlmConfig, outside the cascade. Start from the provider's own
// configuration: built empty, it would reset the provider's penalties.
var config = (provider.BaseConfig ?? LlmConfig.OnProfile()) with
{
    ResponseFormat = LlmResponseFormat.JsonObject(),
};
await provider.GenerateAsync(prompt, config, ct);

WithResponseFormat also accepts a string ("json_object"); WithLlmOverride(LlmConfigOverride) sets the whole task patch.

On the wire

  • OpenAI-compatible family (every provider but Anthropic and Ollama) β€” OpenAICompatibleProviderBase writes response_format once, for every provider that declares the capability: {"type": "json_object"}, or {"type": "json_schema", "json_schema": {"name", "strict", "schema"}}.
  • Anthropic β€” output_config.format, schema only: Anthropic has no json_object equivalent, so a schema-less JSON request is reported (structured warning) rather than sent.
  • Ollama β€” format: the schema object itself, or "json" for json_object.

What a provider cannot honour is never dropped in silence:

  • a provider declaring None sends nothing and logs Option 'response_format' was declared but … does not support it (event id 110);
  • a schema sent to a JsonObject provider is downgraded to json_object with the same warning (response_format.schema) β€” describe the shape in the prompt.

The "json" keyword guard

DeepSeek's JSON mode requires the prompt (system or user message) to contain the word "json" somewhere β€” otherwise the API may emit an unbounded whitespace stream until max_tokens is exhausted. Every provider that declares RequiresJsonKeywordInPrompt (DeepSeek alone) logs a structured Warning (event id 100, LogMissingJsonKeyword) when it detects the situation:

DeepSeek was asked for a JSON response format but no system/user message contains the
word 'json'. This API may then emit an unbounded whitespace stream until max_tokens.
Add 'json' to the prompt to be safe.

We do not mutate the prompt for you β€” the caller stays in control. Add "Reply as a json object." to the system message and the warning disappears.

Provider support matrix

Support is capability-driven (LlmProviderCapabilities.ResponseFormat β€” see the provider comparison). The declaration is per provider while reality is per model: a model that refuses the field answers with the vendor's own error, which surfaces as LlmResponse.Error and fails the task with that reason.

Provider Declared capability Notes
OpenAI, Azure OpenAI, Grok, Gemini, Mistral, Together AI JsonSchema Server-side schema validation.
Anthropic JsonSchema Own dialect (output_config) β€” schema-only, no bare json_object.
Ollama JsonSchema Own dialect (format).
DeepSeek, Kimi, Qwen, HuggingFace, Z.AI JsonObject Well-formed JSON guaranteed; a schema is downgraded with a warning. DeepSeek also requires the json keyword (above).
MiniMax None Accepted but non-binding (a schema is ignored, json_object comes back fenced in markdown); a declared format produces the structured capability warning.
OpenRouter JsonSchema Documented per endpoint. The provider does not send provider.require_parameters, so an endpoint that lacks schema support may still ignore a schema.
Mammouth AI None Accepted but non-binding (json_object returns bare JSON, a strict schema is not enforced); a declared format produces the structured capability warning.

structured_output deliverables

A task whose deliverable is source: structured_output carries its schema (schema_inline or schema_path) in two forms, and the chat-client adapter sends the one the provider honours β€” never both, which llama-server refuses:

  • a GBNF grammar, on an endpoint the settings declare able to take one ("Llm": { "Grammar": true } β€” Docker Model Runner, llama-server);
  • otherwise a json_schema response format (name structured_output, strict: false, since OpenAI's strict mode refuses a schema that leaves an object open or a property optional), on a provider that declares JsonSchema in the matrix above;
  • otherwise nothing binds: the grammar reaches the provider, which drops it with a structured warning naming Llm:Grammar.

A response format the crew sets itself (llm: / llm_override:) always wins over the deliverable's schema. In every case StructuredOutputResolver still parses the answer as JSON before writing the file.

How it travels through the orchestrator

The cascade is fused exactly once per turn by LlmConfigResolver.Resolve. On the chat-client path, ChatOptionsComposer calls it on the agent's configuration and the task's override, then translates the merged configuration onto the ChatOptions field by field: it merges nothing itself, and each field reaches the provider as written. The other call sites have no chat client and call it as they are: LegacyTextAgentLoop and OutputValidationCoordinator (base: the agent's configuration), and NativeToolCallingAgentLoop (base: BuildNativeLlmConfig, the agent's configuration plus the system prompt and the tools). ExecutionOrchestrator drives them:

  1. Legacy text-based [TOOL_CALL] loop β€” _llmProvider.ChatAsync(prompt, effectiveConfig, …)
  2. Native tool-calling loop β€” _fullProvider.ChatAsync(messages, effectiveConfig, …). BuildNativeLlmConfig seeds from agent.LlmConfig, so the model name and Thinking config survive the entry to the native path; an agent without one gets a configuration that names no model (LlmConfig.OnProfile()), and the provider runs the call on its own.
  3. Validation correction retry β€” _llmProvider.ChatAsync(correctionPrompt, effectiveConfig, …)

The 6 process strategies (Sequential, Hierarchical, Autonomous, Graph, Parallel, Consensual) run tasks through IAgentExecutionService.ExecuteTaskAsync, which delegates to ExecutionOrchestrator.ExecuteTaskCoreAsync β€” they get the cascade for free.

LlmBasedManager (manager LLM in Hierarchical mode) does not apply a task override: its calls run on the LLM the crew gives its manager β€” the provider WithManagerLlm sets, else the manager agent's profile and model, else the default profile β€” with that provider's own configuration for everything else (no response format, no task override).

Reference

  • Value objects: Orkeon.Domain.SharedKernel.ValueObjects.LlmResponseFormat, LlmJsonSchema
  • Patch record: Orkeon.Domain.SharedKernel.ValueObjects.LlmConfigOverride
  • Fusion: Orkeon.Domain.SharedKernel.ValueObjects.LlmConfigResolver.Resolve
  • Wire translation: Orkeon.Infrastructure.LLMs.Base.OpenAICompatibleProviderBase.ApplyProviderSpecificOptions (virtual β€” Kimi, OpenRouter, Qwen, Together AI and Z.AI override it and call the base for response_format); AnthropicLlmProvider and OllamaLlmProvider for their own dialects
  • YAML mapping: Orkeon.Infrastructure.Configuration.Yaml.YamlCrewMapper (its private MapResponseFormat); the export is Orkeon.Infrastructure.Configuration.YamlCrewExporter.MapResponseFormat
  • DeepSeek API doc: https://api-docs.deepseek.com/api/create-chat-completion