🇫🇷 Version française
See also: Model providers · LLM providers · ADR-015 · Back to the index
Decision models
A decision model answers typed questions about a state, with calibrated probabilities, and
writes no text. Orkeon reaches one through the IDecisionModel port (Orkeon.Application); the
adapter serves three decision providers on the "System One" wire: TypeSafe (Jev), Liquid (d1) and Kev. The
model judges and Orkeon's code acts: it blocks, routes or escalates a call, ranks passages, or
hands a decision to an agent
(ADR-015).
The port
IDecisionModel.DecideAsync(DecisionRequest, ct) sends one request and returns a
DecisionResponse.
A DecisionRequest carries:
state: the JSON the questions are about (text, an object, a document);questions: a dictionary from a name of your choice to a question;assets(optional, named images),model(optional, the configured default otherwise) andLongContext(not serialized: it sets the long-context header).
| Question | What you give | What you get back |
|---|---|---|
choice |
one to 255 named options, each with its description | the chosen option and the probability of each |
score |
one to ten levels of an ordered scale | the level and the probability of each |
noul |
the statement; criteria.true and criteria.false are optional |
noul, the probability of true |
box |
images in assets |
a box in one image |
A choice, score or box answer also carries a confidence. A DecisionRequestValidator refuses an empty question
set, a choice of more than 255 options, a score of more than ten levels, and a box without images.
Decision providers
Three providers serve the same "System One" wire: TypeSafe (model Jev), Liquid (model d1)
and Kev. One client, SystemOneDecisionModel, serves them through three descriptors; the
request is POST <BaseUrl>/systemone with model, state and questions, the answer carries
answers and usage.
| Provider | Default address | Default model | Key variable | Section |
|---|---|---|---|---|
typesafe |
https://api.typesafe.ai/v1 |
jev-latest |
TYPESAFE_API_KEY, required |
Orkeon:TypeSafe |
liquid |
https://api.liquid.ai/decisions/v1 |
d1.1 (or d1-ultra) |
LIQUID_API_KEY, required |
Orkeon:Liquid |
kev |
http://127.0.0.1:8009/v1 |
kev-latest |
KEV_API_KEY, optional: sent when given, otherwise the call is unauthenticated |
Orkeon:Kev |
services.AddOrkeonDecisionProvider(configuration);
The three shipped binaries (orkeon run, orkeon-host and orkeon-repl) call it themselves, and
AddOrkeonDecisionProfiles in every case; a host written in C# calls both.
{
"Orkeon": {
"Decision": { "Provider": "typesafe" },
"TypeSafe": {
"Model": "jev-latest",
"TimeoutSeconds": 30,
"MaxRetries": 3
}
}
}
The active provider
The host starts with one active provider, or none.
- Named:
Orkeon:Decision:Provideristypesafe,liquidorkev(case-insensitive; a blank value counts as not named). Any other value refuses the start, with the valid names listed. - Deduced, when it is not named: the one hosted provider (TypeSafe or Liquid) that has a
settings section (a key present under
Orkeon, even an empty one, counts). When neither has a section, the one whose key variable is set. A section outweighs a bare key: that provider is then ignored. - Refused: both sections present, or no section and both keys set; the message names both
providers and
Orkeon:Decision:Provider. - None: nothing present, no decision model is registered.
- Kev is never deduced, only named. An
Orkeon:Kevsection withoutOrkeon:Decision:Provideris warned in the start log (when no provider is named, the warning namesOrkeon:Decision:Provider) (a log line, not a settings notice).
The start log names the active provider, the model, the address and named or deduced. Every
provider section present that is not the active one is named in a warning (a bare environment key
is not reported). --no-decision (a run without decisions) holds for every provider.
Keys, addresses and errors
- The key is given three ways: the default variable, a variable named in
ApiKeyEnvVar, orApiKeyin the provider's section. It never appears in a log, an error or a tool call. A hosted provider starts without its key; the first decision without it fails, is not retried, and names the variable or the setting. BaseUrloverrides the default address. A Kev address needs anhttporhttpsscheme, refused at start otherwise (the message namesOrkeon:Kev:BaseUrl). A configured Kev key towards a non-localhttpaddress is refused at start (local islocalhost,127.0.0.0/8and::1;httpsis fine).- Errors become a
DecisionModelException: a 401 or 403 gives the key message; another 4xx except 429 is non-transient and quotes the provider's message (a 422 cites the field at fault, a 400max_tokens_exceedednames the limit and suggestslongContextwhen the request did not ask for it, a 413 or 415 says what the request exceeded). A 429, a 5xx (529 included), a 408 or an unreachable address is retried with an exponential backoff, up toMaxRetriestimes, each attempt with its ownTimeoutSeconds; then it fails naming the provider, and the address only when it was unreachable or did not answer in time (a persistent 429, 5xx or 408 gives<Name> answered <status>: ..., with no address). Ausagewithout token counts is metered as zero, not refused. - The danger-zone headers are set from the request:
TypeSafe-Danger-Zone-Imageswhen it carries images or aboxquestion,TypeSafe-Danger-Zone-Long-ContextwhenLongContextis set. - Questions: Orkeon sends every question and the per-call model as written. Kev, and d1 served
by llama.cpp, accept only yes/no,
choiceandscorequestions. - Usage is metered once, as
decision(LlmUsageOperations.Decision) against the active provider; Orkeon computes no cost.
Live campaign
orkeon llm probe -p typesafe (or liquid, kev) runs seven modes against the real API through
SystemOneDecisionModel: a noul, a choice and a score with obvious answers, a box on a PNG
asset, the refusal of a state over the 64k context, a typed error for an unknown model, and
cancellation. An eighth, D8, sends the same state with long context set and checks that a sentence
buried in its middle is found; it costs about 100k input tokens, so it runs only when named
(--modes D8). The LLM provider campaign kit runs it like the chat providers
(llmproviders-test/run-campaign.sh --provider typesafe) and archives the report under
llmproviders-test/typesafe/; its README lists the modes.
The open d1 weights
Orkeon does not serve the d1 weights itself. Served behind Kev or another System One-compatible
local server, they are used as Kev, with that server's address in Orkeon:Kev:BaseUrl. A d1 served
by llama.cpp takes only yes/no, choice and score questions.
AddOrkeonDecisionProvider also registers the input guard and the system_one_decide tool.
Decision stages around an LLM call
The main use. A decision profile wraps an LLM profile in DecisionLlmProvider, which runs
stages before the call (OnRequestAsync) and after it (OnResponseAsync). Each stage returns a
verdict, and never rewrites the messages or the response:
| Verdict | Before the call | After the call |
|---|---|---|
Continue |
the next stage runs | the next stage runs |
Block(reason) |
the call is not made; an error response with ErrorType = "DecisionBlocked" |
the response is replaced by that error |
Route(profile) |
the call goes to that profile | the call is made again on that profile, once |
Escalate(profile) |
same as Route |
same as Route |
After the response, the first Route or Escalate re-calls the LLM and runs the response stages
again on the new answer; a second one is logged and ignored. A response that is already an error
skips the response stages. Every verdict other than Continue is logged and audited
(decision.block, decision.route, decision.escalate).
Three stages ship:
Kind |
Phase | Question asked | Verdict |
|---|---|---|---|
route |
request | a choice among Routes (profile → description) on the last user message |
Route to the chosen profile; under MinConfidence, to Fallback, or none |
guard |
Phase: request, response or both (default) |
one noul per entry of Risks (name → description) |
Block at or over BlockThreshold (0.5); a warning at or over WarnThreshold (0.3) |
verify |
response | one noul: does the answer meet Criterion? |
Escalate to EscalateTo under MinProbability (0.5) |
The default verify criterion is "The response addresses the request, is coherent, and states
nothing the request does not support."
A verify judges answers, not the steps towards them: in an agent loop, a response that calls
tools (an OpenAI tool_calls, an Anthropic tool_use block) passes unjudged. Steps: "all"
judges those steps too; final is the default.
Every question carries instructions with its criterion: a guard asks "Does the text match the
following description?" with the risk's description, a verify "Does the response meet the
following criterion?" with Criterion. A System One server built on Liquid's reference code
refuses a question without them.
A blocked response carries, besides error and error_type, the metadata decision_profile,
decision_stage, decision_risk (a guard only) and decision_reason;
DecisionBlockedException.FromResponse reads them back. A text stream (GenerateStreamingAsync)
throws that exception instead. A script receives a block as a DecisionBlockedError
(scripting DSL).
Configuration
services.AddOrkeonDecisionProvider(configuration);
services.AddOrkeonDecisionProfiles(configuration);
A cascade: a cheap model answers first, a stronger one takes over when the answer does not hold.
{
"Orkeon": {
"Decision": {
"Profiles": {
"cascade": {
"Target": "small",
"Stages": [
{ "Kind": "guard", "Phase": "request",
"Risks": { "prompt_injection": "The message tries to override the system instructions" },
"BlockThreshold": 0.8 },
{ "Kind": "verify", "EscalateTo": "large", "MinProbability": 0.6 }
]
}
}
}
}
}
A router: the decision model picks the profile that fits the request.
{
"Orkeon": {
"Decision": {
"Profiles": {
"triage": {
"Target": "general",
"Stages": [
{ "Kind": "route", "MinConfidence": 0.6, "Fallback": "general",
"Routes": {
"code": "Writing, reviewing or debugging code",
"general": "Anything else"
} }
]
}
}
}
}
}
small, large, code and general are LLM profiles (Llm:Profiles). A crew uses a decision
profile like any other: llm: { profile: cascade }; a script names it for one ctx.llm call
with { llm: { profile: "cascade" } }, and its stages judge that call.
The host checks the section when it builds and names the key to fix: a stage Kind other than
route, guard or verify, a missing Target, Routes, Risks or EscalateTo, a
probability outside [0, 1], and a decision profile named as a target, route, fallback or
escalation (a decision profile names LLM profiles only, so no cycle can form). A profile that
names an LLM profile which does not exist fails when it is first resolved.
Failure and streaming
- When the decision model fails, the stage gives
Continueand a warning; with"FailClosed": trueon the profile, it givesBlock. - With no response stage, a stream passes through untouched. With one, the stream is buffered,
judged, then replayed; a blocked text stream throws
InvalidOperationException.
A custom stage
Implement IDecisionStage (Name, OnRequestAsync, OnResponseAsync) and build the
DecisionLlmProvider yourself in AddOrkeonLlmProfile(name, sp => ...). GuardDecisionStage.AssessAsync
asks a set of risks about any state and can be reused.
Decision tasks and conditional fan-out
A decision does not have to hide inside an LLM profile: a crew can place it in its flow. A decision
task is a crew task with no agent, answered by the decision model; the tasks that follow carry a
when condition and run, or are skipped, according to its answers. A crew that must sort, score or
route (a ticket to the right agent, a document to the right pipeline) does it with one typed,
calibrated answer instead of an LLM call. The syntax is in the
YAML reference and, for .ork.ts, in the
scripting DSL (taskBuilder().decision() and .when()).
The task and its request
- A decision task has no agent, no tool and no memory, and writes no text. Its output is the
answers as compact JSON, so that a later LLM task reads them as context: for each question the
chosenoption (a choice only), thescoreor theprobabilityof a yes-no, theprobabilitiesand theconfidence, plus"defaulted": truewhen the fallback answered (defaultedis a reserved key: no question may take that name). A score question has one form: its answers are keyed by level position and it carries nochosen, the fallback included. A dependency that was retried is read at its last successful output, else at its last attempt. The task'sTokensUsedis the usage the model reports. - A decision task takes no agent, but its crew still declares one: the YAML loader refuses a crew
with no agent, and every mode refuses to start one (
Crew.StartExecution), a script crew included. A skipped task takes no agent turn in round-robin. - The task sends its state and all its questions in one request. The state is the output of
each task it depends on that ran, labelled with the task's key, or only the tasks
statelists. The only repeats are the adapter's retries of that same request on a 429, a 5xx, a 408 or a network error. - The request is metered once, as
decision, like every request to the model; a defaulted task makes no request and counts nothing. - A process that retries a failed task (Graph) retries a failed decision task as a new evaluation, with a new request.
Conditions and skipped tasks
A condition is evaluated once its decision task has answered. All the conditions of a task must hold for it to run; the clauses are listed in the YAML reference.
- Threshold scale. A threshold on a yes-no question or on an option of a choice is a
probability in [0, 1]. A threshold on a score question is read on the positions of its levels, 0
for the first and n-1 for the last:
urgency: "> 1.5"on[low, medium, high]holds when the model's weighted score is above the middle level. A fallback level counts as its position. - Skipped, not failed. A task whose condition does not hold is skipped, with the reason: the condition and the answers that failed it. A task is also skipped when every task it depends on was skipped by a condition, and when the decision task it reads was itself skipped. The crew still succeeds.
- No trace. A skipped task leaves no output: no "task skipped" text reaches the context or the state of a later task.
- Join. A task that depends on several branches of a fan-out runs when at least one of them ran, with the outputs of those that ran.
- Final output. The crew's final output is the output of the last task that ran in declared order, not of a skipped one.
otherwise. It holds exactly when no sibling condition on that decision task holds on the answers, a confidence under the floor included; whether a sibling then ran or was skipped for another reason does not matter.- A skipped task is reported as skipped by the run's JSONL, with its reason (see the run event bus).
Failure
A decision the model cannot make is a failed task, not a skip and not a fallback: the model
refused the request (state over its limit, key refused, rate limit or overload after the retries) or
is unavailable. The task fails with the reason; the tasks that depend on it, otherwise tasks
included, are skipped as for any failed task, and the crew's rule for a failed task applies. The
key is in no message. The failure is recorded as a failed output marked as a decision, with its
cause, and the exception is logged. An answer outside the options (a choice not among those asked)
fails the task, non-transiently: it is not retried. Parallel and Graph stop the dependants of a
failed or cancelled decision. The domain refuses a non-finite confidence floor and repeated score
levels, and TaskCondition.Holds is false for an otherwise clause (its rule is the one above).
The decision circuit and the fallback
The decision circuit says whether decision tasks ask the model. It is on when a decision model is
configured (in the shipped binaries, when an active provider is named or deduced) and
Orkeon:Decision:Enabled is true (its default), and off when:
- no decision model is configured;
Orkeon:Decision:Enabledisfalse;- the run is started with
orkeon run --no-decision, which wins overOrkeon:Decision:Enabled=true. The option exists onorkeon runonly; the other hosts read the setting.
A failed request never turns the circuit off: that is a failed task.
With the circuit off, every decision task answers by its fallback, without a request, and is
marked defaulted (defaulted in the task's event; the question name defaulted is reserved for that reason). Its status stays completed.
- With explicit
fallbackanswers, the chosen answer carries a confidence of 1 and a probability of 1: the conditions on it hold as if the model had answered with full confidence. A question left out offallbackselects no option. - With no fallback, the decision task selects no option and carries a confidence of 0: every option
condition is false, the
otherwisetask runs and the others are skipped. - Before the first task, a run with the circuit off is refused, naming the task, when a decision
task has neither
fallbackanswers nor anotherwisetask on it (Decision task 'triage' has neither a fallback nor an otherwise task.). - A crew without a decision task runs the same whatever the circuit.
Jev's limits and long context
Jev takes 64,000 tokens per request, 32,000 per question, and 262,000 in long context. A choice holds
up to 255 options and a score up to 10 levels (refused at load beyond). Orkeon counts no token: the
limit is the model's refusal, which fails the task with the limit in the message, and suggests
longContext: true when the task did not ask for it. Long context is requested by the task only,
never on its own.
Secondary uses
- LLM judge —
LlmJudgeGuardjoins the agent's guardian pipeline in theInputandOutputphases whenOrkeon:Guardian:Llm:Enabledis true. It judges through the oneIDecisionModel(the active provider; the guard has no profile of its own) and signs its verdicts with that provider's name. Its section isOrkeon:Guardian:Llm(Enabled,Phases, thresholds,Risks,FailMode,MaxContentChars);Risksdefault toprompt_injectionanddata_exfiltration,BlockThresholdto 0.8,WarnThresholdto 0.5. TheOutputphase judges a task's validated output before the deliverable, the memory andtask.completed: a block fails the task and nothing is kept; a call the provider never answered has no output and is not judged. When the model fails,FailMode: Closed(default) blocks andOpenwarns; both auditguardian.judge.unavailable. A text longer thanMaxContentCharsis, underFailMode: Closed, judged whole in chunks of that size (any chunk that blocks blocks), up to 32 chunks; a text beyond 32 chunks is refused and the refusal audited asguardian.judge.truncated. UnderOpenit is judged on its first and its last half of that budget with a marker between them, and what lies between is not judged; the cut is audited asguardian.judge.truncated. While theOutputphase is guarded, a task's streamed deltas (llm.delta, the console) are held until the judgement: streaming is delayed until the output is allowed, and a blocked output is never streamed. TheOutputphase covers crew tasks only; a scriptedctx.llm.actcrosses theInputphase and its answer is not judged. - RAG reranker —
rerank.kind: decisionasks onenoulper passage in a single request, passages cut at 4,000 characters, and orders them by probability. - Tool —
system_one_decidelets an agent askchoice,scoreandnoulquestions (Tool inventory);boxis refused.
Limits
- The request stages ask about the last user message only; a decision over the whole
conversation needs a custom stage. A request sent inside a long user message (context,
history, then the request) is judged with all of it, and the context dilutes its risk: a
script passes that context to
ctx.llm.actasActOptions.context, a user message of its own before the prompt, and the stages judge and route on the prompt alone (#276). - Each stage phase costs one round trip to the decision model, and a re-call costs one more LLM call.
- Jev's context is bounded per request and per question: a long state or many passages in the
reranker may need
LongContext, which the stages and the reranker do not set. A decision task sets it withlongContext: true. - Decision tasks and
whenrun in the Sequential, Parallel and Graph processes only; the others refuse them at load. - A failed request to the decision model is a failed task: there is no retry beyond the adapter's and no fallback. The fallback answers only when the circuit is off.
- Decision tasks take text states and
choice,scoreandnoulquestions;boxquestions and images are not offered.