Table of Contents

The host API and the health probes

orkeon-host can serve a small HTTP API on a listener of its own: two health probes for systemd, Docker and Kubernetes, and the routes to start a hosted crew, follow it and stop it. It is off by default: a daemon must not open a port nobody asked for. (The A2A server is a separate listener, switched on separately — see the service host.)

{
  "Orkeon": {
    "Host": {
      "Api": {
        "Enabled": true,
        "Prefix": "http://localhost:5190/",
        "Security": { "AllowedAuthSchemes": ["ApiKey"], "ApiKeySecretNames": ["ORKEON_HOST_API_KEY"] },
        "ReadinessTags": [],
        "IdempotencyTtl": "24:00:00",
        "IdempotencyMaxKeys": 10000
      }
    }
  }
}
Key Default Meaning
Enabled false Serves the API. Off, the host listens on no port for it.
Prefix http://localhost:5190/ The listener prefix: scheme, host, port and a closing slash. http://+:5190/ listens on every interface.
Security none What a caller presents on every route but the two probes: AllowedAuthSchemes (Bearer, ApiKey), ApiKeySecretNames, RequireMutualTls, TrustedCertificateAuthorities, TrustedClientCertificateThumbprints — the rules of A2A's A2A:Security, enforced by the same code (HttpRequestAuthenticator). ApiKey reads its keys through the secret provider (ORKEON_<NAME>), on every request. Bearer needs a validator: A2A:Security:AzureAD or A2A:Security:Oidc, or an IAuthenticationProvider the host registers.
ReadinessTags [] Health-check tags added to readiness. llm adds the LLM provider check (see below).
SseKeepAliveInterval 00:00:15 How long a quiet event stream waits before it writes a keep-alive frame.
ReadinessCacheTtl 00:00:03 How long a /readyz answer is reused before the checks run again (00:00:00: every request runs them).
MaxEventStreams 64 The most event streams served at once; the next one is refused with 503 too_many_streams and a Retry-After.
IdempotencyTtl 24:00:00 How long an idempotency key is remembered.
IdempotencyMaxKeys 10000 The most keys held in memory.

A prefix beyond the loopback (http://+:…) with no authentication scheme and no RequireMutualTls is refused at start (exit 78): it would run crews for whoever can route to it. So are a prefix without its closing slash, a Bearer scheme without a validator, RequireMutualTls without a trust anchor, and a port another process holds. Under Windows the listener is HTTP.sys: a prefix other than the loopback needs a URL reservation for the service's account (netsh http add urlacl url=http://+:5190/ user=<account>); without it the start is refused with the same exit code and the hint.

Without an ApiKey or Bearer scheme (the loopback default, no key needed, and also RequireMutualTls alone: a browser presents an installed client certificate on its own) the API still refuses what a web page opened in the operator's browser could send: a POST whose Content-Type is not application/json answers 415 unsupported_media_type (a cross-site form cannot set it), and a request whose Host header is not the listener's own (localhost, 127.0.0.1, [::1] on the configured port) answers 421 misdirected_request (DNS rebinding). Send Content-Type: application/json on every POST, /runs/{id}/cancel included. With an ApiKey or Bearer scheme configured, these two checks are not made: a page cannot set the header. The daemon's A2A listener makes the same two checks under the same condition.

Probes

Both answer without authentication — an orchestrator holds no credential — and with a JSON body { "status", "checks": [ { "name", "status" } ] }. A failed check's exception goes to the log, never to the body.

GET /healthz — liveness

Runs the checks tagged live: system_resources (process memory against Telemetry:MaxMemoryMB; over it, degraded, which still answers 200). It reads the process and nothing else: liveness never depends on the LLM, the state store, the run history or any other service, so a provider outage never makes an orchestrator restart a daemon that is fine. 200, also while the host stops (with "status": "stopping", see below): a draining process is alive, and a liveness 503 would have an orchestrator kill it before its runs finish. 503 only when a live check is unhealthy.

GET /readyz — readiness

Runs the checks tagged ready, plus those whose tag ReadinessTags lists. 200 when none is unhealthy, 503 otherwise.

Check What it verifies Reaches
crew_registry At least one crew is hosted and the host still admits runs. memory
state_store The checkpoint store answers a read (IStateStore). Healthy when none is registered. the store
run_history The run history answers a query (IRunHistoryReader). Healthy when the history is off. the history
memory_provider The memory provider answers (IMemoryProvider). the provider
llm_provider Only when ReadinessTags contains llm. The LLM provider answers. the model service

The LLM check is out of readiness by default: a provider's outage takes every run down but not the daemon, and an orchestrator that pulled the host out of rotation would turn a partial failure into a total one. Add "llm" to ReadinessTags when a load balancer should route away from a host that cannot reach its model.

While the host stops, from the moment admission closes — the whole ShutdownGracePeriod — both probes report "status": "stopping": /readyz answers 503, so a load balancer stops sending traffic while the runs in flight finish, and /healthz keeps answering 200, so a liveness probe never restarts the process mid-drain.

curl -s localhost:5190/healthz
curl -s -o /dev/null -w '%{http_code}\n' localhost:5190/readyz

systemd has no HTTP probe: use ExecStartPost to wait for readiness if a script needs it (curl --retry 10 --retry-connrefused -f localhost:5190/readyz), and leave Type=notify as the unit's own readiness. Docker and Kubernetes:

HEALTHCHECK --interval=30s --timeout=3s CMD curl -fs http://localhost:5190/healthz || exit 1
livenessProbe:  { httpGet: { path: /healthz, port: 5190 } }
readinessProbe: { httpGet: { path: /readyz,  port: 5190 } }

(the container's prefix is http://+:5190/, with an authentication scheme; the probes stay open.)

Runs

Every route below needs the credentials of Security. Bodies and answers are JSON, with snake_case names; an error is { "error": "<code>", "message": "…" }.

One id end to end

The id of a run is the one the daemon mints when it admits the run (a ULID, an ExecutionId). It is the run's identity everywhere: the run_id of this API, the A2A task's run, orkeon runs and the run history, the checkpoint session (--resume), the correlationId of every event. run.started repeats it as hostRunId.

POST /runs

{ "crew": "support", "prompt": "What changed?", "variables": { "topic": "elections" }, "idempotency_key": "order-8841" }

crew and prompt are required; variables holds strings only; unknown fields are refused. The body is capped at 1 MiB.

Answer When
202 { "run_id", "idempotent_replay": false } The run was admitted; it goes on in the background.
200 { "run_id", "idempotent_replay": true } The idempotency_key had been seen with the same request: the first request's run.
409 idempotency_conflict The key had been seen with a different request.
400 invalid_request A malformed body. 413 when it is larger than 1 MiB.
404 unknown_crew No hosted crew by that name.
429 at_limit + Retry-After: 5 The crew is at its MaxConcurrentRuns.
503 host_stopping The host is stopping: nothing starts.

GET /runs, GET /runs/{id}, POST /runs/{id}/cancel

GET /runs?crew=&limit= lists the recorded runs, newest first (limit 1 to 500, 50 by default); GET /runs/{id} reads one: run_id, crew, status (running, succeeded, failed, orphaned, waiting_for_input), started_at, finished_at, error, and pending_input (request_id, kind, prompt, choices?) while the run holds or stopped on a question. POST /runs/{id}/cancel answers 202 and stops a running run (the run then reads failed: its run.finished carries exit code 130), 409 run_not_running for a run that is over, 404 unknown_run for an id nobody knows. The reading routes read the run history: with Orkeon:RunHistory:Enabled off they answer 501 run_history_disabled (the daemon enables it by default).

POST /runs/{id}/input

Answers the question a run waits on (needs Orkeon:Checkpointing:Enabled). Body: {"request_id": "...", "answer": "..."}; request_id is optional (the question the run waits on; for a suspended run, the question it stopped on, the one its exit showed). answer is the text (a confirmation is yes or no, several choices a JSON array, a tool approval allow, allow_for_session or deny). The run that holds on the question in this process goes on: 200 {"status": "delivered"}. A run that was suspended (the hold timeout, Orkeon:HumanInput:Pause: Suspend) has the answer recorded: 202 {"status": "recorded"}, and orkeon run <crew> --resume <id> takes it up (the daemon does not resume a run by itself). The same answer given again before the resume (a retried POST) is the same fact: 200 {"status": "already_answered"}. A different answer, or any answer once the question is closed: 409 no_pending_input, 409 request_mismatch for another question than the one asked, 409 request_id_required (the message lists the held ids) when the run holds several questions at once and the body names none, 400 invalid_request without an answer, 404 for an unknown run, 409 input_unavailable when the host keeps no checkpoints. The route goes through the same authorization as the other /runs routes. Orkeon:HumanInput:Pause is Hold by default in the daemon, with a HoldTimeout of 30 minutes (counted from the question; RunTimeout does not run while the run holds, so the hold expiry, which suspends the run, is what ends the wait); the run's history shows input.needed (with requestId) and input.given, and a suspended run ends run.finished status: waiting_for_input, exitCode: 75.

GET /runs/{id}/events

The run's events, from the history, then live until run.finished. With Accept: text/event-stream it is SSE: id: <seq>, event: <kind>, data: <the event, in the wire form of the run event bus>. Otherwise it is NDJSON (application/x-ndjson), one event per line. ?from=<seq> starts at that seq (inclusive); the Last-Event-ID header, which an SSE client sends when it reconnects, resumes after the seq it names and wins over from. seq increases strictly within a run, across a --resume too. While a run is quiet the stream writes a keep-alive frame every SseKeepAliveInterval (an SSE comment : keep-alive, a blank line in NDJSON): the write is what notices a client that went away, so an abandoned stream ends instead of living as long as the run. A client that disconnects ends its stream quietly (logged at Debug). Beyond MaxEventStreams streams at once, the answer is 503 too_many_streams.

Idempotency

idempotency_key (1 to 255 characters) makes POST /runs safe to replay.

  • What is compared. The key, and a SHA-256 of the request (the crew as the host names it, the prompt, the variables sorted by name). Same key and same hash: the first run's id, 200. Same key, another hash: 409.
  • Concurrency. Two requests with the same key that arrive at once start one run: the first reserves the key under a lock before it starts anything, the second waits for that start and answers with the first's run id. A refused attempt (429, 503, 404) releases the key, so the client may retry it.
  • Where keys live. In memory, for IdempotencyTtl (24 hours by default, counted from the run's start), at most IdempotencyMaxKeys — the oldest finished ones go first. The key and the hash are also written into the run.started event of the run, in the run history.
  • Across a restart. When the API starts, the keys of the runs the history holds from the last IdempotencyTtl (newest 1000) are read back, so a replay after a restart still answers 200 on the same run. This lasts as long as the history keeps the run (Orkeon:RunHistory retention) and the TTL has not passed. With the history off, a restart forgets every key, and a replay starts a new run. A history that cannot be read at start costs the keys of before the restart, not the start; a warning says so.
  • A run sent without a key is never deduplicated.

Not in this API

Resuming a suspended run through the API is not here: the answer is recorded and the CLI resumes it. Streaming the run's LLM tokens is not either: the events carry the task and tool progress the event bus publishes.