Skip to content
Kiberon LabsAgentic Engineering Ebook

Orchestrator

Description

Section titled “Description”

An Orchestrator is the runtime primitive responsible for receiving work, maintaining a durable task registry, and dispatching that work to the correct agent or agent factory. It is the operational counterpart to the higher-level End-to-End Orchestration process: where that process describes what should happen across the delivery lifecycle, the Orchestrator is the component that makes it happen at runtime.

The Orchestrator sits between work intake (issues, triggers, human requests) and the agent layer. It tracks task state, owns the assignment decision, and manages the communication channel through which it delegates and receives progress.

Responsibilities

Section titled “Responsibilities”
  1. Task registry maintains a durable record of in-flight and completed tasks, including which agent or pool is assigned, current status, and a correlation identifier used to join log streams back to the originating task.
  2. Assignment routing decides whether a task should be sent to an already-running agent or whether a new agent must be created. This is the primary decision fork (see Dispatch Modes below).
  3. Progress tracking receives streamed execution logs from the assigned agent and correlates them to the task record, giving the Orchestrator a live view of what is happening without polling.
  4. Lifecycle management handles agent failure, timeout, and retry. For ephemeral agents it can trigger re-creation; for long-running agents it can reconnect or escalate.

Dispatch Modes

Section titled “Dispatch Modes”

1. Long-Running Agent Dispatch

Section titled “1. Long-Running Agent Dispatch”

When the target is a Long-Running Agent for example a Hermes instance, a persistent OpenHands session, or any agent with its own lifecycle the Orchestrator communicates over a persistent, bidirectional channel rather than spawning a new process.

Two integration patterns apply:

SSE / Webhook stream The Orchestrator sends a task payload to the agent’s inbound webhook and immediately opens a Server-Sent Events stream on the agent’s outbound log endpoint. The SSE channel streams tool calls, status updates, and the final result back to the Orchestrator, keyed by the task correlation ID. This is the preferred pattern when the agent exposes an HTTP interface (e.g. Hermes API Server adapter, OpenHands REST API).

MCP tool call Where the agent exposes an MCP server, the Orchestrator acts as an MCP client and issues a tool call to start the task. The MCP protocol’s native progress notifications carry incremental log events back over the same connection until the tool call resolves. This is the preferred pattern when agent-to-agent tool composition is the primary integration model, since request/response correlation and streaming are part of the protocol spec.

The choice between SSE/webhook and MCP is made at pool or cohort configuration time based on what the target agent exposes, not at dispatch time.

2. Agent Factory Dispatch (New Agent Creation)

Section titled “2. Agent Factory Dispatch (New Agent Creation)”

When no suitable running agent exists, the Orchestrator delegates to an Agent Pool to create one. The pool is responsible for scaffold and configuration; the Orchestrator is responsible for triggering creation and establishing the log channel once the agent is live.

Two creation strategies exist:

Local / in-process spawn The agent is started as a subprocess or async task within the Orchestrator’s own runtime. Suitable for lightweight ephemeral agents where latency and isolation requirements are low. Log streams are handled over local IPC (stdin/stdout, Unix socket, or in-process queue).

Network-triggered creation The Orchestrator makes an API call to an external provisioning system for example Coder, a Kubernetes job controller, or a cloud function platform which starts the agent in an isolated environment. The Orchestrator receives a callback or polls for readiness, then establishes the SSE or MCP log channel as in the long-running agent case. This is the required strategy when execution must be auditable, isolated, or billed separately, as described in Execution and Feedback.

Task Correlation

Section titled “Task Correlation”

Every task dispatched by the Orchestrator carries a stable correlation ID. This ID is:

  • Passed to the agent at creation or invocation time
  • Emitted in every log line streamed back to the Orchestrator
  • Used to join Orchestrator-side task records with agent-side execution traces in any observability backend

Correlation IDs make it possible to reconstruct the full delivery path from the originating issue or trigger, through planning and execution, to the final PR or outcome as a single traceable unit.

Durable Execution

Section titled “Durable Execution”

A task registry is not enough on its own. An Orchestrator that loses in-flight state on a crash, double-dispatches on recovery, or stalls forever on a human approval is not production-grade. The field has converged on a durable-execution discipline (the Temporal model) that the Orchestrator should follow.

Determinism on the outside, non-determinism on the inside. The orchestration logic the sequence of stages, the dispatch decisions must be deterministic and replayable. All non-determinism (LLM calls, tool calls, network I/O) is quarantined into side-effect steps (“activities”). This is what lets the Orchestrator replay its own history to recover after a crash without re-deriving a different path. A corollary worth keeping: changing a prompt is safe (it does not change the orchestration sequence); changing the sequence of stages needs a version guard so in-flight runs replay correctly.

Checkpoints are not durability. Persisting state is necessary but insufficient it delegates the hard parts (failure detection, exactly-once recovery, distributed locking) to application code. The Orchestrator needs an actual recovery mechanism: a watchdog/heartbeat that detects a crashed run (see Recovery Triggers), and a lock that prevents two recovery processes double-executing the same task.

Resume from the failure point, not from the beginning. Long agent runs are expensive to restart from scratch. Combined with the scope-small principle (long-horizon work is many short sessions see Runtime Context Management), the Orchestrator should checkpoint per stage and resume mid-pipeline.

Operational hazards to design against

Section titled “Operational hazards to design against”
  • Missing heartbeats during long inference cause spurious timeouts and duplicate model calls. Long-running activities must heartbeat.
  • Unbounded agent loops blow past history/event limits; reset history with preserved state rather than accumulating forever.
  • Large payloads (diffs, code analysis, model outputs) exceed transaction limits and can terminate a run non-recoverably. Use a claim-check pattern store the blob in object storage, pass a reference through the registry (this is the same artifact-store discipline as ACRI log payloads).
  • Human-in-the-loop without timeouts stalls forever. Every approval wait carries a timer and an SLA action (escalate, auto-reject, or re-route) see Controls.
  • Orphaned subagents. Cancelling a parent must define what happens to its children, or terminated parents leave subagents running and billing.
  • Stateful deploys. Agents run continuously and statefully, so a rolling deploy kills in-flight work. Stateful fleets need rainbow/gradual deploys that run old and new versions side by side and shift traffic over.

These hazards map onto the Execution Ledger (the durable record) and Orchestration Triggers (the recovery signals) together they are what make autonomous operation survivable over long time horizons.

Relationship to Other Primitives

Section titled “Relationship to Other Primitives”
  • Agent Pool the Orchestrator calls the pool’s factory interface to create new agents. The pool owns scaffold and config; the Orchestrator owns when and why creation happens.
  • Agent the Orchestrator dispatches to agents and receives log streams from them. It does not own the agent’s internal execution model.
  • Cohort a cohort can define which Orchestrator is authoritative for a set of agents, and can constrain which agents an Orchestrator is permitted to dispatch to. A multi-team delivery structure may have multiple Orchestrators scoped to different cohorts.
  • Agentic Swarm the Orchestrator is the coordination spine that makes a swarm coherent. Without it, agents operate independently; with it, work moves through the swarm as a tracked, recoverable flow.
Section titled “Related Processes”