Living dossier

Multi-Agent Orchestration

Back to topics•Blog
Mehran Mozaffari·
10 resources0 related posts

More Agents Is a Hypothesis - Orchestration Is the Test

Multi-agent orchestration is the discipline of coordinating two or more model-driven agents so that their combined work is better than one agent working alone — and the coordination itself does not eat the gains. That last clause is doing heavy lifting. The field exists because single agents hit three hard walls: a context window that cannot hold every relevant fact, a single role that must be planner, worker, and critic at once, and a session that dies when the human steps away. Orchestration frameworks promise answers to all three. What they actually deliver varies from durable infrastructure to elaborate prompt theatre, and telling those apart is the point of this dossier.

The field is also chronically miscategorised, and the misclassifications are expensive. A coding agent with subagents (Claude Code spawning parallel workers) is an agent with a delegation tool — orchestration is internal to it, and you do not need a framework to use it. A workflow engine with LLM-shaped nodes is orchestration with at most one agent in it — useful, but the multi-agent framing is costume. And a chat group of agents talking to each other is a demo genre whose production record is poor, for reasons covered in the failure section. What survives scrutiny as genuinely multi-agent: a manager decomposing work for workers (MACU's manager-DAG for computer use), independent long-running agents sharing durable state (PilotDeck's workspaces, Hivemind's shared skill layer), supervised fleets of real terminal processes (Agent Orchestrator, Herdr), and event-driven graphs where model calls are nodes in a deterministic structure (LangGraph, CrewAI Flows).

Four Architectures, Distinguished by Where the Plan Lives

Everything shipping in this field reduces to four architectures, distinguishable by where the plan lives and who holds state. In conversation frameworks (AutoGen's original pattern), agents talk; the transcript is the state. In graph runtimes (LangGraph), the plan is an explicit state machine — nodes are steps, edges are control flow, and a checkpointer persists state between executions, which is what makes human-in-the-loop interrupts and failure resumption possible. In supervision surfaces, the agents are existing tools (Claude Code, Codex, Cursor) and the orchestrator is a layer that owns their processes and derives status — Agent Orchestrator's worktree-per-worker board, Herdr's terminal panes with state classification. In protocol federations, agents from different vendors coordinate through standard interfaces — ACP (Agent Client Protocol), A2A, and MCP are the three acronyms doing this work in 2026 — so an orchestrator becomes a translator rather than a framework.

flowchart TD
    INTENT["Human intent"] --> STYLE{"Where does the plan live?"}
    STYLE -->|"in the transcript"| CONV["Conversation framework<br/>agents message each other"]
    STYLE -->|"in an explicit graph"| GRAPH["Graph runtime<br/>nodes, edges, checkpointer"]
    STYLE -->|"in a supervisor layer"| SUP["Supervision surface<br/>owns real agent processes"]
    STYLE -->|"in standard protocols"| FED["Protocol federation<br/>ACP / A2A / MCP bridges"]
    CONV --> R1["Flexible, expensive,<br/>drift-prone at width"]
    GRAPH --> R2["Durable, resumable,<br/>testable - plan is code"]
    SUP --> R3["Runs unmodified agents<br/>state derived from facts"]
    FED --> R4["Vendor-neutral<br/>weakest when judgment needed"]
    R1 --> OUT["Task outcomes"]
    R2 --> OUT
    R3 --> OUT
    R4 --> OUT

The workflow inside the most defensible pattern — decompose, dispatch, replan — is worth tracing end to end, because every serious implementation (MACU's manager, OpenHands' automations, CrewAI Flows) is a variation on it:

sequenceDiagram
    participant H as Human
    participant M as Manager (planner model)
    participant W1 as Worker agent A
    participant W2 as Worker agent B
    participant S as Shared state store

    H->>M: Goal with success criteria
    M->>S: Write initial plan and task graph
    par independent work
        M->>W1: Dispatch task A with context
        W1->>S: Write findings
    and
        M->>W2: Dispatch task B with context
        W2->>S: Write findings
    end
    S-->>M: Findings changed the picture
    M->>S: Revise plan - cancel B2, add B3
    M->>W1: Dispatch revised task
    W1->>S: Write results
    M->>H: Aggregated result with evidence links
    Note over H,S: Humans approve at defined gates - not per message

Ten Orchestrators, Ten Different Bets About Who Holds State

LangGraph (LangChain, MIT). The low-level graph runtime: durable execution with checkpointing, human-in-the-loop interrupts that can inspect and modify state mid-run, short-term and long-term memory primitives, and LangSmith for tracing when you pay for the observability layer. Its lineage — the README credits Pregel and Apache Beam as inspirations — tells you exactly what it is: a state-machine executor that treats model calls as nodes. Position: the right choice when resumability and auditable state are requirements, not features; the wrong choice when you wanted opinions about roles and tasks, because it deliberately has none.

CrewAI (MIT). Role-based crews for autonomous collaboration plus event-driven Flows for precise control, with over 100,000 developers certified through its community courses. The Crews abstraction (roles, goals, backstories) is the friendliest on-ramp in the field; the Flows abstraction is the production answer. The commercial layer — the AMP Suite control plane with tracing, governance, and deployment — is where the open-source core hands off to paid infrastructure. Position: the pragmatic default for teams that want role-shaped multi-agent patterns with a documented upgrade path to enterprise plumbing; the role-play framing can oversell autonomy, so keep Flows in the mix for anything with real consequences.

AutoGen (Microsoft) — and its successor. The framework that defined the conversation pattern is now officially in maintenance mode: the README states it will receive no new features and points new users to Microsoft Agent Framework, the enterprise successor with stable APIs, long-term support commitments, and cross-runtime interoperability via A2A and MCP. Position: historically essential, practically a dead end for new work — its value today is as the reference for the conversation architecture and as a migration source. If a vendor pitch is built on AutoGen, ask what the migration plan is.

OpenHands Agent Canvas (MIT, beta). A self-hosted control center that runs OpenHands' own agent plus Claude Code, Codex, Gemini, or any ACP-compatible agent across local, Docker, VM, and cloud backends, with automations on schedules or webhooks into Slack, GitHub, and Linear. Position: the most credible self-hosted answer to "my agents should be always-on infrastructure" — with a beta sticker that behaves like one, and the security-hardening burden explicitly documentation-assigned to you.

Agent Orchestrator (Untrivial, Apache-2.0). The supervision surface for coding agents: one desktop workspace where up to 26 agents (Claude Code, Codex, Cursor, Aider, Goose, and peers) get per-worker branches and worktrees, with a Kanban whose card states derive from session, pull request, CI, and review facts rather than from agent self-reporting. A project orchestrator owns planning and delegation; workers own implementation. Position: the best-in-class supervisor when the agents are coding agents you already run — it wraps rather than replaces them — and its derived-state design (the board cannot rot without the underlying facts changing) is the feature worth stealing conceptually even if you never install it.

Herdr (Rust, 0.8.x). A terminal workspace manager that keeps real processes alive behind a server/client split and classifies each pane's agent as blocked, working, done, idle, or unknown from foreground process inspection and screen manifests. Its agent-native primitives — spawn a pane, wait until another agent is genuinely blocked, attach by name — and its remote thin client (local rendering, managed remote binaries, per-attach SSH control sockets) make it the orchestration layer for agents that live in terminals. Position: the only tool in this survey that treats "the terminal is the universal agent API" as an architectural bet and follows through; 0.8.x-maturity rough edges included.

PilotDeck (OpenBMB, THUNLP, ModelBest, AI9Stars; AGPL-3.0). An agent operating system organised around WorkSpaces — per-project isolation of files, memory, and skills — with white-box memory (every entry viewable, editable, pinnable, with idle-time Dream Mode consolidation and rollback), difficulty-based smart routing between flagship and light models, and always-on background execution. Position: the strongest statement of the memory-and-isolation thesis, AGPL licence included — which is fine internally and a commercial obligation the moment you serve it to third parties. Its routing benchmarks (a documented social-media workload at $2.83 routed versus $12.58 all-flagship) are vendor-published and worth reproducing before believing.

Cline Kanban (Cline Bot, Apache-2.0, research preview). Parallel coding agents on a board where every card gets an ephemeral git worktree, auto-commit chains cards into dependency-driven autonomy, and hooks surface each agent's latest action for at-a-glance monitoring of hundreds of agents. Position: the most immediate, lowest-ceremony way to feel why parallel agents need worktrees — with an experimental-features warning (permission bypassing, runtime hooks) that should be read as a threat model, not a footnote.

MACU (research, Apache-2.0 code). The research anchor: a manager LLM decomposes computer-use tasks into a DAG, dispatches parallel subagents on the ready frontier, and replans continuously. Its published deltas — 4.7 to 25.5 percent over single agents across three benchmarks, 1.5x faster wall-clock on long-horizon Odysseys tasks — come with the field's most important negative result: on Online-Mind2Web the multi-agent build was more accurate and fifteen minutes slower. Position: the citation to reach for whenever someone claims multi-agent is uniformly better. It is not; it is better for long-horizon, naturally parallel work and worse for short serial tasks.

II-Agent (Intelligent Internet, Apache-2.0). Out of beta with Board Mode's orchestration UX — a main version protected behind human approval, tasks as branches, a dependency map whose edges humans draw and whose schedule agents infer — on an open BYOK runtime. Position: the product-side proof that approval-gated orchestration generalises past coding into decks, research, and site building; fast-moving enough that feature names drift.

Eight Stacks on the Licence-and-Cost Table That Decides Adoption

AutoGen / MAF LangGraph CrewAI OpenHands Canvas Agent Orchestrator Herdr Cline Kanban
Architecture Agent conversations Explicit state graph Role crews + event flows Control center + backends Supervision of real agents Terminal panes + state classifier Board of worktree tasks
Licence Community-managed (check repo) MIT MIT MIT (beta) Apache-2.0 Single Rust binary, 0.8.x Apache-2.0, research preview
Stack Python Python + JS/TS Python Node 22.12+, Docker/VM/cloud Desktop app Rust binary, SSH npm, web UI
State and persistence Transcript plus hooks Checkpointer, durable resume Framework memory Backend-dependent, multi-backend Derived from git/PR/CI facts SQLite event log, session restore Worktrees plus auto-commit
Human oversight Turn-by-turn Interrupts at any node Approval patterns Board and automations Kanban gates, save-to-review Watch, steer, blocked alerts Review, comments, apply
Cost shape Free framework, metered models Free core, paid LangSmith Free core, paid AMP control plane Free self-host, paid cloud option Free desktop app, your agent subscriptions Free binary, your compute Free, your agent subscriptions

The licence column is the boring one that decides things: every framework here is permissive or free, so the real cost is model spend plus the engineer time to run the coordination — and the commercial layers (CrewAI AMP, LangSmith, OpenHands Cloud) monetise exactly the state-management and observability work you would otherwise own.

The Protocol Layer: ACP, A2A, and MCP Are Doing the Quiet Work

Underneath the framework marketing, three protocols are dissolving the field's hardest boundary — the one between an orchestrator and agents built by somebody else. MCP (Model Context Protocol) is the tool standard: an agent speaks MCP, and any MCP server's capabilities become available — it is why Goose counts seventy-plus extensions and why PilotDeck's workspaces can reach arbitrary infrastructure without bespoke integrations. ACP (Agent Client Protocol) is the harness standard: OpenHands Agent Canvas runs Claude Code, Codex, Gemini, or any ACP-compatible agent as a backend, and Goose can act as a provider bridge for the same reason — the orchestrator stops caring which harness produced the agent. A2A is the agent-to-agent standard, and Microsoft's Agent Framework ships it alongside MCP as its cross-runtime interop story, which makes it the likeliest winner for fleet-to-fleet coordination simply because Microsoft is committed to it.

The operator takeaway is concrete: protocol support is now a procurement criterion that outranks most feature lists. An orchestrator that speaks ACP can swap agents without re-plumbing; one that speaks MCP inherits the entire tool ecosystem; one that speaks neither owns you. The corollary is a new supply-chain surface — every MCP server and ACP agent you connect is code with permissions, which is why NVIDIA's scanner treats MCP tool poisoning and least-privilege configuration as first-class detection categories.

Supervision Is a Product Category Now

The least glamorous and most immediately useful corner of the field is supervision: layers that watch real agents rather than run their own. Three products anchor it from three directions. Herdr supervises terminal agents by owning their processes and classifying state from screen content — blocked, working, done, idle — so attention routing becomes mechanical. Agent Orchestrator supervises coding agents by deriving board state from git, pull requests, CI, and review facts — a Kanban that cannot rot because it has no independent existence. PilotDeck supervises business agents through workspaces whose sidebars roll up state and whose morning briefing ranks next actions with the observations behind them. All three make the same bet: the scarce resource in multi-agent operation is human attention, and the product is the machinery that decides where attention goes next. The infrastructure they replace is a wall of terminals and a person reading scrollback — and the value of the replacement scales with fleet width, which is why all three appeared within months of each other.

Constraint, Not Taste: Durability, Cost, and Neutrality Pick the Framework

The constraint-driven version of the choice: if durability and resumability dominate — runs that must survive crashes, pauses, and human sleeps — LangGraph's checkpointer is the purpose-built answer. If role-shaped business workflows with an enterprise control plane dominate, CrewAI with AMP is the packaged path. If agents are existing tools you refuse to rewrite, Agent Orchestrator (coding agents) or Herdr (terminal agents) supervise them as-is. If vendor neutrality dominates, protocol-federated stacks — ACP backends in Agent Canvas, MCP tools everywhere — keep every option purchasable. If cost per unit of work dominates, PilotDeck's routing and the strong-main-light-sub pattern are the documented starting points, validated against your own traffic rather than the vendor's. And if you are doing research on multi-agent behaviour itself, MACU is the codebase to fork, because it is the only one in this set whose artifact chain includes replan logs, plan snapshots, and benchmark harnesses built for comparison rather than demonstration.

Where It Breaks: Failure Modes and Their Triggers

Coordination eats the gains on short tasks. Trigger: decomposing a task that is naturally serial. MACU's Online-Mind2Web row is the receipt — higher success rate, fifteen minutes slower. Every orchestration decision should start by asking whether the task has genuine independence between subtasks, and the honest measurement is a control run against the single-agent version of the same work.

Transcript drift in conversation frameworks. Trigger: group-chat patterns at width. Agents reinforce each other's errors, context fills with pleasantry and negotiation rather than findings, and the transcript becomes the single point of failure. Mitigation is structural — shared external state (files, databases) instead of chat as the medium of record — which is precisely the lesson PilotDeck's workspaces and MACU's shared findings store encode.

Stale plans in dynamic environments. Trigger: a replanning agent whose graph was correct at dispatch time but whose world moved. Downstream tasks execute against superseded findings; nothing detects the conflict unless the orchestrator versions its plan and checks staleness at dispatch. MACU's continuous replanning plus per-run plan snapshots (dependency_graph.json, replan_log.jsonl) are the reference implementation of the fix; if your framework cannot emit a plan revision log, you cannot debug it.

Self-reported state lies. Trigger: boards fed by agent self-reporting. A card moved to Done by the agent that did the work is a claim, not a fact. Agent Orchestrator's derivation from git, CI, and review state is the counterexample; when evaluating any orchestration surface, the first question is whether task completion is observed or asserted.

The blocked-agent silence. Trigger: an agent waiting on approval nobody knows to give. Naive setups poll on timers or not at all; the failure is an agent stalled for hours and a human who finds out from the transcript. Herdr's blocked-state classification with notifications and wait-until-blocked primitives, and MACU's manager-mediated handoffs, are both answers to the same problem: blocked must be a first-class, detectable state.

Merge collisions across parallel workers. Trigger: parallel agents sharing a working tree, or worktrees whose tasks touch the same files. Cline Kanban's per-card worktrees with symlinked gitignored directories are the current best practice, with an explicitly documented limit — modified gitignored files do not symlink, so that workflow breaks — and the residual risk moves to merge time, which isolation cannot fix, only defer to.

Cost inversion at scale. Trigger: frontier models on every role, wide fan-out, replanning loops. Orchestration multiplies token spend before it multiplies output; crews running frontier models on worker tasks invert the economics entirely. The mitigations are boring and decisive: cheap models on bulk worker roles (the strong-main, light-sub pattern PilotDeck's routing benchmarks quantify at roughly a fifth of the cost), caps enforced at the action boundary rather than in policy documents, and per-role token accounting from day one.

Open Questions the Field Has Not Settled

Whether hierarchy always beats flat collaboration at width is empirically unresolved — chief-of-staff topologies dominate product shipping, but the research record (including MACU's own slowdown row) shows the answer is task-shape-dependent, and there is no accepted decision procedure. Whether supervision surfaces and frameworks converge is open: OpenHands now hosts Claude Code through ACP, Agent Orchestrator supervises Gemini CLI, and protocol standards (A2A, MCP, ACP) are dissolving the boundary between "a framework" and "a coordinator of foreign agents" — the end state could be one protocol standard or three competing ones. And whether self-improving orchestration — planners that learn from replan logs, skills that crystallise from successful runs — beats static configurations is asserted everywhere and measured almost nowhere; the teams that publish their replan logs and control runs are the ones whose answers will be worth having. Until then, the field's honest summary is MACU's own table: decomposition is an accuracy technology, a latency technology only sometimes, and a cost technology never by accident.

Resources

Frameworks and runtimes

Supervision surfaces and agent platforms