The Orchestrator Bake-Off

Prove multi-agent beats one agent on a decomposable task — with traces — or admit it doesn't.

Your win for this lesson: a head-to-head result, backed by traces, showing whether an orchestrator + subagents actually beats a single agent on your task — and an honest verdict either way. The rubric's first M3 exercise. Prerequisites: Lesson 0003, Lesson 0008.

1 · Pick a genuinely decomposable task

Multi-agent's one real advantage is context isolation (Lesson 0003), so the task must have independent sub-parts that each benefit from a clean window. Good: "review these 5 documents against a checklist and produce a combined report" (each doc → its own subagent, isolated context). Bad: anything sequential where step N needs step N−1's full reasoning — that's one agent's job, and splitting it just adds coordination cost.

2 · Build both arms

Single agent: your existing loop, all 5 documents in one context, produce the report.
Orchestrator: a lead loop that spawns one subagent call per document (each a fresh messages list — that's the isolation), collects the structured results, and synthesizes the report. Keep the subagent prompt identical across the five.

Instrument both (you'll formalize this in Lesson 0012): log per-arm total tokens, wall-clock, number of model calls, and the final quality score against a rubric you write first.

3 · Run the bake-off

Arm	Quality (/rubric)	Total tokens	Wall-clock	Model calls
Single agent
Orchestrator + 5 subagents

Run each arm 3 times (agents are non-deterministic; one run isn't a result). Score quality blind if you can.

Pass: you have a defensible verdict. Two honest outcomes, both valuable: the orchestrator wins on quality (isolation kept each doc's analysis clean) and you can show the token/latency price you paid for it — or it doesn't win, and you can say so. "It didn't beat the single agent" is a passing result. Manufacturing a multi-agent win the traces don't support is a failing one.

The M3 discipline: most teams reach for multi-agent too early (Anthropic's own account). You just made the decision the way it should always be made — with traces and a named advantage (context isolation), not with a hunch that "more agents = smarter." Every extra agent must earn its coordination cost and multiplied failure surface.

4 · Read the coordination failures

Look at the orchestrator's traces for the failure modes multi-agent adds: a subagent that misread its slice, the synthesizer dropping a subagent's finding, duplicated work across subagents. These don't exist in the single-agent arm — they're the tax you're weighing.

Pass: you can point to at least one coordination-specific issue in the orchestrator traces (or confidently state there were none across 3 runs).

Self-grade

Task was decomposable — independent sub-parts, isolation-justified (not a sequential task forced apart)
Both arms measured — quality + tokens + latency + calls, 3 runs each
Verdict is honest — win or no-win, backed by the numbers; no manufactured multi-agent victory
Coordination cost named — you identified the failure surface multi-agent added, or verified it was clean