Traces Before Prompts

M3 in one move: when an agent fails, you read the trajectory — and every architecture choice is a context decision at system scale.

Your win for this lesson: know your first three moves when an agent fails (trace → failure taxonomy → eval), and be able to argue against adding an agent. Those instincts are the M3 gate on paper. (Prerequisites: Lesson 0001, Lesson 0002.)

1 · From M2 to M3

M2 taught you to design one agent well. M3 is what happens when agents run as systems: architectures, lifecycles, failure modes — and the observability and evaluation machinery that catches regressions before users do.

2 · Single-agent is the best default

One loop, one context: simplest, cheapest, easiest to debug. The only load-bearing reason to go multi-agent is context isolation — an orchestrator decomposes work and hands each subagent a clean, focused window (M1's lesson, applied at system scale). It is not "more brains." ⚠️ Most teams reach for multi-agent too early: every added agent multiplies failure surface, coordination cost, and tokens. Anthropic's multi-agent research system is the honest reference: their early agents spawned 50 subagents for simple queries and distracted each other with excessive updates — the wins came from disciplined decomposition, not from agent count. Justify every extra agent, with traces.

3 · A run is a lifecycle, not a call

Agent state is the full picture of a run: goal, history, memory, pending tools. The lifecycle is start → plan → act → observe → loop → terminate — plus the unglamorous parts that make it reliable: retries, timeouts, stop conditions. Long runs need durability: checkpointing and resuming, the problem workflow runtimes like Temporal exist to solve. You already operate this: Vernant's pipeline runs on Temporal precisely so a crashed run resumes at the failed step instead of restarting.

4 · Agents fail silently and confidently

The M3 failure taxonomy: hallucination, looping, context overflow, tool misuse, goal drift, and silent partial failure. ⚠️ The dangerous ones don't crash — they return plausible-looking wrong answers. Crashes page someone; a confident wrong answer ships. Self-correction patterns (reflection, verification steps, critic/validator agents) exist to convert silent failures into caught ones — and their value is measurable: count the silent wrong answers before and after.

5 · Observability and evals are the floor, not the polish

Tracing captures the full step-by-step trajectory of a run — every model call, tool invocation, and retrieval step with inputs, outputs, and timing (tools like Langfuse, Arize, and Opik do this; the discipline matters more than the vendor). You debug agents by reading traces, not by staring at prompts. Evals score both trajectories and outcomes — ⚠️ a right answer via a broken path will break at scale, so judging only final answers hides rot. Husain's essay is the practitioner canon: failed AI products share one root cause — no eval system. Where you use an LLM as judge, measure its agreement with humans before trusting it, and keep recalibrating.

The M3 load-bearing sentence: agents fail silently and confidently, so observability comes before debugging and evals come before shipping — and when you do scale the architecture, you're managing context windows, not adding brains.

6 · Retrieval quiz — closed book

Answer from memory. Click an option for instant feedback.

1. You're designing a new agent system. Best default architecture?

One loop, one context — simplest and easiest to debug. Everything else must earn its place with evidence.

2. The main legitimate reason to split one agent into several?

Context isolation is the reason — each subagent gets a clean, focused window. "More brains" is the myth; parallelism is sometimes a bonus, never the justification.

3. Your agent produced a confident, wrong answer in production. First move?

You debug agents by reading traces, not staring at prompts. The trace tells you which step went wrong; every other move is a guess.

4. An agent got the right answer via a broken path. Why does it matter?

Eval on trajectories, not just outcomes — a right answer via a broken path will break at scale. Outcome-only scoring hides the rot.

5. In production, the most dangerous agent failures are…

Agents fail silently and confidently. Crashes get caught by machinery; a plausible wrong answer sails through to the user.

6. Per the M3 intuition: every architecture decision is really a decision about…

Every architecture decision is a context-management decision in disguise — orchestrator/subagent exists to isolate windows. M1's lesson never stops applying.

7 · Diagnosis plan — the gate skill

The M3 signal is debugging from traces alone and building evals that catch regressions. On paper: you inherit a multi-step agent that "mostly works" but users report occasional nonsense answers. Write: what your traces must capture, the first three failure modes you'd hunt and why, and the eval set (trajectory + outcome checks) that would catch a planted regression. Then reveal the rubric and self-grade. Bring it to the next session.

A passing plan covers all four:

Traces first — named what the trajectory must record (every model call, tool call, retrieval, inputs/outputs, timing), before any prompt speculation
Silent failures prioritized — the hunt list targets plausible wrong answers (hallucination, goal drift, silent partial failure), not just crashes
Both eval levels — trajectory checks (did it take a sane path?) and outcome checks (was the answer right?), with a planted regression as the test of the tests
Architecture challenged — asked whether fewer agents (or one) would shrink the failure surface, and demanded trace evidence for any extra agent

Hands-on from here: the orchestrator-vs-single-agent bake-off (settled with traces), full tracing instrumentation with a planted-regression eval, and a critic/verifier step with a measured reduction in silent wrong answers. The sister M4 lesson, Own the Feedback Loop, takes these instruments into production.