Learning / Agentic Operations / Lesson 0016

Agentic Operations · Lesson 0016 · Level 4 · Operational Leadership · Capstone → gate

The Full Improvement Loop

Ownership means owning the loop. Run it once, end to end — that's the last thing to prove.

Your win for this lesson: one complete improvement cycle — catch a live regression, ship a gated fix, and prove your eval suite would have blocked the bad version — the rubric's third M4 exercise, and the whole map's final proof. Prerequisites: everything, especially 0012 (evals), 0014 (SLOs), 0015 (gate).

The loop, run once for real

The M4 improvement loop is observe → evaluate → hypothesize → change → re-evaluate, run continuously (Lesson 0004). You'll execute one full turn of it — the way you'd run it forever in production.

Observe. Introduce a "live regression" the way reality would: bump the model version, or change a prompt, or swap a tool — something plausibly innocent. Let your 0014 dashboard surface the damage: an SLO drops (success rate falls, or escalation spikes). You find it because you're watching the metrics, not because a user complained.
Evaluate. Reach for the traces (0012), not the prompt. Read the failing runs and locate the faulting step. This is the M3 reflex serving the M4 loop.
Hypothesize. State what you think broke and why, as a falsifiable claim ("the new prompt made the agent skip the verifier on short inputs").
Change. Ship the fix — through your 0015 guardrails/gate, not around them. The fix is a normal gated change, not a hero patch.
Re-evaluate. Run the eval suite (0012). It must go green again — and, critically, add a test case that reproduces this regression, so the suite now catches this class forever. Prove it: run the new suite against the bad version and watch it go red.

Pass: you closed the loop unassisted — spotted the drift on the dashboard, diagnosed from traces, shipped a gated fix, and left the eval suite strictly stronger than before (it now blocks the bad version you started with). Prove the last part explicitly: the enhanced suite is red on the regression and green on the fix.

The eval suite is your ground truth. ⚠️ The model will change under you — new versions silently shift behavior (Lesson 0004). The one stable thing you own is the suite, and it just got stronger. That's what "owning the loop" means: not shipping the current build, but running the machine that keeps every future build honest.

The M4 readiness check — and the whole arc

Bring to your teaching agent: your SLO dashboard for a live agent (0014) and a complete observe→fix→verify cycle you ran independently (this lesson). Those are the literal M4 gate — and passing it is the top of the map.

The doc's own signals that you're ready for ownership — the check probes each:

You reach for a trace before a prompt when something breaks
You can say "no" to adding an agent or a bigger model, with a cost/reliability argument
You treat the eval suite as ground truth when a model version shifts behavior
You design human checkpoints by risk, balancing safety against throughput — without being asked
You close your own improvement loops — spotting drift, shipping gated fixes, proving the fix held

Demonstrate these and you've reached Head-of-Agentic-Operations capability: the destination of the whole knowledge map.

The map is complete

Sixteen lessons: four mental-model spines (0001–0004) and twelve hands-on proofs (0005–0016), one arc per milestone, each gated on demonstrated capability. You started at "an LLM predicts the next token" and end owning the loop that keeps a production agent honest as the model shifts beneath it. There's nothing above this in the rubric — from here it's depth, scale, and the judgment that only real operational reps build. That's the wisdom the method always said comes from practice, not pages.