References: Reliability is a design choice

Source material

This lesson’s workflow-level review patterns are drawn from Anthropic’s public engineering writing, quoted verbatim where marked. The specific design judgments (that sentiment and self-reported confidence are unreliable escalation proxies, that segment-level accuracy beats one aggregate, and that a fresh instance catches what self-review misses) are Clawdemy’s own position, drawn from general engineering experience with these systems, and the Meridian claims-intake example is invented for teaching.

• Building Effective AI Agents, Anthropic engineering blog, December 19, 2024. The source of three ideas this lesson leans on. On human checkpoints: “Agents can then pause for human feedback at checkpoints or when encountering blockers.” On the review workflow: “In the evaluator-optimizer workflow, one LLM call generates a response while another provides evaluation and feedback in a loop,” most effective “when we have clear evaluation criteria, and when iterative refinement provides measurable value.” And among its parallelization examples, a guardrail where one model instance handles user queries while a second instance screens them: the checker is not the doer, stated as a pattern.
• How we built our multi-agent research system, Anthropic engineering blog, June 2025. The evaluation half of this lesson’s argument, from a team running agents in production: rubric-based judging of agent outputs, and the reminder this lesson quotes, “Human evaluation catches what automation misses.” People find the edge cases the metrics were not measuring.
• Configure permissions, Claude Agent SDK documentation. The mechanical seam for a human checkpoint, in current, canonical form: declarative allow and deny rules decide what runs automatically, and everything else falls through to a runtime callback where an application can put a human approval in front of the action.

Going deeper

• Anthropic Academy, free official courses. “Building with the Claude API” covers the full platform this track builds on, and “Introduction to subagents” is the hands-on companion to this lesson’s fresh-instance review idea: subagents get clean context by construction.

On this site

• What could go wrong: the risk gate (Track 25). The gate this lesson kept pointing back to: a separate role with real veto power, read in working code. That lesson showed the gate; this one designed the criteria that feed it.
• Anatomy of an AI Agent Team (Track 25). The full read-a-real-system track, including bounded review rotations and the deliberate disagreement this lesson’s independent-review principle echoes.

Adjacent topics

• The previous lesson in this track, on orchestration, is where the seams in this lesson’s multi-pass review come from: each pass is a step with one job, and the handoffs between passes are designed, not accidental.
• The next lesson removes the human from the room: agents in CI/CD pipelines, working headless, where judgment written down in advance is the only human presence left.