Reliability is a design choice, in brief
This lesson takes the track’s spine, architecture is deciding where judgment lives, to its human end. Lesson 1 split judgment between the model and the code. Lesson 6 adds the third place it can live: a person. Some calls belong to neither the model nor the structure around it, and the architect’s job is designing when those calls get made and by whom.
The capability: after this lesson, you can write escalation criteria that survive an audit, budget scarce human review where it changes outcomes, and structure review so it catches what a system cannot catch about itself.
What the lesson covers. Escalation criteria that work are explicit, rule-based, and checkable, walked through on an invented insurance claims intake system. Two tempting proxies fail: sentiment-based escalation routes attention to the loudest cases instead of the hardest, and a model’s self-reported confidence is one more generated output, poorly calibrated to real difficulty. Two escalations are never optional: a person who asks for a human gets one immediately, and a situation the policy does not cover goes to a person rather than being improvised.
Human-in-the-loop design is a budget problem: route the low-signal and ambiguous cases to reviewers, and measure accuracy by segment, because a strong aggregate hides a weak segment. Independent review comes next: a fresh model instance catches what self-review misses, because the author carries its own reasoning context. Anthropic’s engineering essay names the workflow shape (evaluator-optimizer, one model generating while another evaluates); the judgment that fresh-instance review beats self-review is Clawdemy’s, applied to code review, extraction checking, and any generated work. Large inputs get multi-pass review, per-item passes plus a separate cross-item pass, because one giant pass dilutes attention.
Why this order. The reader saw a gate with veto power in the AI Agent Teams track. This lesson designs the criteria that feed a gate, right before lesson 7 removes the human from the room entirely.