Practice: Reliability is a design choice
Self-check
Section titled “Self-check”Six short questions. Answer each in your head before opening the collapsible. Active retrieval is where the learning sticks.
1. What is wrong with the escalation rule “escalate when you are unsure”?
Show answer
It delegates the escalation decision to the exact component whose judgment you were worried about, and it can never be audited. When a case that should have escalated did not, there is no rule to point at, no threshold to adjust, nothing to check the behavior against. Working criteria are explicit, rule-based, and checkable after the fact.
2. Why is sentiment a poor escalation signal?
Show answer
Emotion is poorly correlated with difficulty. An angry person with a routine, fully documented case does not need expert review; a polite person whose file quietly contradicts itself does. Sentiment-based escalation spends scarce human attention on the loudest cases instead of the hardest ones, and teaches persistent users that anger is the price of service.
3. Why can’t you trust a model’s self-reported confidence score?
Show answer
Because the score is one more generated output, produced by the same process that produced the answer, not a measurement taken from outside it. It is poorly calibrated to real difficulty: high confidence on a subtly contradictory file, hedging on a trivial one. Route on observable signals instead: schema validation failures, disagreement between passes, missing fields, historically weak document types.
4. Which two escalations are never optional, in any system?
Show answer
First, an explicit request for a human is honored immediately, with no negotiation and no one-more-try. Second, when the policy is ambiguous or silent, the system escalates rather than improvises, because a model will fill any gap with fluent, confident text, and filling a policy gap is judgment that belongs to a person.
5. A system reports 96 percent accuracy overall. Why is that number not enough?
Show answer
Because a strong aggregate can hide a weak segment. If handwritten witness statements fail 29 percent of the time but make up a small share of volume, they barely dent the mean. Measure accuracy by segment, route the weak segments to human review, and keep a small random sample even in strong segments so the statistics stay honest.
6. Why does a fresh model instance catch errors that self-review misses?
Show answer
The authoring instance carries its entire reasoning history in its context window, so it re-reads the work through the same assumptions that produced the mistakes, the way you cannot proofread your own essay minutes after writing it. A fresh instance has no investment in the conclusions and reads what is actually on the page. The checker should not be the doer.
Try it yourself
Section titled “Try it yourself”Design exercise: write the escalation criteria.
Pick one real task from your own work that an agentic system could plausibly handle: processing a kind of form, screening a kind of submission, drafting a kind of document. On paper or in a note, work through four steps:
- Write five explicit escalation rules for it, in the style of the lesson’s list. Every rule must be checkable: someone auditing a case later should be able to see whether the rule fired. Ban the words “unsure,” “confident,” and “seems” from your list.
- Mark the two never-optional rules. Where does “asks for a person” show up in your task, and what does “the policy is silent” look like there?
- Name your segments. Split the task’s inputs into three or four kinds (by format, source, or difficulty) and guess which segment would be weakest. That segment is where your human review budget goes first.
- Design the review. What would a fresh instance check your system’s output against, and if the input is large, what would the per-item passes verify and what would the cross-item pass compare?
Keep the sheet. In the capstone you will design a full system, and the escalation section will lift straight from this exercise.
Flashcards
Section titled “Flashcards”Q. What makes an escalation criterion one that actually works?
It is explicit, rule-based, and checkable after the fact. An auditor can see which rule fired on any escalated case, and a debugger can see which rule was too narrow when a case slipped through.
Q. Why is 'escalate when unsure' a blank check?
It delegates the escalation decision to the exact component whose judgment you were worried about, and there is nothing to audit it against.
Q. What is wrong with sentiment-based escalation?
Emotion is poorly correlated with difficulty. It routes scarce human attention to the loudest cases instead of the hardest ones.
Q. Why is a model's self-reported confidence score unreliable?
It is one more generated output, produced by the same process it claims to measure, and poorly calibrated to real difficulty. It has no outside.
Q. What are the two never-optional escalations?
An explicit request for a human is honored immediately. And when the policy is ambiguous or silent, escalate rather than improvise.
Q. What escalation signals should replace vibes and self-report?
Observable ones, measured by code from outside the model: validation failures, disagreement between passes, missing required fields, historically weak document types.
Q. Why measure accuracy by segment instead of one aggregate?
A strong average hides a weak segment. Ninety-six percent overall can contain a segment failing a third of the time, because rare segments barely dent the mean.
Q. Where should scarce human reviewer attention go?
To the cases where it changes outcomes: low-signal cases, ambiguous ones, and historically weak segments. Plus a small random sample of strong segments, to keep calibration honest.
Q. Why does a fresh model instance beat self-review?
The author carries its own reasoning context and rarely questions its own decisions; it sees what it meant. A clean-context instance reads what is actually on the page.
Q. What is the evaluator-optimizer workflow?
One model call generates a response while another provides evaluation and feedback in a loop. It works best with clear evaluation criteria and when iterative refinement adds measurable value.
Q. How do you review an input too large for one pass?
Per-item passes plus a separate cross-item pass. Per-item passes catch local errors; the cross-item pass catches contradictions, which live between documents, not inside one.
Q. Why should escalation never be punished in your metrics?
Punish it and the system learns to improvise instead, converting visible handoffs into invisible errors. An escalation on an ambiguous case is the design working.