Practice: Industry perspective

Self-check

Seven short questions. Answer each before opening the collapsible.

1. State the arc of the track in one sentence and name the through-line.

Show answer

The arc is demo to production-grade application, end to end: Phase 1 ships the minimum app and treats prompts as engineering; Phase 2 pushes past the closed corpus, designs the interface, and adds operational discipline; Phase 3 surveys field directions, deep-dives the fine-tune and agent points on the build spectrum, and closes with industry perspective. The through-line is lesson 2’s three productive limits (context, cost, latency) + lesson 7’s five engineering pillars (observability, eval-in-production, prompt versioning, cost-and-latency monitoring, regression testing), which carry forward into anything you build next.

2. What are the three rules for reading a fireside chat well?

Show answer

(1) Attribute, do not absorb: when the speaker says “I think X,” the lesson notes “the speaker thinks X.” Don’t adopt fast-moving opinions as durable canon. (2) Separate durable bets from speaker bets: durable bets are things the field has converged on across many sources; speaker bets are one practitioner’s view on an unsettled question. Act on the first; treat the second as questions to ask of your own product. (3) Use the chat as a question generator: the most valuable output is the questions the speaker thinks builders should be asking now, not predictions.

3. State the five durable bets the field has converged on, and one implication for the reader.

Show answer

(1) Base models keep getting better and cheaper per token. Implication: do not over-optimize for the current model; build to swap. (2) Evaluation is the moat, not the model. Implication: build a real held-out eval set for your application before you build clever prompts. (3) The interaction surface keeps expanding. Implication: what you can build is constrained more by design taste than raw capability. (4) Most teams should not train their own model. Implication: hosted for user-facing layer; fine-tune specific high-volume inner sub-tasks; train-from-scratch is Track 15 territory. (5) Operational discipline beats clever architecture. Implication: ship lesson 7’s discipline before lesson 4 or 10’s cleverness.

4. Why does the lesson treat industry-fireside content differently from the rest of the track?

Show answer

The rest of the track teaches engineering practices the field has broadly converged on (prompt versioning, RAG, LLMOps, the agent loop). A fireside chat is a primary source of industry perspective: one experienced practitioner speaking in their own voice, on a specific date, about questions where there is no settled answer. It is valuable for what it surfaces (right questions, live tensions) and limited in the same ways (opinions are opinions, predictions age fast, a single speaker is not the field). The track-close synthesis is the load-bearing pedagogical move; the fireside is the prompt for that synthesis, not the canon.

5. What are the three concrete moves for the reader, post-track?

Show answer

(1) Ship the smallest version of your application that includes lesson 7’s discipline (prompt versioning, held-out evals, basic logging, visible per-call cost and latency). That bar is the difference between a demo and an application. (2) Pick one of the five durable bets and let it shape one decision this month. For most builders, “evaluation is the moat” is the right pick: a focused week building a real held-out eval set has compounding return. (3) Read the fireside (and one other industry source) and write down the three questions it raises about your product, then ask them in the next planning meeting.

6. State a “speaker view” the lesson named as worth taking seriously (but as a view), and why it is not on the durable-bets list.

Show answer

Example: “How fast should you build before the next model release surpasses your scaffolding?” This is a real and important production question that experienced practitioners hold different views on. It is not on the durable-bets list because the field has not converged on an answer (the right answer depends on how durable your scaffolding is, how easy to remove, your product’s release cadence, and your willingness to rebuild). The reader’s job is to ask the question of their own product, not to adopt a speaker’s position on it. Other examples in this class: “what does an LLM-first product feel like,” “where in the stack does the moat actually live,” “what is the right level of agent autonomy for production.”

7. Why does the lesson explicitly hold out contested debates about agent autonomy, alignment, safety, and wider AI policy as out of scope?

Show answer

Because the capstone’s job is engineering-side synthesis + careful read of a primary source, not policy adoption. The same discipline that lessons 6, 7, 9, and 10 of this track applied (technical-not-policy, technical-not-legal, technical-primer) continues through the capstone. The contested debates are real and important, but they live in their own forum with the right stakeholders (legal, policy, ethics, security); they are not what a production-shipping track teaches, and a fireside chat does not retroactively license absorbing them as canon. The reader can take questions home; the lesson does not adopt positions.

Try it yourself: the track-arc synthesis

About 10 minutes, no code. Synthesize what you learned.

Part A: arc-trace. For each phase of the track, name (a) the central question of the phase, (b) the one lesson you would re-read first if you could only re-read one, and (c) the one practice from that phase you will use most often.

Phase 1 (lessons 1-3): ship a minimum app + prompts as engineering
Phase 2 (lessons 4-7): augmentation + UX + LLMOps
Phase 3 (lessons 8-11): field directions + deep dives + capstone

What you’ll get (an example, not the canonical answer)

This is a personal synthesis exercise; the right answer is the one your product makes you arrive at. An illustrative example:

Phase 1: Central question: “what is the simplest LLM-powered thing that produces real value for a user?” Re-read first: lesson 2 (the three properties + three productive limits are the lens for every later decision). Most-used practice: prompt versioning with a held-out test set from lesson 3.
Phase 2: Central question: “what does it take to turn that thing into an application real users trust?” Re-read first: lesson 7 (LLMOps is the gap between demo and product). Most-used practice: trajectory-level logging from lesson 7 extended into lesson 10’s agent context.
Phase 3: Central question: “where are the durable bets, where are the live tensions, and what should I build next?” Re-read first: lesson 8 (the build-vs-buy spectrum and mix architecture frame every later choice). Most-used practice: the three-tests rule from lesson 10 (most agent ideas should not be agents).

Your version of this will be different; that is the point. Write yours down somewhere durable.

Part B: a durable bet to act on. Pick one of the five durable bets that most applies to your current or next project. Write the bet, name the specific change you will make this month, and name the lesson(s) from this track you will re-read while making it.

What strong answers look like

A strong answer is specific and small. Not “I will improve evaluation” but “I will build a 50-example held-out eval set for the support-classifier feature, run it after every prompt change, and chart its pass rate over the next four weeks.” Not “I will swap models” but “I will move the prompt-template and tool-schema definitions out of inline strings and into a versioned config file so a model swap is a one-line change.” Not “I will adopt the mix architecture” but “I will fine-tune a 7B model for the inner classifier in the pipeline and keep the user-facing summarizer on the frontier hosted API.”

The five bets are general; the action you take is specific. Lessons to re-read while making each move are named at the bottom of the lesson body; pick the one or two most relevant to your specific change.

Part C (reasoning). Why is the capstone framed as synthesis + a careful read of a primary source, rather than a forecast of where the field is going?

What you should notice

Because forecasts age fast and the reader needs durable framing, not perishable predictions. A track that closes with “models will do X by date Y” stops being useful the day model Z ships. A track that closes with “here is the arc, here are the durable bets, here are the questions worth asking your own product about” stays useful indefinitely, even as specific predictions are overtaken. The fireside is the prompt for the synthesis (a primary source of live thinking from an experienced practitioner) but the synthesis is what the reader takes home. The same discipline this track applied to lessons 6, 7, 9, and 10 (technical-primer scope; broader debates out of scope) carries through; the capstone does not retroactively license absorbing fireside opinions as canon.

Flashcards

Nine cards. Click any card to reveal the answer. Use the Print flashcards button to lay the set out one card per page for offline review.

Q. The arc of Track 21 in one sentence?

Demo to production-grade application, end to end. Phase 1 ships the minimum app + treats prompts as engineering; Phase 2 pushes past closed corpus + designs the interface + adds operational discipline; Phase 3 surveys directions + deep-dives fine-tune and agents + closes with industry perspective.

Q. The through-line that carries forward into anything you build next?

Lesson 2’s three productive limits (context, cost, latency) + lesson 7’s five engineering pillars (observability, eval-in-production, prompt versioning, cost-and-latency monitoring, regression testing). These are the lens and the discipline for every future LLM-application decision.

Q. The three rules for reading a fireside chat well?

(1) Attribute, do not absorb (speaker view ≠ canon). (2) Separate durable bets from speaker bets (field-converged vs one-practitioner view; act on first, ask questions about second). (3) Use the chat as a question generator (the questions builders should be asking now, not predictions).

Q. The five durable bets the field has converged on?

(1) Base models keep getting better and cheaper. (2) Evaluation is the moat, not the model. (3) The interaction surface keeps expanding (tools, agents, multimodal, long-context, structured outputs). (4) Most teams should not train their own model. (5) Operational discipline beats clever architecture.

Q. The three concrete reader-moves post-track?

(1) Ship the smallest version with lesson 7’s discipline (prompt versioning, held-out evals, basic logs, visible cost/latency). (2) Pick one durable bet and let it shape one decision this month (usually: build a real eval set). (3) Read the fireside + write down the three questions it raises about your product; ask them in next planning.

Q. Why is industry-fireside content treated differently from the rest of the track?

Rest of track teaches engineering practices the field has converged on. Fireside is a primary source of industry perspective: one practitioner’s view on a specific date about unsettled questions. Valuable for what it surfaces; limited in the same way. Synthesis is the pedagogical move; the chat is the prompt for it, not the canon.

Q. What is a 'speaker view' vs a 'durable bet'?

Speaker view: one practitioner’s position on a question the field has NOT converged on (e.g., “how fast should you build before the next model release surpasses your scaffolding”). Ask this of your own product. Durable bet: field-converged observation many sources confirm (e.g., “evaluation is the moat”). Act on it directly.

Q. Why is the capstone framed as synthesis + primary-source read, not as a forecast?

Forecasts age fast and the reader needs durable framing. “Models will do X by Y” stops being useful the day model Z ships. “Here is the arc, the durable bets, the questions to ask your product” stays useful indefinitely. Same discipline as L6/L7/L9/L10: technical-primer scope, broader debates out of scope.

Q. What's out of scope in this capstone, and why?

Any framing that treats fireside opinions as canon; predictions of specific model capabilities or release dates; contested debates about agent autonomy, alignment, safety, or wider AI policy. Different forum, different stakeholders (legal, policy, ethics, security). The fireside does not retroactively license adopting those debates as canon.