Skip to content
Clawdemy

References: Should AI do this task?

Source material

Section titled “Source material”

This lesson is an original adaptation. Its two-filter framework, classroom discussion, and quoted staleness caveat come from the sixth class of a Harvard Kennedy School course; the prose, framing, and refreshed examples for Clawdemy readers are our own. Clawdemy is independent of Harvard, which has not reviewed or endorsed this track.

  • The Science and Implications of Generative AI (HKS DPI-681M), Harvard Kennedy School, Spring 2024. Faculty: Sharad Goel, Dan Levy, and Teddy Svoronos. This lesson adapts Class 6: When and how to use generative AI from the Spring 2024 course site, whose content is licensed under Creative Commons Attribution 4.0. The two filters, the four task criteria, the four practical considerations, the good-enough ladder, and the quoted caveat “out of date in a year or maybe even a couple of months” come from this class’s lecture videos.
  • Official course lecture playlist on YouTube, Harvard Kennedy School. The full lectures, free to watch. We mean it when we encourage you to take the original course alongside this track: good teachers deserve more students.
  • Provenance note: quotations and classroom details in this lesson were drawn from the transcripts of the official Class 6 lecture videos in the playlist above, obtained from the official Harvard sources only. No third-party re-uploads or mirrors were used.

The examples behind the lesson

Section titled “The examples behind the lesson”

Per this track’s freshness rule, the lesson keeps the course’s questions and re-sources its examples. Each example below was verified against the primary source named.

  • NYC’s AI Chatbot Tells Businesses to Break the Law, Colin Lecher, The Markup, March 29, 2024, copublished with Documented and THE CITY. The reporting behind the lesson’s opening: the tips answer quoted in the lesson, the cash-free and housing-voucher examples, the observation that the bot “appears authoritative,” and a housing expert’s “dangerously inaccurate” characterization.
  • The Markup’s follow-up report on the chatbot’s shutdown, The Markup, January 30, 2026 (updated February 4, 2026). The source for the lesson’s ending: the disclaimers added after the 2024 reporting, the narrowed scope, the “functionally unusable” quote, and the shutdown. The follow-up also reports the bot cost nearly $600,000 to build.
  • Moffatt v. Air Canada, 2024 BCCRT 149, British Columbia Civil Resolution Tribunal, tribunal member Christopher Rivers, decided February 14, 2024. The decision on CanLII. The lesson’s quoted line is the tribunal’s characterization, at paragraph 27, of the airline’s argument that it was not liable for what its chatbot said. For a legal analysis, see McCarthy Tetrault’s TechLex summary of the decision; for a press account, see CBC News’s coverage of the ruling, February 2024.
  • Hallucinating Law: Legal Mistakes with Large Language Models are Pervasive, Stanford HAI, January 11, 2024, reporting the study by Dahl, Magesh, Suzgun, and Ho. The source of the lesson’s dated finding that widely used models of that era, tested without grounding, gave false or fabricated answers about a court’s core ruling at least 75 percent of the time.
  • Are Emily and Greg More Employable Than Lakisha and Jamal? A Field Experiment on Labor Market Discrimination, Marianne Bertrand and Sendhil Mullainathan, American Economic Review, 2004. The resume audit behind the lesson’s point that human screening is measurably biased: nearly identical resumes under different names drew very different callback rates.
  • Gender, Race, and Intersectional Bias in Resume Screening via Language Model Retrieval, Kyra Wilson and Aylin Caliskan, University of Washington, AAAI/ACM Conference on AI, Ethics, and Society (AIES), 2024. The study behind the lesson’s finding that AI screening models preferred white-associated names roughly 85 percent of the time; the study tested resume screening built on text-embedding retrieval models. A readable summary is at Brookings, April 25, 2025.
  • Fortune’s report on the Deloitte Australia refund, October 7, 2025. The source for the lesson’s closing example: the 237-page review for an Australian government department, the invented citations and a fabricated court quotation found by a University of Sydney researcher, the corrected version with the AI-use disclosure, and the agreement to refund the final installment of the fee on the $290,000 contract.
  • Ideas Are Dimes A Dozen: Large Language Models For Idea Generation In Innovation, Karan Girotra, Lennart Meincke, Christian Terwiesch, and Karl T. Ulrich, Mack Institute working paper, July 2023. The primary study behind the Wall Street Journal test the course cites: idea quality was measured with consumer purchase-intent surveys, the model’s ideas rated somewhat more purchase-worthy on average, and its strongest run came when it was seeded with examples of highly rated ideas.

Going deeper

Section titled “Going deeper”

On this site

Section titled “On this site”

Adjacent topics

Section titled “Adjacent topics”
  • The next lesson takes everything from lessons 2 through 4 and runs it on one real project, end to end.
  • The confident wrong answer this lesson teaches you to price gets its own full treatment near the track’s end, in the arc on risks and what to do about them.