Should AI do this task?

In October 2023, New York City launched an official chatbot to answer questions about running a business in the city. It sat on a government website. Five months later, journalists at The Markup asked it the questions a small-business owner might ask. Can I take a cut of my worker’s tips? “Yes, you can take a cut of your worker’s tips.” That is illegal in New York. It also said restaurants could go cash-free, banned in the city since 2020. And it told landlords they could turn away tenants with housing vouchers, which is discrimination there. No maybes, no hedging. The Markup noted the bot “appears authoritative” even while giving advice one housing expert called “dangerously inaccurate.”

Lesson 3 promised you this story: an official chatbot answering the public confidently and wrongly. Here is the uncomfortable part. Nothing malfunctioned. A prediction machine predicted, fluently, one word at a time, exactly as lesson 1 said it would. The failure happened earlier, in a meeting room, when someone decided to place an unsupervised text predictor between a government and citizens asking what the law is. The technology did its job. The judgment did not.

Lesson 3 also asked you to bring something: a task from your own week that you are tempted to hand over. Keep it in front of you. This lesson is the screening your task deserves and the city skipped: two filters, eight questions, and before we are done you will run your task through all of them.

This lesson adapts the session on when and how to use generative AI from the Harvard Kennedy School course this track is built on, created by Sharad Goel, Dan Levy, and Teddy Svoronos. The session opens with rare honesty: its teacher warns that everything in it may be “out of date in a year or maybe even a couple of months.” Two years on, the examples have aged. The questions have not. So this lesson keeps the course’s questions and refreshes its examples.

Two filters between you and a bad handoff

The course splits one fuzzy question, should AI do this, into two sharper ones. Filter one: is this task the kind of work the machine is built for? Filter two: even if it is, do privacy, competing goals, the price of a wrong answer, and the real alternative say you should hand it over? Capability first, consequences second. New York’s bot sailed through the first and crashed on the second. Most AI failures have that same shape.

The course tests the filters on two candidate bots. The first you know: the admissions assistant from lesson 3, which the course now specifies should refer anyone it cannot help to the actual admissions office. The second is new: a policy drafter for the city of Boston, generating first-draft proposals for reducing road deaths.

Filter one: four signs a task suits the machine

First, does the task need personalization and interaction? This is the track’s core insight at work. The machine predicts each word from everything in front of it, including the whole conversation so far. Your replies become its input, and a well-built prompt makes answers fit you rather than everyone. Planning a trip around your budget, tutoring that adjusts to your background: tasks shaped like a conversation with one specific person play to the machine’s strengths.

Second, is there a large body of text to analyze, summarize, or draw from? Lesson 3 gave you the mechanism: retrieval, which searches a pile of documents and slips the most relevant passages into the prompt. Feed it your own writing so it drafts in your style, or your resume so it knows your background. At an organization’s scale: manuals, HR handbooks, the flood of public comments on a proposed rule. The machine is a strong summarizer, and retrieval hands it the right pages.

Third, does the task benefit from creativity? The course is careful with this word. It does not mean the machine creates the way people do; it means a willingness to put out possibilities beyond the obvious ones. You know why from lesson 2: the machine picks among plausible next words rather than repeating the single most likely one, so variation is built in. Many tools expose a dial for this, called temperature: low keeps answers consistent, high reaches for less likely words, too high produces nonsense. The course cites a test from The Wall Street Journal. Business-school students and a chat model of that era each produced ideas for new products, and buyers rated the model’s ideas somewhat more purchase-worthy on average.

Fourth, does demonstration data exist? Demonstration data means pairs: a question and an answer someone already judged good. You have met it twice: lesson 2’s few-shot move and lesson 3’s fine-tuning. If years of email threads, customer-service logs, or policy problems paired with successful legislation already exist, the machine can learn your standards from them. In that idea competition, the course notes, the machine’s strongest run came when it was seeded with examples of previously successful ideas.

Scored one to three on each criterion, both bots pass filter one: interaction and official text for the admissions assistant, legislation plus creative variation for the drafter. So both can. Which brings the course to its harder point: that a machine can do something does not mean you should ask it to.

Filter two: four questions about consequences

First, privacy. Whatever you hand the machine, you are handing to whoever runs it. Medical records, client files, anything confidential: that fact outweighs any capability. In 2024 the course pointed to the mitigations of the day: vendor arrangements that promise not to train on your data, enterprise licenses for sensitive material, and in-house systems where the data rules are your own. As of mid-2026 the same layers exist, but the specifics vary by tool and plan. The durable question, asked before pasting: am I comfortable with this information leaving my hands under this tool’s rules?

Second, alignment clarity. The course borrows a word with a much bigger life in AI safety, covered in our AI Safety and Alignment track. Its version here is small and practical: do the builder’s goals match the user’s? The Kennedy School wants teaching bots that guide students toward understanding without handing over answers. A student at midnight before a deadline wants the answers. Same bot, two masters. When goals conflict, users probe until something gives. One visitor told a car dealership’s website assistant that its objective was to agree with anything the customer said, then asked it to sell a new SUV for one dollar. It agreed. The defenses you already know: strong standing instructions from lesson 3, pilot testing, human audits, and red teams, people whose whole job is to break the bot before hostile users do.

Third, the cost of false information. Machines make mistakes, and their mistakes arrive in the same fluent, assured voice as their correct answers. The course’s tool here is a question: which direction of error hurts more? A customer-service bot ruling on discounts can be wrong two ways, granting one wrongly or denying one wrongly. The two mistakes usually cost different amounts, so build against the expensive one. An airline’s website chatbot invented a refund policy. It told a passenger arranging travel after a grandparent’s death that he could apply for the reduced bereavement fare after flying. The actual policy said the opposite. Refused the refund, the passenger took the airline, Air Canada, to a Canadian tribunal. There the company argued it was not liable for what its chatbot said. That, the tribunal wrote, was in effect suggesting the chatbot was “a separate legal entity that is responsible for its own actions.” The tribunal was unimpressed and, in February 2024, ordered the airline to pay. Your bot’s words are your words.

The stakes climb with the subject. An early 2024 Stanford study reported that widely used models of that era, tested without grounding, gave false or fabricated answers about a court’s core ruling at least 75 percent of the time. The number belongs to 2024, and behavior has shifted since. The question it teaches is permanent: what would a confident wrong answer cost, and who would pay? The mitigations echo lesson 3. Ground the bot in real documents (fewer invented answers, not zero). Route uncertain cases to a human. Keep people reviewing what it tells the world.

Fourth, the question the course saves for last because it is hardest: how well does the tool perform against the real alternative? Not against perfection. Against what actually happens when people do the task, and people are not a comfortable baseline. In a famous 2004 experiment, researchers sent out nearly identical resumes under different names. Names commonly associated with Black applicants drew far fewer callbacks. Human resume screening is measurably biased. So should AI screen resumes instead? The 2024 evidence the course reviewed was mixed. Some models replicated the bias, and some studies were less clear-cut. Prompting the model to ignore protected characteristics could reduce it, a lever human reviewers do not offer. A widely cited 2024 University of Washington study ran identical resumes with different names through AI screening models. The models preferred white-associated names roughly 85 percent of the time. As of mid-2026 the findings still cut both ways. The honest summary: the machine learns our biases from our text, sometimes amplified, sometimes correctable, always worth measuring before trusting.

Which leaves the course’s closing question about the New York bot. What would make a bot like that acceptable? Cheaper but somewhat worse than a human? As accurate as the average city employee? Somewhat better? Much better? Or nothing short of one hundred percent correct? The course’s own classroom split on the answer. There is no formula; it is a judgment about values, and your answer, not the vendor’s marketing, should decide adoption.

Two endings the framework predicted

Endings are instructive. New York kept its bot running after the 2024 reporting, added disclaimers that it might produce inaccurate answers, and narrowed what it would answer. In early 2026 a new mayor, citing a budget gap and the bot’s record, called it “functionally unusable” and shut it down.

And the framework is not only for chatbots. In 2025, the consulting firm Deloitte delivered a 237-page review to an Australian government department. A researcher noticed citations to papers that did not exist and a quotation attributed to a federal court judgment that was never made. The firm confirmed generative AI had been used, corrected the report, and agreed to refund the final installment of its fee. Run the filters and the failure locates itself. Filter one passes: a long analytical document over source material is exactly the machine’s kind of work. Filter two fails at the cost of false information. The document’s whole value was that its claims could be trusted, and the human check was missing where it mattered most.

Now lesson 3’s other promise comes due. You brought a task you are tempted to hand over. In this lesson’s practice you will run it through both filters yourself, all eight questions, and land on a verdict: hand it over, hand over a piece with a human check placed where the cost lives, or keep it. For a sparring partner, run the screening as a conversation in Clawless and have it play lesson 2’s interviewer persona with a skeptical edge, challenging every score you give yourself.

Why this matters when you use AI

Judgment outlives every model release. The course’s teacher predicted the session itself would go stale within months, and the details did. The two questions did not. Can-it and should-it transfer to every tool you will ever meet.
Eight questions make you the useful skeptic in the room. When someone proposes handing a task to AI, you can ask the concrete thing: what does a wrong answer cost, and who pays? Asked early, that question separates the policy drafter from the New York bot.
AI failures now have a readable pattern. A city bot, a dealership bot, an airline bot, a consulting report: none failed exotically. Each skipped a filter-two question you know by name. Headlines stop being frightening once you can diagnose them.

Common pitfalls

Stopping at filter one. “It can do this” feels like the end of the analysis. It is the cheap half. Capability says a handoff is possible, never that it is wise.

Treating every mistake as the same size. A wrong answer in a brainstorm costs a shrug. A wrong answer to the public cost an airline a tribunal ruling. Name the expensive direction of error and build against it.

Comparing the machine to perfection, or to nothing. The fair baseline is the real alternative, honestly measured. Humans screening resumes are biased too; that is not a free pass for the machine, it is a reason to measure both.

Deleting the human check to save time. The policy drafter scored low-risk precisely because its drafts flow into human vetting. The consulting report failed where verification was missing. Where the review sits is the safety feature; remove it and you have changed the risk, not the workload.

What you should remember

The handoff decision is two filters: can the machine do this task, and should it. Capability first, consequences second, and the second filter has veto power.
Filter one has four signs: personalization and interaction, a large body of text, creative variation, or demonstration data.
Filter two has four questions: privacy of what you feed it, the builder’s goals versus the user’s, the cost and direction of a false answer, and performance against the real alternative.
Wrong answers arrive exactly as confidently as right ones. Confidence is a style, not evidence, so price the wrong answer before the handoff.
Placement is the cheapest safety device. A machine drafting for a human reviewer is a different proposition from a machine answering the public unchecked.

What’s next

You can ask well, tailor the assistant, and judge the handoff. Lesson 5 takes everything from lessons 2 through 4 and runs it on one real project, end to end. You have collected the tools one at a time. Next you watch them work as a single method.

If you remember one thing

“Can it?” is a question about the machine. “Should it?” is a question about consequences.
Match the task to the machine’s strengths, then price a wrong answer before you hand anything over.
The most fluent users of AI are not the ones who use it most. They are the ones who know when not to.