When Claude is
confidently wrong

Editing makes bland drafts sharp. This chapter is about the harder failure, the one that actually burns people: when the model is not bland but wrong, and wrong in a calm, fluent, completely convincing way. It will happen to you. The question is whether you catch it.

This is the chapter the hype skips. AI does not fail by sputtering and admitting it is unsure. It fails by inventing a fact, a quote, a number, or a source and presenting it with exactly the same confidence as the true ones. The tone never changes. That is what makes it dangerous, and that is why this single skill separates people who use AI well from people who get embarrassed by it.

Here is one that happened to me, so you know I am not being dramatic. I was deep in a long session building this very website, hours in, going well. Then, in a perfectly friendly summary, Claude got my name wrong. It kept my first name and handed me a brand new surname, with total confidence, as if it had always been mine. And it was not random: a word from the project had been floating around in our conversation, so it grabbed that, bolted a name ending onto it, and served it up as my last name without a flicker of doubt.

That is the whole lesson in a harmless package. Nothing broke. It did not malfunction. It did exactly what these tools do: when it does not actually have something, it builds the most plausible-looking version from whatever is lying around and presents it as fact. My name was easy to catch, I know my own name. The dangerous version is when it does the identical thing with a number, a date, or a source you cannot check at a glance. Same mechanism, same confident tone, much harder to spot.

Hold this idea, because it is the core of the whole course. Confidence and correctness are two separate dials. In a person, they tend to move together: someone unsure usually sounds unsure. In an AI, they are unhooked. It can be fluent and certain and totally wrong, with no tell in the voice. So you can never use how sure it sounds as a signal for whether it is right. You have to check the thing itself.

You do not have to fact-check every sentence, that would defeat the point. You learn where invention clusters, and you check there. These are the four places a confident answer is most likely to be quietly fabricated.

Here is the practical move, because checking everything is impossible and checking nothing is reckless. Find the one load-bearing thing, the single fact or claim that, if it is wrong, the whole output is wrong, and verify just that. Most outputs have exactly one. Protect it.

And you can recruit the model into its own honesty. Paste this instruction in, and it stops papering over uncertainty with confident prose. This is your honesty instruction, a keeper for the artifact pack at the end of the course.

The hype tells you AI is almost always right. The honest version is more useful: it is often right, it is never unsure, and telling those two apart is your job. Now you can.