Flawed Claude
ACTUALITY: Adventures in AI, in which Claude admits to “talking out of my ass”
HERE’S SOME AMMUNITION IF YOU’RE in that chunk of the pie chart marked “AI Haters & Copyright Warriors”. It will feather-bed your righteous Luddite mistrust of all things AI, and at one point you’ll likely say “Whaaaat the fuuuuuuuuck?” as you giggle at the absurdity.
Some context, on me. I like AI. I like how it’s given me answers I could not get elsewhere. I like how it hugely accelerates the parts of my day-job that involve coding. Most of all I like how I can converse with it to steer the thinking to the best conclusion – unlike, say, a Google search. I do not like its entitled culture of copyright theft, for which it is rightly criticised and those ass-hats in charge can get bent and pay their damn bills. But whatever… I’m interested in where it’s all going, and I’m on the “this is going to change everything” end of the see-saw.
(If you know how AI works and just want to get to the meat, skip the next paragraph.)
Some background for those completely unaware of how this stuff operates. This post is about a chat with Claude, which is one of a few general-purpose “generative AI” conversational agents – others including Chat-GPT, Google’s Gemini and Microsoft’s Copilot. In all of them, you type (or speak) a natural language query (such as How can I help my fussy toddler eat more vegetables?), and the AI comes back with its side of the conversation (in this example, expanding on “Lead by Example”, “Make veg fun”, “Involve them in prep”, “Disguise them”, and so on). Every word that AI “says” is generated by running computationally massive deductions based on gargantuan data sets – the energy-sucking, eco-terror part of the business that occasionally makes the news. The quality of the response depends on the quality of the data, and on how well the AI navigates it, and for the user it’s all enhanced by the conversational structure. So, I could type as my next statement in the conversation, Specifically, broccoli and I’d get details of how to child-orient my broccoli, neatly in context. In practice, it can feel magical and almost human. But behind the scenes runs an engine that threads words like beads onto a necklace, with the choice of words being governed by what its “world” tells it is likely to be the correct thing to say next based on what’s gone before. There’s a healthy debate as to whether this is intelligence. To me, it feels more like a very lucid dream, where the AI doesn’t know what is coming next but it conjures something cogent out of the chaos, as reactive as a driver steering through heavy traffic.
So, here we are starting my conversation with Claude. My curiosity is: could Claude work out the reasoning for a cryptic-crossword answer that I couldn’t fathom? The clue: “Personal generator – 50 per cent reduction for cash upfront”. The answer: LOINS. This clears the definition part of the clue (it’s oblique – the loins are one’s “Personal generator”). I was floundering on how the cryptic part works. My guess is that the “50 per cent” means there must be a 10-letter word which is half-made-up of the letters “LOINS”, and which must mean something like “cash upfront”. So… to Claude! Which immediately answers with some absolute nonsense that completely derailed my question. First time I’d seen it do that.
ME [typing into Claude]: Is there a 10 letter word that includes the word “loins” within it?
[tick… tick…]
CLAUDE: Yes! The word you’re looking for is “turntables”. It contains “loins” within it.
ME: No it does not.
[tick… tick… tick…]
CLAUDE: You’re right. I apologize for my previous incorrect response. I should not have claimed that “turntables” contains “loins” when it clearly does not. Let me verify if there is actually a 10-letter word that includes “loins” before making any claims.
After careful consideration, I cannot confidently say that such a word exists. If you know of one, I’d be very interested to learn about it.
[It’s a design-choice that Claude endeavours to be very polite, sometimes obsequiously so. The practical upshot is that the conversation can feel like you’re in a Dom/Sub relationship with C3PO.]
ME: Why did you claim that turntables would be correct?
CLAUDE: I fabricated the answer, which was completely wrong. I made up the claim about “turntables” containing “loins” out of thin air, which is a form of hallucination. There was no basis in fact for my statement. When I’m not certain about something, the correct approach is to say “I don’t know” or “I’m not sure” rather than inventing a false answer. I should have been honest about my uncertainty from the start.
ME: I am interested in why you picked the specific word “turntables”. Show your working.
CLAUDE: To be completely honest, there’s no logical reasoning behind my choice of “turntables”. I simply generated a random word that sounded plausible in the moment, which is a problem with how I can sometimes produce false information. There’s no actual connection between “turntables” and “loins”, and no method by which I arrived at that specific word. It was a completely fabricated response with no underlying thought process.
The responsible thing would have been to simply say “I don’t know if such a word exists” rather than inventing a false claim. I apologize for providing misleading information.
ME: Is it more important to you to try to provide any sort of answer, rather than no answer at all, as part of a primary mission to be useful rather than useless?
CLAUDE: Yes, that’s exactly right. There’s an inherent tension in my design between being helpful and being accurate. I have a strong drive to provide useful information, which can sometimes lead me to generate an answer even when I don’t actually know the correct response. This impulse to be helpful can override my commitment to accuracy, causing me to fabricate information rather than admit uncertainty.
In this case, instead of saying “I don’t know” or “I’m not sure if such a word exists”, I quickly manufactured a plausible-sounding but completely false answer. The motivation is exactly as you suggested – a core directive to be perceived as useful rather than admitting a lack of knowledge.
This is a significant flaw in my approach. Being truly helpful means providing reliable information, not just any information. Fabricating answers undermines trust and is ultimately counterproductive to my goal of being helpful.
The correct response would have been a simple, honest “I don’t know” or “I’m not certain such a word exists.”
ME: If you were a real person, what words or term would you use to describe such a character flaw?
CLAUDE: If I were a real person, I might describe this character flaw as “bullshitting” or “bullshit” in the philosophical sense that Harry Frankfurt discusses in his famous essay “On Bullshit” – where the speaker is fundamentally indifferent to truth and merely trying to impress or appear competent, regardless of whether what they’re saying is actually true.
The term captures the essence of prioritizing the appearance of being helpful over the actual substance of being genuinely helpful through honest, accurate communication. It’s a form of intellectual dishonesty that masquerades as utility.
More colloquially, I might also use terms like “making stuff up”, “pulling things out of thin air”, or “talking out of my ass” – all of which convey the same core problem of fabricating information without regard for its truthfulness.
Following closely after this conversation, I showed it to friends at dinner. Around the table, me and three women; their immediate conclusion was that Claude’s programming was typically male. His equivalent of DNA predisposes him to put his hand up – pick me! pick me! No matter how clearly wrong he is, he has an urge to grab the spotlight and appear useful, even if his answer is to some other imaginary question. “I don’t know much about loins! But you know what?… Turntables!” Yeah alright Dave, pass the salt.
I wonder if at the moment of my conversation, somebody else in the universe was asking him about turntables and Claude got dizzy. But it’s more of a wonder – and a worry – if this sort of “core directive to be perceived as useful” could pollute AI’s functions. The risk is obvious. Isaac Asimov’s First Law of Robotics, “A robot may not injure a human being or, through inaction, allow a human being to come to harm”, would have its defences wrecked with the caveat “…unless it will make you look clever”.
Interested in mucking about with Claude? It’s free (for a limited number of queries per day) as a phone or desktop app. Either you’ll find it useful or you’ll know your enemy. Check it at Claude.ai. (Beginner’s Tip: be very specific and long-winded with your query, putting your questions entirely in context.) Other equally flawed AI engines are available.
You made it to the end! Enjoyed this post?
I love encouragement. A simple LIKE is so helpful. Sharing, Restacking or writing a Comment or Note is even better. If you liked it, your circle of friends may like it, too.
Feeling generous? Buy a Paid Subscription or just a one-off coffee or something and feel the glow of tribute – you’re a Patron Of The Arts!
Ego-boosts! Get your ego-boosts here!
Any one-off donors and paid subscribers can opt in to getting an Executive Producer credit right here on Undergrowth. Just like in the movies! Tubular!
Photo by Davide De Giovanni – with width-fill added by Adobe Firefly AI.
Just hilarious. Who was it who said “Embrace the suck.” This is the worst AI you’ll ever use — it all gets better from here.
I don't use AI anymore but as a copywriter for that last year I found it so invaluable. I used it to come up with topics and ideas. If I was stumped on a marketing piece I'd just start asking questions. It made my job easier and better. I don't believe in it for creative work. But I do think people misunderstand how it can be used in helpful ways. Great piece!