What AI does well and what it does badly
and why its tone won't tell you which
What it is
An AI has strengths and weaknesses, and they are fairly predictable. Knowing them matters more than it seems: half of all disappointment comes from pointing a task in the wrong direction.
But the main thing in this article is something else. The tone of an answer has nothing to do with how right the AI is. It writes a boilerplate function it has seen a thousand times, and invents a library method that does not exist, with exactly the same confidence.
What it looks like in code
Where the AI is strong
| What | Why |
|---|---|
| Routine code: CRUD, parsing, formats | there was a great deal of it in training |
| Plumbing and chores: configs, arguments, logging | boring for a human, easy for a machine |
| Translating between languages | the structure stays, the syntax changes |
| Explaining someone else's code | a comprehension task, not an invention task |
| A first draft | faster than a blank page, even if you rewrite it |
| Finding your own typos | when you point at where to look |
What the whole list has in common: the task is common, there is one right answer, and checking it is easy.
Where the AI is weak
| What | Why |
|---|---|
| Your project and your data | it has not seen them (article 3.1, reason 2) |
| Recent library versions | training holds more old code than new |
| Rare libraries | few examples — more invention |
| Decisions with trade-offs | it doesn't know what matters more to you |
| Anything requiring memory of earlier decisions | a new session has no context |
| Judging its own work | it defends its own code (article 3.8) |
Why confidence doesn't help
An AI does not signal uncertainty — not by tone, not by hedging. An answer about a library it knows cold and an answer about a library it has seen twice sound exactly alike.
Hence the practical consequence: you decide how much to check, based on the nature of the task, not on how convincing the answer sounds.
What to tell the AI
Confidence can be pulled out with a direct question — and the AI answers it more honestly than its tone implies:
How confident are you in this code? Separately, tell me which
parts you would check first and why.For an unfamiliar library it is worth asking outright:
Are you sure session.fetch_all() exists in aiohttp 3.9?
If you are not sure, say so and I will check the docs.This phrasing works: by explicitly allowing "I'm not sure", you get a far more honest answer than from "does this method exist?".
What to look for
The more specific the task, the more carefully you read. Routine code can be skimmed; code for a rare library is read line by line.
A confident tone on a rare topic is a reason to tense up, not to relax.
"This is common practice" is not an argument. Common where? Your project may work differently, and the AI does not know that.
A trade-off handed to the AI. "Which is better, SQLite or Postgres?" without your conditions gives you an average answer, not yours.
Cheat sheet
What decides is how common the task is, not how hard.
Strong at: routine code, chores, translation, explanation, first drafts.
Weak at: your project, recent versions, rare libraries, trade-offs.
The tone of an answer is unrelated to how right the AI is.
You choose how much to check, based on the nature of the task.
Allow "I'm not sure" and you will hear an honest answer.