Reading a diff

what exactly the AI changed

What it is

A diff is a list of changes: what was removed, what was added. When the AI edits code, the diff is what you read — not the whole file. It shows exactly what changed and nothing else.

This is the main tool of acceptance. All of your control over a project rests on the habit of looking at the diff before taking a change.

What it looks like in code

We asked for the KeyError on a missing city to be fixed. Here is what came back:

--- a/weather.py
+++ b/weather.py
@@ -2,6 +2,8 @@ import logging
 
 log = logging.getLogger(__name__)
 
+DEFAULT_TEMP = 0
+
 
 def fetch_weather(city: str) -> dict:
     data = call_api(city)
@@ -9,5 +11,5 @@ def fetch_weather(city: str) -> dict:
 
 
 def format_report(data: dict) -> str:
-    temp = data["main"]["temp"]
+    temp = data.get("main", {}).get("temp", DEFAULT_TEMP)
     return f"It is {temp} degrees right now"

How to read it

A - at the start of a line means the line was removed. + means added. A line with no sign is context: it did not change and is shown so you can see where you are.

@@ -9,5 +11,5 @@ is a position marker: there were five lines starting at nine, now five starting at eleven. The exact numbers are rarely needed, but they show whether the file has shifted.

--- a/ and +++ b/ say which file. When several files change, there are several such blocks.

A modified line shows up as a removal plus an addition. A -/+ pair next to each other is one line that was rewritten.

What we see in this diff

One line changed. data["main"]["temp"] became data.get("main", {}).get("temp", DEFAULT_TEMP).

It will stop crashing. But now, when the reply is "city not found", the user gets "It is 0 degrees right now". The error is not fixed — it is hidden, and hidden in a way that looks like a real answer.

This is exactly the case from article 3.5: "make the error go away" and "fix the cause" are different tasks. And the only way to see it is by reading the diff: a run will show that the bot no longer crashes and leave you feeling everything is fine.

How to look at a diff

git diff                  # changed but not yet staged
git diff --staged         # already staged
git diff --stat           # just the files and the line counts
git diff -- weather.py    # one file only

Start with --stat. One line of output where you expected one — good. Seven files where you expected one — a signal to stop (article 3.13).

 weather.py | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

The order of reading

1. --stat — how many files and lines. Does it match what you expected?
2. Removed lines (-). What disappeared? Removal is more dangerous than addition: added code is at least visible, while a deleted check is hard to notice.
3. Added lines (+). Do they do what was asked?
4. What did not change although it should have.

The second point is the one people skip most, and it matters more than the third.

What to tell the AI

Show me a diff of your changes, not the whole file.

In this diff: does the behaviour change for cases that used
to work? If so, which ones?

You removed the line with the length check. Why?

The second prompt is a decent self-check after a large edit, but remember: about its own code the AI is lenient (article 3.8). Serious diffs you read yourself.

What to look for

A diff bigger than expected. A two-line change with a fifty-line diff means something else was done too.

Reformatting mixed in with the change. Reordered imports, changed quotes, rewrapped lines — the real change drowns in the noise. This has to be forbidden up front (article 3.13).

Deleted checks. if not data: return, if len(text) > MAX — lines like these are the first to be lost in a rewrite and look optional.

A quiet change of behaviour. As in the example: there was an exception, now there is a zero. Formally one line, in substance a different program.

Changes to files nobody asked about. Look at --stat first.

Cheat sheet