Anatomy of a Python project

packages, imports and why a file won't run

What it is

Article 2.3 covered how a project is laid out in general. Here is what is laid out the Python way: packages, __init__.py, imports and running things with python -m.

The topic looks dull right up until the first ModuleNotFoundError in a project that worked yesterday. Nine out of ten of those come from here.

What it looks like in code

weatherbot/
├── __init__.py          makes the folder a package
├── __main__.py          what runs on python -m weatherbot
├── config.py
└── handlers/
    ├── __init__.py
    └── weather.py

# weatherbot/handlers/weather.py
from weatherbot.config import MAX_TEXT


def cmd_weather(city: str) -> str:
    return f"weather for {city} (limit {MAX_TEXT})"

# weatherbot/__main__.py
from weatherbot.handlers.weather import cmd_weather


def main() -> None:
    print(cmd_weather("Oslo"))


if __name__ == "__main__":
    main()

The main trap

Run the file directly, the usual way:

python3 weatherbot/handlers/weather.py

ModuleNotFoundError: No module named 'weatherbot'

The code is right, the import is right, the file is where it should be — and it doesn't work.

The reason is where Python starts looking for modules. When you run a file, it looks from that file's folder, that is, from handlers/. There is no weatherbot package there — it is one level up.

The correct run is from the project root, as a module:

python3 -m weatherbot

weather for Oslo (limit 200)

Now the search starts from the current folder, weatherbot is found, and the imports work.

What __init__.py is

A file that tells Python: "this folder is a package, you can import from it". Most often it is empty, and that is fine.

If it isn't empty, take a look: the code inside runs the first time the package is touched. People sometimes put convenience imports there:

# weatherbot/handlers/__init__.py
from .weather import cmd_weather     # now: from weatherbot.handlers import cmd_weather

Convenient, but it is one more place where code runs unexpectedly (article 2.2).

Two kinds of import

from weatherbot.config import MAX_TEXT    # absolute
from .config import MAX_TEXT              # relative
from ..config import MAX_TEXT             # one level up

The dot means "from the current package". Relative imports are shorter, but they have an unpleasant property: a file that uses them can never be run directly at all.

Practical advice: use absolute imports in your own projects. They are longer, and they work the same way everywhere and read without counting dots.

What else lives in the root

pyproject.toml increasingly replaces requirements.txt. If a project has both, find out which one is actually used at install time, or you will discover the versions have drifted apart.

What to tell the AI

Show the project file tree without .venv and __pycache__.
One line per folder, saying what is in it.

It won't run: ModuleNotFoundError: No module named 'weatherbot'.
The project structure is: <tree>. I run it like this: <command>.
What is the correct way to run it?

When creating a new project it is worth setting the structure straight away:

Create a project skeleton: a weatherbot package with a handlers
subpackage, entry point via python -m, absolute imports.
Empty files, just the structure and __init__.py.

What to look for

Relative imports mixed with absolute ones. from .config in one file, from weatherbot.config in another — it will work and it will read badly.

sys.path.append in the code. A sign the project structure is wrong and the problem was patched by hand. Almost always cured by running it correctly.

Code in __init__.py. It runs when the package is imported, and that is easy to forget.

Circular imports. a.py imports b.py, which imports a.py. Python fails with ImportError: cannot import name. The AI loves fixing this with an import inside a function — it works, and the cause remains: the structure is tangled.

A thousand-line file in the root. The project is undivided, and any change disturbs everything else.

Cheat sheet