S012 · Modules, Packages, Virtualenvs, pip & uv
How Python code is organised, versioned, and shipped — from a single script to an installable package, with reproducible envs on every machine.
🎯 Take a folder of .py files and turn it into an installable, testable, reproducibly-buildable package that any teammate can `pip install` in 30 seconds.
Why this session exists
Writing one Python file is easy. Writing fifty — and having them import each other cleanly, share dependencies without collision, and land on a coworker's laptop in one command — is where 90% of Python beginners get stuck. The ImportError: attempted relative import with no known parent package, the "why does it work in the REPL but not when I run the file", the pip install pandas that breaks another project — all of it is the same story: you're missing the mental model of what a module is, what a package is, and what a virtual environment does. This session builds that model and hands you the modern toolchain (uv) that makes it all disappear.
- Explain the difference between a script, a module, a package, and a distribution — in one sentence each.
- Read Python's import resolution order (sys.path) and predict where an import will resolve from.
- Turn a folder of .py files into an installable package with a pyproject.toml and `uv pip install -e .`
- Create + pin an isolated environment with `uv venv` and reproduce it on a fresh machine from a lockfile.
- Diagnose the top-5 import errors (relative import, missing __init__, wrong cwd, wrong venv, shadowed name) in under 60 seconds.
Prerequisites
- S001 · Dev Environment — you need
uvinstalled and know how to activate a venv. - S005–S011 — comfortable writing functions, classes, and running Python from the command line.
(a) Intuition · 5 min
Think of your Python project as a library building. A single book is a module — one .py file with related functions inside. A shelf holding related books is a package — a folder with an __init__.py and several modules. A whole section with many shelves and a catalogue at the front is a distribution — the thing you actually publish to PyPI so others can install it.
The card catalogue at the entrance is pyproject.toml. Without it, nobody can find your books.
Concretely: math is a module, numpy is a package, and the file on PyPI called numpy-2.1.0-cp312-cp312-linux_x86_64.whl is a distribution.
You write modules. You organise them into packages. You describe the package as a distribution in pyproject.toml. Then uv build turns your source tree into a .whl that pip install understands.
The three problems the whole packaging stack solves
- Discoverability — Python needs to know WHERE to look when you write `import foo`. That's sys.path.
- Isolation — project A's numpy 2.1 must not collide with project B's numpy 1.24. That's virtual environments.
- Reproducibility — a coworker (or future-you on a fresh laptop) must be able to install the exact same versions. That's lockfiles.
A quick history so you know why the world looks like this
- 1991Python 0.9 · imports existGuido ships `import` in the first release. Modules are just .py files in the current directory.
- 2000distutils landsFirst official ‘how do I ship this?’ tool. Painful `setup.py` boilerplate.
- 2008pip releasedOne command to install from PyPI. Replaces easy_install. Becomes the de-facto standard.
- 2012virtualenv & pyvenvPer-project Python environments. `python -m venv` becomes stdlib in 3.3.
- 2018pyproject.toml (PEP 518)Declarative project config. Kills setup.py for 95% of projects.
- 2024uv 0.1 · AstralRust-based, 10-100× faster than pip, replaces pip + venv + pyenv + pip-tools in one binary.
(b) Visual walkthrough · 15 min
How an import actually resolves
The layers of the packaging stack
From source tree to installed package
The install path — one time per project
Creates myproj/ with pyproject.toml, README, .python-version, and an empty package skeleton.
Adds requests to pyproject.toml AND installs it into .venv AND updates uv.lock atomically.
Runs the script inside the project's venv without ever needing `source .venv/bin/activate`.
Refresh uv.lock. Commit it — it pins every transitive dep so coworkers get the exact same tree.
On a fresh machine: creates .venv and installs exactly what uv.lock says. Reproducible in seconds.
pip vs uv vs poetry vs conda
The 2024+ default
- Rust, 10-100× faster than pip
- Manages Python versions too
- Replaces pip + venv + pip-tools + pyenv
- Lockfile is a first-class citizen
- Use this for new projects
The stdlib fallback
- Ships with Python
- No lockfile (requirements.txt is not one)
- Slow resolver
- Still the lingua franca — you'll always know it
- Fine for tiny scripts
The 2018–2023 favourite
- First tool to bring lockfiles to the mainstream
- Own pyproject.toml dialect (not fully PEP-standard)
- Slower than uv
- Still popular in existing repos
- Use it if the repo already does
Data-science land
- Manages non-Python deps too (CUDA, MKL)
- Own package repo (Anaconda / conda-forge)
- Heavy, opinionated
- Use when you need C/CUDA libraries pip cannot ship
- Overkill for pure Python
The mental model to hold
"import finds and runs the file each time I import it. So if two modules both import config, each gets its own copy — and re-importing is how I pick up changes."
A module is executed once per process, on first import. The resulting module object is cached in sys.modules, and every later import of that name is a dictionary lookup returning the same object. Module-level state is therefore process-global singleton state.
Because "import = load the file" is the right intuition for the first import and produces correct predictions for pure-function modules, which is most of what you write early on. It breaks in three places at once: module-level mutable state (a cache, a connection, a registry) is shared by everyone who imports it and persists for the process lifetime; expensive module-level work runs once and its timing is unpredictable; and editing a file has no effect on a running interpreter, which is why a notebook keeps using your old function until you restart or explicitly importlib.reload. The caching is not an optimisation detail — it is what makes circular imports possible at all, and also what makes them fail in the particular half-initialised way they do.
Same object, every time — and the cache is inspectable:
# mod.py
# print('executing mod')
# registry = []
import mod # prints 'executing mod'
import mod as m2 # prints nothing
mod.registry.append(1)
print(m2.registry) # [1] <- same object
import sys
print(sys.modules['mod'] is mod) # True
print(mod.__file__) # where it actually came fromWhy do circular imports sometimes work and sometimes fail with "cannot import name X from partially initialized module"? The dependency cycle is identical in both cases — derive what makes the difference.
- 1Importing a module means executing its top-level code, statement by statement, to build its namespace.forced by · Python has no separate declaration phase; definitions are produced by running the file
- 2To prevent infinite recursion, the module object is inserted into
sys.modulesbefore its body runs — initially empty, filling up as execution proceeds.forced by · a cycle would otherwise re-enter the same import forever - 3So if module A imports B, and B imports A while A is still executing, B receives the real A object — but only partially populated, containing whatever A defined above its import of B.forced by · the cache hit succeeds; only the contents are incomplete
- 4Therefore
import ainside B succeeds unconditionally (it just binds the module object), whilefrom a import Xfails unlessXwas defined before A reached its import statement.forced by ·from ... importperforms an attribute lookup at import time, and the attribute may not exist yet - 5And a reference used only inside a function body is resolved at call time, by which point both modules have finished executing.forced by · function bodies are compiled at definition but names in them are looked up on execution
Therefore the failure depends entirely on when the name is needed relative to where the import sits — not on the existence of the cycle. This is why adding one innocuous top-level import can break a codebase that had cycles working for years.
And note what this predicts, and each is verifiable: switching from a import X to import a plus a.X inside functions usually fixes it · moving the import inside the function that needs it always fixes it · and if TYPE_CHECKING: imports never cause cycles because they don't execute at runtime. It also predicts the real diagnosis: a cycle means the two modules share a concern that belongs in a third module.
Two things happen on import x, in order: find (walk sys.path for a matching module) and execute once (run its top level, cache the result in sys.modules). Every subsequent import of that name skips straight to the cache.
So a module is not a file — it is a live object that happened to be built by running a file. Its top level is not a declaration section; it is startup code that runs at an unpredictable moment, exactly once, in an order determined by whoever imported first.
- Module top level should define, not do. Connections, file reads, and expensive setup at import time make import order significant and testing painful — hide them behind a function or a lazy accessor.
- Absolute imports everywhere; relative (
from .x import y) only inside a package. Never rely on the script's directory being onsys.path— that's whypython -m pkg.modandpython pkg/mod.pybehave differently. if __name__ == "__main__":exists because a module can be either imported or run, and the same file must not execute its CLI when imported.- Ship with
pyproject.tomland install withpip install -e .during development. Manipulatingsys.pathin code is a symptom that packaging was skipped.
Fire this model the moment you see: ImportError: cannot import name ... (most likely due to a circular import) · sys.path.append at the top of a file · a notebook still running old code after an edit · a module that opens a database connection when imported · tests that pass alone and fail together.
You have shared utility code used by three services. Do you copy it, extract an internal package, or keep everything in a monorepo?
The decision is about who absorbs the cost of a change, not about code reuse. Versioned packages push it onto consumers at a time of their choosing; monorepos push it onto the author immediately; copies push it onto whoever discovers the divergence months later, which is the only genuinely bad answer for code carrying business logic.
The practical trap in the middle option is under-specified dependencies. An internal package that floats its own requirements will eventually pull an incompatible transitive version into a consumer's environment — so pin ranges deliberately, and give the package the same lockfile discipline you would give a service.
(c) Hands-on · 25 min
You're going to build a tiny installable package called wordstats — it counts words in a text file — and prove it's installable, importable, and reproducible on a fresh venv. Copy this whole script into bootstrap.sh and run it.
What each block does
Anatomy of the script
Run these commands and observe:
# 1. Convert this project to uv-native layout
uv add pytest --dev # writes pyproject + uv.lock
uv lock # explicitly refresh the lockfile
cat uv.lock | head -30 # see exact resolved versions
# 2. Now simulate ‘fresh machine’
deactivate 2>/dev/null || true
rm -rf .venv
uv sync # rebuilds .venv from uv.lock, byte-for-byte
# 3. Verify it works
uv run pytest -qBonus challenge: add rich as a dependency (uv add rich), import it in __main__.py to pretty-print the table, commit pyproject.toml + uv.lock, then have a friend clone and run uv sync && uv run wordstats sample.txt. If it doesn't Just Work, you missed something.
(d) Production reality · 15 min
Instagram famously runs the largest Django deployment on Earth. Their monorepo has hundreds of internal packages. Early on, engineers used ad-hoc PYTHONPATH hacks to import between them, and requirements.txt without pins.
Result: two teams could deploy the same commit hash and get different behaviour because a transitive dep had released a new version between the two deploys. Nightmare to debug.
uv.lock (or poetry.lock, or requirements.lock) into Git. Always.cd into myproj/, run python src/myproj/main.py, and get ImportError: attempted relative import with no known parent package. You Google. Stack Overflow has 400 answers, all contradicting.The rule: never run a file inside a package directly. Always run it as a module from the project root:
# wrong
python src/myproj/main.py
# right (after: pip install -e .)
python -m myproj.main
# or
uv run python -m myproj.mainOr, better, register a console script entry point so users get a real command.
from .core import x) only work when Python knows the module's parent package. Running a file by path breaks that. Use python -m or console scripts.pip installs tensorflow 2.15 to try something, which downgrades numpy, which breaks another engineer's pandas code, which breaks a scheduled notebook, which alerts at 3 a.m.uv venv committed alongside the code. Use JupyterLab's kernel selector to attach to the project venv. The 30-second cost of activation is invisible compared to the hours of ‘who upgraded what’ archaeology.Where this shows up in the rest of the plan
(e) Recall + stretch · 10 min
Explain-out-loud test
If you can't teach these three to a friend without notes, redo the session:
- What is the difference between a module, a package, and a distribution? (one sentence each, with an example)
- What is a lockfile, and why does it matter more than requirements.txt? (one concrete failure it prevents)
- What is sys.path, and how would you use it to debug an ImportError? (name two lines of Python)
What comes next
Hub: The 6-Month Learning Plan
Part of a 130-session evergreen learning series. Session structure: intuition → visual → hands-on → production war stories → recall. Duration: 90 minutes.