Poetry: managing dependencies and publishing a Python package

Definition

Writing Python code quickly demands more than code: declaring the libraries used, making sure their versions hold together, installing them in an environment of their own, and publishing the result. The classic chain answers these needs with four separate tools. Poetry brings them under a single command.

Everything starts from a single file, pyproject.toml, standardised by a PEP and common across the ecosystem. Poetry reads it and uses it to find versions that hold together, create the virtual environment that hosts them, and build the archive to publish.

Here is what a very first use looks like, from an empty project to a script that runs.

poetry new my-project               # creates pyproject.toml
cd my-project
poetry add requests                 # resolves, installs, locks
poetry run python -m my_project     # runs inside the project environment

What these lines replace: venv for isolation, pip for installing, requirements.txt for the dependency list, and the setup.py, wheel and twine trio for publishing on PyPI. Fewer files to keep in sync, fewer chances for them to drift apart.

Declaring is not resolving

This is the distinction that justifies the tool. A file produced by pip freeze does not say what the project asks for: it photographs what happened to sit on a computer on a given day, without separating what was wanted from what was simply dragged in behind.

Poetry splits the two into two files. The first, pyproject.toml, holds the intent: for instance, "this project needs requests in version 2.x". The second, poetry.lock, holds the outcome of the computation, the exact versions kept for the whole tree. The first is edited by hand, the second never.

The table below sums up what each approach keeps track of, and what it leaves to chance.

What the table does not show is the consequence over time: an installation made from the lock file gives the same tree six months later, on another machine or in continuous integration. A plain dependency list guarantees nothing of the sort.

The everyday commands

The Poetry manual lists several dozen commands. Six of them cover an ordinary working week.

poetry add pandas               # adds, resolves, installs, updates the lock
poetry add --group dev pytest   # development dependency only
poetry remove pandas            # removes the package and what it pulled in
poetry install                  # replays the lock file exactly
poetry update                   # recomputes, within the declared bounds
poetry run pytest               # runs inside the project environment

The split between regular and development dependencies answers a real question: why install pytest or ruff on the production server, if nobody there ever runs a test? Filing them under a development group settles the matter: they get installed on your side, not on the server.

Two more commands cover publishing a package: poetry build makes the archive, poetry publish sends it to the repository. They replace a hand-written setup.py and two extra tools.

The first-day mistake

It is almost always the same one: the environment genuinely exists, but the terminal is not inside it. The add command really did install the library in the right place, then python script.py run straight after goes through the system Python, which has never heard of it. The result is a ModuleNotFoundError on a perfectly correct import line.

Prefixing the command with poetry run is enough, since it runs inside the project environment. That form works whatever version of the tool is installed, unlike the sub-shell commands that have changed name across versions. In an editor, the equivalent reflex is to select the project interpreter; poetry env info --path gives its path.

When it is not worth it

A forty-line utility importing two libraries has nothing to lock: only one dependency tree is possible, too simple to need a computation. A virtual environment and pip do the job, and one more lock file only means one more file against a risk that does not exist.

There are also cases where the tool gets in the way: a deployment chain that only accepts a classic dependency list, a host that reads no other format, a scientific team whose packages are half binaries that conda installs better.

So when does a lock file actually start paying off? As soon as two people install the same project, or the same project gets installed on two machines on two different dates.

Frequently asked questions

Yes for an application, whose whole purpose is to reinstall identically everywhere. For a library installed on other people's machines, the lock file only serves your own tests: the declared version bounds are what counts.

Because finding a set of mutually compatible versions is a combinatorial problem: the tool fetches metadata, tries a combination, then backtracks as soon as one constraint contradicts another. That computation happens once; later installations merely replay it.

Yes, and that is even the normal case. The build command produces a standard archive, which anyone then installs with pip. An installation launched by hand inside the project environment works, but nothing records it in the lock file: it will vanish on the next rebuild. The module added that way exists on your machine only.