Step 3 — Make it pip-installable

From a folder of scripts to pip install yourthing

You have a working codebase. It has a README.md, a LICENSE, and probably a Zenodo DOI. Right now, if someone wants to try it, they:

  1. Clone your repo.
  2. Read the README to figure out which Python version and which dependencies.
  3. Fumble with a requirements.txt (or worse, no dependency file at all).
  4. Set up the PYTHONPATH to import from your code directory.
  5. Hope everything works on their OS.

Steps 3–4 of the handbook replace all of that with one command:

pip install yourthing

That’s what a pyproject.toml buys you.


The minimum viable pyproject.toml

Create a file called pyproject.toml at the repo root:

[build-system]
requires = ["setuptools>=64", "setuptools_scm>=8"]
build-backend = "setuptools.build_meta"

[project]
name = "yourthing"
dynamic = ["version"]
description = "What your code does, in one sentence."
readme = "README.md"
requires-python = ">=3.10"
license = {file = "LICENSE"}
authors = [
  {name = "Jane Smith", email = "smith.9999@osu.edu"},
]
keywords = ["earth observation", "machine learning", "your topic here"]
classifiers = [
  "Development Status :: 4 - Beta",
  "Intended Audience :: Science/Research",
  "License :: OSI Approved :: MIT License",
  "Programming Language :: Python :: 3",
  "Programming Language :: Python :: 3.10",
  "Programming Language :: Python :: 3.11",
  "Programming Language :: Python :: 3.12",
  "Topic :: Scientific/Engineering",
]

dependencies = [
  "numpy>=1.23",
  "pandas>=2.0",
  "matplotlib>=3.6",
  # ...list the *real* runtime dependencies, not what you happened to have installed
]

[project.optional-dependencies]
dev = ["pytest>=8", "pytest-cov", "ruff"]
notebooks = ["jupyterlab", "seaborn", "ipywidgets"]

[project.urls]
"Homepage" = "https://github.com/yourorg/yourthing"
"Documentation" = "https://github.com/yourorg/yourthing#readme"
"Issues" = "https://github.com/yourorg/yourthing/issues"

[tool.setuptools.packages.find]
where = ["."]
include = ["yourthing*"]

[tool.setuptools_scm]

That’s the whole file. A few things worth understanding:

dynamic = ["version"] + setuptools_scm

Instead of hard-coding your version in pyproject.toml (which you will forget to bump), setuptools_scm derives the version from your git tags. Tag v0.1.0 and setuptools_scm turns it into version 0.1.0. Push five commits past the tag with no new tag, and it becomes 0.1.1.dev5+g<sha>. Retag to v0.2.0 and the next build is 0.2.0.

This is the single most important QoL improvement over the traditional __version__ = "0.1.0" in __init__.py pattern.

dependencies vs. optional-dependencies

  • dependencies — the ones every user needs. Be conservative here. Every extra dependency is a reason the install might fail on someone else’s machine.
  • optional-dependencies — extras with names, installable as pip install yourthing[dev] or pip install yourthing[notebooks].

Split heavy or platform-specific stuff (PyTorch, GDAL, CUDA) into optional extras. This keeps pip install yourthing fast and portable.

requires-python

Set a floor. Not everything works everywhere. Set it to >=3.10 if you use pattern matching; >=3.11 if you use Self type hints; higher if you use str.removeprefix and so on.


Test the install locally

From the repo root:

# clean, isolated environment
mamba create -n test-install -y python=3.11
mamba activate test-install

# install your package in "editable" mode
pip install -e .

# verify the version is what setuptools_scm derived
python -c "import yourthing; print(yourthing.__version__)"

# try one of your public functions
python -c "from yourthing import main; print(main.__doc__)"

If it works from a clean env, it will work on someone else’s machine — mostly. Some pitfalls specific to scientific Python:

  • GDAL and PROJ — on macOS especially, pip-installed GDAL wheels sometimes conflict with system libraries. Recommend a mamba/conda-forge install path in your README, with pip as a fallback.
  • PyTorch — CUDA-specific wheels are only available from PyTorch’s own index. If your users need CUDA, document pip install torch --index-url https://download.pytorch.org/whl/cu121 as a separate step.
  • JAX — similar story to PyTorch.

The geoai-datacubes install docs show one working pattern: recommend mamba for the heavy dependencies, then pip install -e . for the project itself.


MANIFEST.in — trim your sdist

By default, when you build a source distribution (sdist), setuptools includes almost everything in your repo — including test outputs, cached notebooks, and any half-gigabyte data files you forgot about. Add a MANIFEST.in:

include LICENSE
include README.md
include CITATION.cff
include CHANGELOG.md
recursive-include yourthing *.py
recursive-include yourthing/data *.json *.yaml *.csv
recursive-exclude tests *
recursive-exclude notebooks *.ipynb
recursive-exclude data *
prune data
prune outputs
prune runs
prune .github
global-exclude __pycache__ *.py[cod] .DS_Store

Then test-build both distributions:

pip install build
python -m build            # produces dist/*.whl and dist/*.tar.gz
tar tzf dist/yourthing-*.tar.gz | head -50      # inspect sdist contents
unzip -l dist/yourthing-*.whl | head -50        # inspect wheel contents

If either archive is much bigger than you expected, tighten MANIFEST.in.

TipKeep the sdist under a few megabytes

PyPI has a 100 MB per-file limit and there’s no good reason to be anywhere near it. If your sdist is more than a few megabytes, you are almost certainly bundling data or notebook outputs that don’t belong. Fix MANIFEST.in before Step 4.


Namespace: what is yourthing, exactly?

Three names have to line up:

  • The [project].name in pyproject.toml — what PyPI calls it, what users type into pip install.
  • The top-level import name in Python — import yourthing.
  • The top-level directory in your repo — yourthing/ containing __init__.py.

By convention these are all the same string (or PyPI uses hyphens where Python uses underscores: pip install my-thingimport my_thing).

Pick one string and use it consistently. Renaming later means users have to change both their pip install line and their import line, and you lose downstream links.


Checklist

Once pip install -e . works from a clean env, you’re ready to publish it publicly — on to Step 4 — PyPI Trusted Publishing.