a-organvm/python-packaging-patterns

Python Packaging Patterns

Structure Python projects for reliable distribution and dependency management.

Project Layout

The src Layout (Recommended)

my-project/
├── pyproject.toml
├── src/
│   └── my_package/
│       ├── __init__.py
│       ├── core.py
│       └── cli.py
├── tests/
│   ├── conftest.py
│   └── test_core.py
└── README.md

Why src layout: Prevents accidental imports from the working directory. Forces installation before testing, catching packaging errors early.

Flat Layout (Simple Projects)

my-project/
├── pyproject.toml
├── my_package/
│   ├── __init__.py
│   └── core.py
└── tests/

Acceptable for internal tools and single-organ repos where distribution is not a concern.

pyproject.toml Configuration

Minimal Configuration

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[project]
name = "my-package"
version = "0.1.0"
description = "A concise description"
requires-python = ">=3.11"
license = "MIT"
dependencies = [
    "httpx>=0.27",
]

[project.optional-dependencies]
dev = [
    "pytest>=8.0",
    "ruff>=0.5",
    "mypy>=1.10",
]

Build Backend Selection

| Backend | When to Use | |---------|-------------| | hatchling | Default choice. Fast, minimal config, good monorepo support | | setuptools | Legacy projects, C extensions, complex build needs | | flit | Pure Python, minimal config, simple projects | | poetry-core | When using Poetry for dependency management |

Optional Dependency Groups

Organize optional dependencies by use case:

[project.optional-dependencies]
dev = ["pytest>=8.0", "ruff>=0.5", "mypy>=1.10"]
docs = ["sphinx>=7.0", "myst-parser"]
dashboard = ["fastapi>=0.110", "uvicorn"]
metrics = ["prometheus-client"]

Install specific groups: pip install -e ".[dev,dashboard]"

Entry Points

Console Scripts

[project.scripts]
my-cli = "my_package.cli:main"

Plugin Entry Points

[project.entry-points."my_app.plugins"]
csv = "my_package.plugins.csv:CsvPlugin"
json = "my_package.plugins.json:JsonPlugin"

Version Management

Single Source of Truth

# In pyproject.toml
[project]
dynamic = ["version"]

[tool.hatch.version]
path = "src/my_package/__init__.py"

# In __init__.py
__version__ = "0.1.0"

CalVer for System Packages

For infrastructure packages where semantic versioning adds little value:

__version__ = "2026.03.1"  # YYYY.MM.patch

Dependency Pinning Strategy

| Context | Strategy | Tool | |---------|----------|------| | Library | Loose bounds (>=1.0,<2.0) | pyproject.toml | | Application | Exact pins | pip-compile / uv lock | | CI | Lockfile | uv.lock / requirements.txt |

Generating Lockfiles

# Using uv (recommended)
uv pip compile pyproject.toml -o requirements.txt
uv pip compile pyproject.toml --extra dev -o requirements-dev.txt

# Using pip-tools
pip-compile pyproject.toml -o requirements.txt

Editable Installs

# Standard editable install
pip install -e .

# With dev dependencies
pip install -e ".[dev]"

# In a fresh venv
python -m venv .venv && source .venv/bin/activate
pip install -e ".[dev]"

Tool Configuration in pyproject.toml

Ruff

[tool.ruff]
line-length = 100
target-version = "py311"

[tool.ruff.lint]
select = ["E", "F", "I", "N", "W", "UP"]

Pytest

[tool.pytest.ini_options]
testpaths = ["tests"]
asyncio_mode = "auto"
pythonpath = ["."]

Mypy

[tool.mypy]
strict = true
python_version = "3.11"

Publishing

To PyPI

# Build
python -m build

# Upload (use trusted publishing when possible)
python -m twine upload dist/*

Trusted Publishing (GitHub Actions)

- uses: pypa/gh-action-pypi-publish@release/v1
  with:
    password: ${{ secrets.PYPI_API_TOKEN }}

Common Patterns

Namespace Packages

For multi-repo packages sharing a namespace:

# Repo A
src/organvm/engine/__init__.py

# Repo B
src/organvm/dashboard/__init__.py

Use implicit namespace packages (no __init__.py at namespace level).

Conditional Dependencies

dependencies = [
    "tomli>=2.0; python_version < '3.11'",
    "typing-extensions>=4.0; python_version < '3.12'",
]

Package Data

[tool.hatch.build.targets.wheel]
packages = ["src/my_package"]

[tool.hatch.build.targets.wheel.force-include]
"assets" = "my_package/assets"

Anti-Patterns to Avoid

• setup.py without pyproject.toml — Always use pyproject.toml as the single config source
• Pinning exact versions in libraries — Use compatible ranges to avoid dependency conflicts
• Importing from project root in tests — Use src layout or ensure editable install
• Multiple version sources — Keep version in exactly one place
• requirements.txt as sole dependency spec — Use pyproject.toml; generate lockfiles from it