github/python-pypi-package-builder

Python PyPI Package Builder Skill

A complete, battle-tested guide for building, testing, linting, versioning, typing, and publishing a production-grade Python library to PyPI — from first commit to community-ready release.

AI Agent Instruction: Read this entire file before writing a single line of code or creating any file. Every decision — layout, backend, versioning strategy, patterns, CI — has a decision rule here. Follow the decision trees in order. This skill applies to any Python package type (utility, SDK, CLI, plugin, data library). Do not skip sections.

---

Quick Navigation

| Section in this file | What it covers | |---|---| | 1. Skill Trigger | When to load this skill | | 2. Package Type Decision | Identify what you are building | | 3. Folder Structure Decision | src/ vs flat vs monorepo | | 4. Build Backend Decision | setuptools / hatchling / flit / poetry | | 5. PyPA Packaging Flow | The canonical publish pipeline | | 6. Project Structure Templates | Full layouts for every option | | 7. Versioning Strategy | PEP 440, semver, dynamic vs static |

| Reference file | What it covers | |---|---| | references/pyproject-toml.md | All four backend templates, setuptools_scm, py.typed, tool configs | | references/library-patterns.md | OOP/SOLID, type hints, core class design, factory, protocols, CLI | | references/testing-quality.md | conftest.py, unit/backend/async tests, ruff/mypy/pre-commit | | references/ci-publishing.md | ci.yml, publish.yml, Trusted Publishing, TestPyPI, CHANGELOG, release checklist | | references/community-docs.md | README, docstrings, CONTRIBUTING, SECURITY, anti-patterns, master checklist | | references/architecture-patterns.md | Backend system (plugin/strategy), config layer, transport layer, CLI, backend injection | | references/versioning-strategy.md | PEP 440, SemVer, pre-release, setuptools_scm deep-dive, flit static, decision engine | | references/release-governance.md | Branch strategy, branch protection, OIDC, tag author validation, prevent invalid tags | | references/tooling-ruff.md | Ruff-only setup (replaces black/isort), mypy config, pre-commit, asyncio_mode=auto |

Scaffold script: run python skills/python-pypi-package-builder/scripts/scaffold.py --name your-package-name to generate the entire directory layout, stub files, and pyproject.toml in one command.

---

1. Skill Trigger

Load this skill whenever the user wants to:

• Create, scaffold, or publish a Python package or library to PyPI
• Build a pip-installable SDK, utility, CLI tool, or framework extension
• Set up pyproject.toml, linting, mypy, pre-commit, or GitHub Actions for a Python project
• Understand versioning (setuptools_scm, PEP 440, semver, static versioning)
• Understand PyPA specs: py.typed, MANIFEST.in, RECORD, classifiers
• Publish to PyPI using Trusted Publishing (OIDC) or API tokens
• Refactor an existing package to follow modern Python packaging standards
• Add type hints, protocols, ABCs, or dataclasses to a Python library
• Apply OOP/SOLID design patterns to a Python package
• Choose between build backends (setuptools, hatchling, flit, poetry)

Also trigger for phrases like: "build a Python SDK", "publish my library", "set up PyPI CI", "create a pip package", "how do I publish to PyPI", "pyproject.toml help", "PEP 561 typed", "setuptools_scm version", "semver Python", "PEP 440", "git tag release", "Trusted Publishing".

---

2. Package Type Decision

Identify what the user is building before writing any code. Each type has distinct patterns.

Decision Table

| Type | Core Pattern | Entry Point | Key Deps | Example Packages | |---|---|---|---|---| | Utility library | Module of pure functions + helpers | Import API only | Minimal | arrow, humanize, boltons, more-itertools | | API client / SDK | Class with methods, auth, retry logic | Import API only | httpx or requests | boto3, stripe-python, openai | | CLI tool | Command functions + argument parser | [project.scripts] or [project.entry-points] | click or typer | black, ruff, httpie, rich | | Framework plugin | Plugin class, hook registration | [project.entry-points."framework.plugin"] | Framework dep | pytest-*, django-*, flask-* | | Data processing library | Classes + functional pipeline | Import API only | Optional: numpy, pandas | pydantic, marshmallow, cerberus | | Mixed / generic | Combination of above | Varies | Varies | Many real-world packages |

Decision Rule: Ask the user if unclear. A package can combine types (e.g., SDK with a CLI entry point) — use the primary type for structural decisions and add secondary type patterns on top.

For implementation patterns of each type, see references/library-patterns.md.

Package Naming Rules

• PyPI name: all lowercase, hyphens — my-python-library
• Python import name: underscores — my_python_library
• Check availability: https://pypi.org/search/ before starting
• Avoid shadowing popular packages (verify pip install <name> fails first)

---

3. Folder Structure Decision

Decision Tree

Does the package have 5+ internal modules OR multiple contributors OR complex sub-packages?
├── YES → Use src/ layout
│         Reason: prevents accidental import of uninstalled code during development;
│         separates source from project root files; PyPA-recommended for large projects.
│
├── NO → Is it a single-module, focused package (e.g., one file + helpers)?
│         ├── YES → Use flat layout
│         └── NO (medium complexity) → Use flat layout, migrate to src/ if it grows
│
└── Is it multiple related packages under one namespace (e.g., myorg.http, myorg.db)?
          └── YES → Use namespace/monorepo layout

Quick Rule Summary

| Situation | Use | |---|---| | New project, unknown future size | src/ layout (safest default) | | Single-purpose, 1–4 modules | Flat layout | | Large library, many contributors | src/ layout | | Multiple packages in one repo | Namespace / monorepo | | Migrating old flat project | Keep flat; migrate to src/ at next major version |

---

4. Build Backend Decision

Decision Tree

Does the user need version derived automatically from git tags?
├── YES → Use setuptools + setuptools_scm
│         (git tag v1.0.0 → that IS your release workflow)
│
└── NO → Does the user want an all-in-one tool (deps + build + publish)?
          ├── YES → Use poetry (v2+ supports standard [project] table)
          │
          └── NO → Is the package pure Python with no C extensions?
                    ├── YES, minimal config preferred → Use flit
                    │   (zero config, auto-discovers version from __version__)
                    │
                    └── YES, modern & fast preferred → Use hatchling
                        (zero-config, plugin system, no setup.py needed)

Does the package have C/Cython/Fortran extensions?
└── YES → MUST use setuptools (only backend with full native extension support)

Backend Comparison

| Backend | Version source | Config | C extensions | Best for | |---|---|---|---|---| | setuptools + setuptools_scm | git tags (automatic) | pyproject.toml + optional setup.py shim | Yes | Projects with git-tag releases; any complexity | | hatchling | manual or plugin | pyproject.toml only | No | New pure-Python projects; fast, modern | | flit | __version__ in __init__.py | pyproject.toml only | No | Very simple, single-module packages | | poetry | pyproject.toml field | pyproject.toml only | No | Teams wanting integrated dep management |

For all four complete pyproject.toml templates, see references/pyproject-toml.md.

---

5. PyPA Packaging Flow

This is the canonical end-to-end flow from source code to user install. Every step must be understood before publishing.

1. SOURCE TREE
   Your code in version control (git)
   └── pyproject.toml describes metadata + build system

2. BUILD
   python -m build
   └── Produces two artifacts in dist/:
       ├── *.tar.gz   → source distribution (sdist)
       └── *.whl      → built distribution (wheel) — preferred by pip

3. VALIDATE
   twine check dist/*
   └── Checks metadata, README rendering, and PyPI compatibility

4. TEST PUBLISH (first release only)
   twine upload --repository testpypi dist/*
   └── Verify: pip install --index-url https://test.pypi.org/simple/ your-package

5. PUBLISH
   twine upload dist/*          ← manual fallback
   OR GitHub Actions publish.yml  ← recommended (Trusted Publishing / OIDC)

6. USER INSTALL
   pip install your-package
   pip install "your-package[extra]"

Key PyPA Concepts

| Concept | What it means | |---|---| | sdist | Source distribution — your source + metadata; used when no wheel is available | | wheel (.whl) | Pre-built binary — pip extracts directly into site-packages; no build step | | PEP 517/518 | Standard build system interface via pyproject.toml [build-system] table | | PEP 621 | Standard [project] table in pyproject.toml; all modern backends support it | | PEP 639 | license key as SPDX string (e.g., "MIT", "Apache-2.0") — not {text = "MIT"} | | PEP 561 | py.typed empty marker file — tells mypy/IDEs this package ships type information |

For complete CI workflow and publishing setup, see references/ci-publishing.md.

---

6. Project Structure Templates

A. src/ Layout (Recommended default for new projects)

your-package/
├── src/
│   └── your_package/
│       ├── __init__.py           # Public API: __all__, __version__
│       ├── py.typed              # PEP 561 marker — EMPTY FILE
│       ├── core.py               # Primary implementation
│       ├── client.py             # (API client type) or remove
│       ├── cli.py                # (CLI type) click/typer commands, or remove
│       ├── config.py             # Settings / configuration dataclass
│       ├── exceptions.py         # Custom exception hierarchy
│       ├── models.py             # Data classes, Pydantic models, TypedDicts
│       ├── utils.py              # Internal helpers (prefix _utils if private)
│       ├── types.py              # Shared type aliases and TypeVars
│       └── backends/             # (Plugin pattern) — remove if not needed
│           ├── __init__.py       # Protocol / ABC interface definition
│           ├── memory.py         # Default zero-dep implementation
│           └── redis.py          # Optional heavy implementation
├── tests/
│   ├── __init__.py
│   ├── conftest.py               # Shared fixtures
│   ├── unit/
│   │   ├── __init__.py
│   │   ├── test_core.py
│   │   ├── test_config.py
│   │   └── test_models.py
│   ├── integration/
│   │   ├── __init__.py
│   │   └── test_backends.py
│   └── e2e/                      # Optional: end-to-end tests
│       └── __init__.py
├── docs/                         # Optional: mkdocs or sphinx
├── scripts/
│   └── scaffold.py
├── .github/
│   ├── workflows/
│   │   ├── ci.yml
│   │   └── publish.yml
│   └── ISSUE_TEMPLATE/
│       ├── bug_report.md
│       └── feature_request.md
├── .pre-commit-config.yaml
├── pyproject.toml
├── CHANGELOG.md
├── CONTRIBUTING.md
├── SECURITY.md
├── LICENSE
├── README.md
└── .gitignore

B. Flat Layout (Small / focused packages)

your-package/
├── your_package/         # ← at root, not inside src/
│   ├── __init__.py
│   ├── py.typed
│   └── ... (same internal structure)
├── tests/
└── ... (same top-level files)

C. Namespace / Monorepo Layout (Multiple related packages)

your-org/
├── packages/
│   ├── your-org-core/
│   │   ├── src/your_org/core/
│   │   └── pyproject.toml
│   ├── your-org-http/
│   │   ├── src/your_org/http/
│   │   └── pyproject.toml
│   └── your-org-cli/
│       ├── src/your_org/cli/
│       └── pyproject.toml
├── .github/workflows/
└── README.md

Each sub-package has its own pyproject.toml. They share the your_org namespace via PEP 420 implicit namespace packages (no __init__.py in the namespace root).

Internal Module Guidelines

| File | Purpose | When to include | |---|---|---| | __init__.py | Public API surface; re-exports; __version__ | Always | | py.typed | PEP 561 typed-package marker (empty) | Always | | core.py | Primary class / main logic | Always | | config.py | Settings dataclass or Pydantic model | When configurable | | exceptions.py | Exception hierarchy (YourBaseError → specifics) | Always | | models.py | Data models / DTOs / TypedDicts | When data-heavy | | utils.py | Internal helpers (not part of public API) | As needed | | types.py | Shared TypeVar, TypeAlias, Protocol definitions | When complex typing | | cli.py | CLI entry points (click/typer) | CLI type only | | backends/ | Plugin/strategy pattern | When swappable implementations | | _compat.py | Python version compatibility shims | When 3.9–3.13 compat needed |

---

7. Versioning Strategy

PEP 440 — The Standard

Canonical form:  N[.N]+[{a|b|rc}N][.postN][.devN]

Examples:
  1.0.0          Stable release
  1.0.0a1        Alpha (pre-release)
  1.0.0b2        Beta
  1.0.0rc1       Release candidate
  1.0.0.post1    Post-release (e.g., packaging fix only)
  1.0.0.dev1     Development snapshot (not for PyPI)

Semantic Versioning (recommended)

MAJOR.MINOR.PATCH

MAJOR: Breaking API change (remove/rename public function/class/arg)
MINOR: New feature, fully backward-compatible
PATCH: Bug fix, no API change

Dynamic versioning with setuptools_scm (recommended for git-tag workflows)

# How it works:
git tag v1.0.0          →  installed version = 1.0.0
git tag v1.1.0          →  installed version = 1.1.0
(commits after tag)     →  version = 1.1.0.post1  (suffix stripped for PyPI)

# In code — NEVER hardcode when using setuptools_scm:
from importlib.metadata import version, PackageNotFoundError
try:
    __version__ = version("your-package")
except PackageNotFoundError:
    __version__ = "0.0.0-dev"    # Fallback for uninstalled dev checkouts

Required pyproject.toml config:

[tool.setuptools_scm]
version_scheme = "post-release"
local_scheme   = "no-local-version"   # Prevents +g<hash> from breaking PyPI uploads

Critical: always set fetch-depth: 0 in every CI checkout step. Without full git history, setuptools_scm cannot find tags and the build version silently falls back to 0.0.0+dev.

Static versioning (flit, hatchling manual, poetry)

# your_package/__init__.py
__version__ = "1.0.0"    # Update this before every release

Version specifier best practices for dependencies

# In [project] dependencies:
"httpx>=0.24"            # Minimum version — PREFERRED for libraries
"httpx>=0.24,<1.0"       # Upper bound only when a known breaking change exists
"httpx==0.27.0"          # Pin exactly ONLY in applications, NOT libraries

# NEVER do this in a library — it breaks dependency resolution for users:
# "httpx~=0.24.0"        # Too tight
# "httpx==0.27.*"        # Fragile

Version bump → release flow

# 1. Update CHANGELOG.md — move [Unreleased] entries to [x.y.z] - YYYY-MM-DD
# 2. Commit the changelog
git add CHANGELOG.md
git commit -m "chore: prepare release vX.Y.Z"
# 3. Tag and push — this triggers publish.yml automatically
git tag vX.Y.Z
git push origin main --tags
# 4. Monitor GitHub Actions → verify on https://pypi.org/project/your-package/

For complete pyproject.toml templates for all four backends, see references/pyproject-toml.md.

---

Where to Go Next

After understanding decisions and structure:

1. Set up pyproject.toml → references/pyproject-toml.md All four backend templates (setuptools+scm, hatchling, flit, poetry), full tool configs, py.typed setup, versioning config.

2. Write your library code → references/library-patterns.md OOP/SOLID principles, type hints (PEP 484/526/544/561), core class design, factory functions, __init__.py, plugin/backend pattern, CLI entry point.

3. Add tests and code quality → references/testing-quality.md conftest.py, unit/backend/async tests, parametrize, ruff/mypy/pre-commit setup.

4. Set up CI/CD and publish → references/ci-publishing.md ci.yml, publish.yml with Trusted Publishing (OIDC, no API tokens), CHANGELOG format, release checklist.

5. Polish for community/OSS → references/community-docs.md README sections, docstring format, CONTRIBUTING, SECURITY, issue templates, anti-patterns table, and master release checklist.

6. Design backends, config, transport, CLI → references/architecture-patterns.md Backend system (plugin/strategy pattern), Settings dataclass, HTTP transport layer, CLI with click/typer, backend injection rules.

7. Choose and implement a versioning strategy → references/versioning-strategy.md PEP 440 canonical forms, SemVer rules, pre-release identifiers, setuptools_scm deep-dive, flit static versioning, decision engine (DEFAULT/BEGINNER/MINIMAL).

8. Govern releases and secure the publish pipeline → references/release-governance.md Branch strategy, branch protection rules, OIDC Trusted Publishing setup, tag author validation in CI, tag format enforcement, full governed publish.yml.

9. Simplify tooling with Ruff → references/tooling-ruff.md Ruff-only setup replacing black/isort/flake8, mypy config, pre-commit hooks, asyncio_mode=auto (remove @pytest.mark.asyncio), migration guide.