aleonsa/fastapi-verification

FastAPI Verification Loop

Run before PRs, after major changes, and pre-deploy to ensure FastAPI application quality and security.

When to Activate

• Before opening a pull request for a FastAPI project
• After model changes, Alembic migration updates, or dependency upgrades
• Pre-deployment verification for staging or production
• Running full environment → lint → migrations → tests → security → config pipeline

Phase 1: Environment Check

# Python version (match pyproject.toml / .python-version)
python --version

# Verify virtual environment is active
which python  # Should be inside venv or .venv

# Check for outdated dependencies
uv pip list --outdated
# or: pip list --outdated

# Verify critical env vars are set
python -c "
import os
required = ['DATABASE_URL', 'SECRET_KEY']
missing = [v for v in required if not os.environ.get(v)]
if missing:
    print('MISSING:', missing)
else:
    print('All required env vars set')
"

If environment is misconfigured, stop and fix before proceeding.

Phase 2: Code Quality & Formatting

# Type checking
mypy app/ --ignore-missing-imports

# Linting with ruff (replaces flake8 + isort + pyupgrade)
ruff check . --fix

# Formatting
ruff format . --check
ruff format .  # auto-fix

# Or if using black + isort separately:
black . --check && black .
isort . --check-only && isort .

Common issues:

• Missing type hints on public functions (mypy)
• Unused imports, undefined names (ruff)
• Async functions not awaited
• Pydantic v1 patterns used instead of v2

Phase 3: Alembic Migrations

# Check current migration state
alembic current

# Verify all migrations are applied
alembic history --verbose

# Check for model changes that need a new migration
# (compare ORM models against the DB)
alembic check
# Returns non-zero if autogenerate would produce changes

# Dry-run next migration (review before applying)
alembic upgrade head --sql | head -50

# Apply migrations (test environment)
alembic upgrade head

# Verify downgrade works (important for rollback safety)
alembic downgrade -1
alembic upgrade head

Report:

• Current migration revision
• Any unapplied migrations
• Any ORM changes without a corresponding migration

Migration safety checklist:

• [ ] New columns have server defaults or are nullable
• [ ] No destructive changes (DROP COLUMN, DROP TABLE) without data backup
• [ ] Large table migrations tested for lock impact
• [ ] Downgrade script is correct and tested

Phase 4: Tests + Coverage

# Run full test suite with coverage
pytest --cov=app --cov-report=term-missing --cov-report=html

# Run fast tests only (exclude slow/integration)
pytest -m "not slow and not integration"

# Run with verbose output to identify failing tests
pytest -v

# Open HTML coverage report
open htmlcov/index.html  # macOS

Report:

• Total: X passed, Y failed, Z skipped
• Overall coverage: XX%
• Per-module coverage breakdown

Coverage targets:

| Component | Target | |-----------|--------| | Routers | 85%+ | | Services | 90%+ | | Auth / security | 90%+ | | Schemas / validators | 80%+ | | Overall | 80%+ |

Fail verification if overall coverage drops below 80%.

Phase 5: Security Scan

# Dependency vulnerability scan
pip-audit

# Static security analysis
bandit -r app/ -f json -o bandit-report.json
bandit -r app/  # human-readable summary

# Secret detection (gitleaks or trufflehog)
gitleaks detect --source . --verbose
# or: trufflehog filesystem .

# Check for known CVEs in transitive deps
uv pip check

Report:

• Vulnerable packages + CVE IDs
• Bandit: HIGH/MEDIUM issues to fix, LOW to review
• Exposed secrets (should be zero; stop deployment if any found)

Auto-fail conditions:

• Any HIGH severity bandit finding
• Any known CVE with a fix available (pip-audit)
• Any hardcoded secret detected

Phase 6: Configuration Review

# run: python -c "$(cat << 'EOF'
from app.config import get_settings

s = get_settings()
checks = {
    "debug is False": not s.debug,
    "secret_key is set": bool(s.secret_key),
    "database_url is set": bool(s.database_url),
    "cors_origins not wildcard": s.cors_origins != ["*"],
    "access_token_expire set": s.access_token_expire_minutes > 0,
}
for check, result in checks.items():
    icon = "✓" if result else "✗"
    print(f"{icon} {check}")
# EOF
# )"

Phase 7: Performance Checks

# Check for sync blocking calls in async context
# (grep for common offenders: requests, time.sleep, open())
grep -rn "requests\." app/ --include="*.py" | grep -v test
grep -rn "time\.sleep" app/ --include="*.py" | grep -v test

# Check SQLAlchemy for missing eager loading on relationships
# (look for lazy loaded relationships accessed in async context)
grep -rn "lazy=" app/models/ --include="*.py"

# Profile a specific endpoint (optional, dev only)
# pip install pyinstrument
# uvicorn app.main:app & pyinstrument -m httpx GET http://localhost:8000/api/v1/products/

Report:

• Sync calls in async routes (blocks the event loop)
• Relationships with unsafe lazy loading for async

Phase 8: API Schema Validation

# Generate OpenAPI schema
python -c "
import json
from app.main import app
schema = app.openapi()
with open('openapi.json', 'w') as f:
    json.dump(schema, f, indent=2)
print(f'Schema generated: {len(schema[\"paths\"])} paths')
"

# Validate the schema file is valid JSON
python -c "import json; json.load(open('openapi.json')); print('Valid JSON')"

# Optional: lint with spectral (if installed)
spectral lint openapi.json

Phase 9: Diff Review

# Show diff statistics
git diff --stat

# Check for common issues in the diff
git diff | grep -n "print("           # Debug statements
git diff | grep -n "TODO\|FIXME\|HACK" # Open issues
git diff | grep -n "debug=True"        # Debug mode
git diff | grep -n "password\s*=\s*['\"]" # Hardcoded passwords
git diff | grep -n "secret\s*=\s*['\"]"   # Hardcoded secrets

# Show all changed files
git diff --name-only

Checklist:

• [ ] No print() debug statements
• [ ] No hardcoded secrets or credentials
• [ ] No debug=True in production config
• [ ] No TODO/FIXME in critical paths
• [ ] Alembic migration included for model changes
• [ ] Error handling present for all external calls
• [ ] New dependencies added to pyproject.toml

Output Template

FASTAPI VERIFICATION REPORT
============================

Phase 1: Environment
  ✓ Python 3.12.2
  ✓ Virtual environment active
  ✓ DATABASE_URL, SECRET_KEY set

Phase 2: Code Quality
  ✓ mypy: no errors
  ✗ ruff: 2 issues (auto-fixed)
  ✓ ruff format: OK

Phase 3: Alembic Migrations
  ✓ Current: abc123 (head)
  ✓ No unapplied migrations
  ✓ alembic check: no new changes detected
  ✓ Downgrade/upgrade roundtrip OK

Phase 4: Tests + Coverage
  Tests: 134 passed, 0 failed, 3 skipped
  Coverage:
    Overall: 86%
    app/routers:  88%
    app/services: 91%
    app/auth:     94%

Phase 5: Security Scan
  ✓ pip-audit: no vulnerabilities
  ✗ bandit: 1 MEDIUM issue (B105 hardcoded_password_string — review)
  ✓ No secrets detected

Phase 6: Configuration
  ✓ debug = False
  ✓ secret_key set
  ✓ database_url set
  ✓ cors_origins not wildcard
  ✓ token expiry configured

Phase 7: Performance
  ✓ No sync blocking calls found
  ✓ Relationships using selectin loading

Phase 8: API Schema
  ✓ OpenAPI schema valid
  ✓ 18 paths exported

Phase 9: Diff Review
  Files changed: 7
  +210, -45 lines
  ✓ No debug statements
  ✓ No hardcoded secrets
  ✓ Migration included

RECOMMENDATION: ⚠️ Review bandit B105 finding before merging

NEXT STEPS:
1. Fix bandit MEDIUM finding
2. Re-run security scan
3. Merge PR

Pre-Deployment Checklist

• [ ] All tests passing
• [ ] Coverage ≥ 80%
• [ ] No security vulnerabilities (pip-audit)
• [ ] No HIGH bandit findings
• [ ] Alembic migrations up to date
• [ ] debug = False
• [ ] secret_key via env var (not hardcoded)
• [ ] CORS origins locked down (not ["*"])
• [ ] HTTPS enforced at reverse proxy
• [ ] Rate limiting on auth endpoints
• [ ] Database connection pooling configured
• [ ] Structured logging configured
• [ ] Error monitoring (Sentry etc.) configured
• [ ] Health check endpoint available (/healthz)
• [ ] OpenAPI schema reviewed (no sensitive data exposed)

Continuous Integration (GitHub Actions)

# .github/workflows/fastapi-verification.yml
name: FastAPI Verification

on: [push, pull_request]

jobs:
  verify:
    runs-on: ubuntu-latest
    services:
      postgres:
        image: postgres:16
        env:
          POSTGRES_PASSWORD: postgres
          POSTGRES_DB: test_db
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5
        ports:
          - 5432:5432

    steps:
      - uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install uv
        uses: astral-sh/setup-uv@v3

      - name: Install dependencies
        run: uv sync --frozen

      - name: Code quality
        run: |
          uv run mypy app/ --ignore-missing-imports
          uv run ruff check .
          uv run ruff format . --check

      - name: Check migrations
        env:
          DATABASE_URL: postgresql+asyncpg://postgres:postgres@localhost:5432/test_db
        run: |
          uv run alembic upgrade head
          uv run alembic check

      - name: Run tests
        env:
          DATABASE_URL: postgresql+asyncpg://postgres:postgres@localhost:5432/test_db
          SECRET_KEY: ci-test-secret-key-not-for-production
        run: uv run pytest --cov=app --cov-report=xml --cov-fail-under=80

      - name: Security scan
        run: |
          uv run pip-audit
          uv run bandit -r app/ -ll  # only MEDIUM+ findings

      - name: Upload coverage
        uses: codecov/codecov-action@v4
        with:
          files: ./coverage.xml

Quick Reference

| Check | Command | |-------|---------| | Type check | mypy app/ | | Lint + format | ruff check . && ruff format . --check | | Migration state | alembic current && alembic check | | Apply migrations | alembic upgrade head | | Tests + coverage | pytest --cov=app | | Dependency CVEs | pip-audit | | Security lint | bandit -r app/ | | Secret scan | gitleaks detect --source . | | Diff stats | git diff --stat |

Remember: Automated verification catches common issues but does not replace manual code review and staging environment testing.