thesoftwarehouse/tsh-implementing-backend

Implementing Backend

Provides patterns for building backend API services with modular architecture, structured testing, and production-ready infrastructure following TSH best practices.

When to Use

• Building new REST or GraphQL API endpoints
• Implementing CRUD operations with filtering, sorting, and pagination
• Setting up authentication and authorization (JWT)
• Integrating with external/third-party services
• Writing integration tests for endpoints or unit tests for business logic
• Configuring database migrations, seeding, or repository patterns
• Setting up Docker and docker-compose for local development
• Implementing logging and observability
• Documenting APIs with Swagger/OpenAPI
• Designing modular architecture with vertical slices

Guiding Principles

| Principle | Application | |---|---| | SRP | Each class/module has one reason to change. Controllers handle HTTP, services handle business logic, repositories handle data access. | | DRY | Extract shared logic into reusable services or utilities. Do not duplicate validation, mapping, or query logic. | | KISS | Prefer simple, readable solutions. Avoid over-engineering. Do not add abstractions until they are needed. | | YAGNI | Do not build features or infrastructure "just in case". Implement what is needed now. | | Pragmatism | Follow patterns when they add value. Break rules when strict adherence creates unnecessary complexity. Document the reasoning. |

Architecture: Vertical Slice / Modular Structure

Organize code by domain/feature, not by technical layer. All artifacts related to a domain live in the same directory.

src/
├── users/
│   ├── users.controller.ts      # HTTP layer (routes, request/response)
│   ├── users.service.ts          # Business logic
│   ├── users.repository.ts       # Data access
│   ├── users.module.ts           # Module registration / DI wiring
│   ├── dto/
│   │   ├── create-user.dto.ts
│   │   └── update-user.dto.ts
│   ├── entities/
│   │   └── user.entity.ts
│   ├── tests/
│   │   ├── users.integration.test.ts
│   │   └── users.service.unit.test.ts
│   └── users.swagger.yml         # (if using separate swagger files)
├── orders/
│   ├── orders.controller.ts
│   ├── orders.service.ts
│   ├── orders.repository.ts
│   └── ...
├── shared/                       # Cross-cutting concerns only
│   ├── middleware/
│   ├── guards/
│   ├── filters/
│   ├── interceptors/
│   └── utils/
└── config/
    ├── database.config.ts
    ├── auth.config.ts
    └── app.config.ts

Rules:

• A module should be self-contained. Moving or removing a feature module should not break other modules.
• Cross-module communication goes through well-defined interfaces (service interfaces, events), never direct imports of internal classes.
• Shared utilities go in shared/ only when used by 3+ modules. Otherwise keep them in the feature module.

REST API Design

Resource Naming & HTTP Methods

| Method | Path | Purpose | Success Code | |---|---|---|---| | GET | /resources | List with filtering, sorting, pagination | 200 | | GET | /resources/:id | Single resource details | 200 | | POST | /resources | Create resource | 201 | | PATCH | /resources/:id | Partial update | 200 | | PUT | /resources/:id | Full replace (use sparingly) | 200 | | DELETE | /resources/:id | Remove resource | 204 |

Naming conventions:

• Use plural nouns for resource names: /users, /orders, /products
• Use kebab-case for multi-word resources: /order-items
• Nest sub-resources max 1 level deep: /users/:id/orders (avoid deeper nesting)
• Use query parameters for filtering, not path segments

Standard Error Response Codes

| Code | Meaning | |---|---| | 400 | Validation errors (malformed request body, missing fields) | | 401 | Unauthenticated (missing or invalid token) | | 403 | Unauthorized (valid token but insufficient permissions) | | 404 | Resource not found | | 409 | Conflict (e.g. duplicate unique field) | | 422 | Business logic errors (foreign key violation, state conflict) | | 500 | Unexpected server error (never expose stack traces in production) |

Standard Error Response Format

{
  "statusCode": 400,
  "error": "Bad Request",
  "message": "Validation failed",
  "details": [
    { "field": "email", "message": "must be a valid email address" }
  ]
}

DataGrid: Filtering, Sorting & Pagination (TSH Standard)

Every list endpoint returning paginated data MUST follow this schema.

Request Query Parameters

| Parameter | Format | Description | |---|---|---| | page | page=1 | Page number (starting from 1) | | limit | limit=10 | Max results per page | | sort[field] | sort[lastName]=ASC | Sort by field, direction: ASC or DESC | | filter[field] | filter[firstName]=John | Filter by field value | | search | search=john | General text search (implementation-specific: LIKE, full-text, etc.) |

Example: GET /users?page=1&limit=10&sort[lastName]=ASC&filter[status]=active&search=john

Filter Behavior

• Same field, multiple values → interpreted as OR:

  ?filter[firstName]=Ewa&filter[firstName]=Adam
  → WHERE (firstName = 'Ewa' OR firstName = 'Adam')
  

• Different fields → interpreted as AND:

  ?filter[firstName]=Ewa&filter[lastName]=Kowalska
  → WHERE (firstName = 'Ewa' AND lastName = 'Kowalska')
  

• LIKE search → use URL-encoded %25 suffix:

  ?filter[lastName]=Now%25
  → WHERE lastName LIKE 'Now%'
  

Advanced Filter Operators (when applicable)

| Operator | SQL Equivalent | Example | |---|---|---| | eq | = | filter[status][eq]=active | | neq | <> | filter[status][neq]=deleted | | lt, lte | <, <= | filter[age][lt]=30 | | gt, gte | >, >= | filter[age][gte]=18 | | include | LIKE %val% | filter[name][include]=john | | in | IN (...) | filter[status][in]=active,pending |

Response Format (Mandatory)

{
  "meta": {
    "pagination": {
      "page": 1,
      "limit": 10,
      "total": 57,
      "totalPages": 6
    },
    "filter": {
      "status": "active"
    },
    "sort": {
      "lastName": "ASC"
    },
    "search": "john"
  },
  "data": [{ "..." }]
}

Rules:

• meta always reflects the actual applied parameters back to the client.
• Invalid filter/sort field names are silently ignored (no error thrown), but not applied.
• Default limit should be defined in app configuration (e.g. 20 or 50).
• Maximum limit should be capped to prevent abuse (e.g. 100 or 250).

Authentication & Authorization

JWT-Based Authentication

• Use JSON Web Tokens (JWT) for stateless authentication.
• Token is passed in the Authorization header: Bearer <token>.
• Validate the token signature, expiration (exp), and issuer (iss) on every protected request.
• Store secrets/keys in environment variables, never in code.
• Use short-lived access tokens (15-60 minutes) with refresh token rotation where appropriate.

Current User Endpoint

• Expose a GET /me endpoint that returns the profile of the currently authenticated user.
• This endpoint should extract the user identity from the JWT (e.g. sub claim) and return the full user profile.
• Do not accept user ID as a parameter — always derive from the token.

GET /me
Authorization: Bearer <token>
→ 200 { "id": "...", "email": "...", "roles": [...] }

Authorization

• Implement role-based access control (RBAC) or attribute-based access control (ABAC) depending on project complexity.
• Authorization checks happen via middleware/guards before reaching the controller action.
• Always validate that the authenticated user has permission to access/modify the specific resource (not just the endpoint).

Dependency Injection

• Always use a DI container. Register services, repositories, and infrastructure in a central container/module.
• Inject dependencies via constructor injection.
• Depend on interfaces/abstractions, not concrete implementations.
• DI enables testability: in tests, swap real implementations with mocks/stubs.

See the technology-specific references below for recommended DI frameworks per language.

Database Handling

ORM & Repository Pattern

• Use the project's ORM for all database operations (TypeORM, MikroORM, Doctrine, Entity Framework, Hibernate, GORM, etc.).
• Implement the Repository Pattern: all database queries go through repository classes, never directly from services or controllers.
• Repositories return domain entities/models, not raw database rows.
• Use transactions for operations that modify multiple tables/records.

Migrations

• All database schema changes go through migration files. Never modify the database manually.
• Each migration must have both up (apply) and down (revert) methods.
• Migrations run automatically on application startup (for containerized apps) or via a dedicated migration command/lambda (for serverless).
• Never modify an existing migration that has been deployed. Create a new migration instead.
• Name migrations descriptively: 2025-02-08-add-status-column-to-orders.

Seeding

• Provide seed data for development, test, and staging environments only.
• Never seed production or UAT environments with test data.
• Seeds should be idempotent — running them multiple times produces the same result.
• Separate seed files by domain (e.g. seed-users.ts, seed-products.ts).

Database Best Practices

• Use UUID or ULID for primary keys where appropriate (better for distributed systems).
• Define proper indexes for foreign keys, frequently queried columns, and unique constraints.
• Use snake_case naming for tables and columns (e.g. order_items, created_at).
• Always define created_at and updated_at timestamps.
• Use soft deletes (deleted_at) when business rules require record retention.

External Service Adapters (Third-Party Clients)

When integrating with external APIs, always create a dedicated client/adapter class.

Pattern

src/
├── integrations/
│   ├── payment-gateway/
│   │   ├── payment-gateway.client.ts   # HTTP calls, request/response mapping
│   │   ├── payment-gateway.types.ts    # External API types/interfaces
│   │   └── payment-gateway.module.ts   # DI registration
│   ├── email-provider/
│   │   ├── email-provider.client.ts
│   │   └── ...

Rules

1. Isolate all HTTP communication with external services into a client class. Never call HTTP clients (Axios, fetch, HttpClient) directly from services or controllers.
2. Map external types to internal domain types at the adapter boundary. The rest of the application should not know about the external API's data format.
3. Store configuration (API URLs, keys, tokens) in environment variables and inject via config.
4. Handle errors gracefully: catch HTTP errors, map them to domain-specific exceptions, and log the details.
5. Make clients testable: depend on an interface so the client can be mocked in tests.
6. Add retry logic and timeouts for resilience. Use circuit breaker patterns for critical integrations.

Testing Strategy

Test Pyramid

| Level | What to Test | |---|---| | Unit Tests | Pure business logic in services, domain models, utility functions. Mock all external dependencies. | | Integration Tests | API endpoints end-to-end (HTTP request → response). Use a real test database. | | E2E Tests | Critical user flows across the full stack. |

See the technology-specific references below for recommended testing tools per language.

Integration Tests for Endpoints

• Test every endpoint with valid and invalid inputs.
• Use a dedicated test database (same engine as production, e.g. PostgreSQL).
• Each test should set up its own data (arrange), call the endpoint (act), and verify the response (assert).
• Clean up test data after each test (use transactions or truncation).
• Verify: HTTP status code, response body structure, side effects (database state, events emitted).

describe('POST /users', () => {
  it('should create a user and return 201', async () => {
    // Arrange
    const payload = { email: '[email protected]', name: 'Test User' };

    // Act
    const response = await request(app).post('/users').send(payload);

    // Assert
    expect(response.status).toBe(201);
    expect(response.body.data.email).toBe('[email protected]');
  });

  it('should return 400 for invalid email', async () => {
    const response = await request(app).post('/users').send({ email: 'invalid' });
    expect(response.status).toBe(400);
  });
});

Unit Tests for Business Logic

• Test services and domain models in isolation.
• Mock repositories, external clients, and infrastructure.
• Focus on edge cases, error paths, and business rules.
• Keep unit tests fast — no database, no network, no filesystem.
• Use descriptive test names: should throw InsufficientFundsError when balance is below transfer amount.

Testing Rules

• Every new endpoint or business rule must have tests before merging.
• Aim for meaningful coverage of critical paths, not arbitrary percentage targets.
• Integration tests are the primary quality gate for API behavior.
• Unit tests are the primary quality gate for business logic.
• Mock external services (payment gateways, email providers) — never call real external APIs in tests.

API Documentation

Swagger / OpenAPI

• Every API must be documented using OpenAPI/Swagger specification.
• Prefer auto-generated docs from code annotations/decorators when the framework supports it.
• If auto-generation is not available, maintain a separate swagger.yml file split by domain.
• Serve documentation at /api-docs endpoint.
• Document: request/response schemas, query parameters, authentication requirements, error responses, and example values.
• Keep documentation in sync with the actual API — stale docs are worse than no docs.

See the technology-specific references below for recommended Swagger tooling per language.

Docker & Local Development

Docker Setup

• Every project must include a Dockerfile and docker-compose.yml for local development.
• The docker-compose.yml should include all required services: app, database (PostgreSQL), cache (Redis), mail catcher (Mailhog), etc.
• Use docker-compose.override.yml for developer-specific customizations (additional ports, volumes, debug settings).
• Application should be fully runnable with a single docker-compose up command.

Dockerfile Best Practices

• Use multi-stage builds to keep images small.
• Pin base image versions (e.g. node:20-alpine, php:8.3-fpm-alpine, mcr.microsoft.com/dotnet/aspnet:8.0).
• Install only production dependencies in the final stage.
• Use .dockerignore to exclude build artifacts, test files, etc.
• Run as a non-root user in the container.

Health Check

Every application must expose a GET /health endpoint:

• Placed before all middleware and auth guards.
• Publicly accessible (no authentication required).
• Returns 200 with status information.

{
  "status": "ok"
}

For more thorough health checks, optionally verify database connectivity and critical service availability.

Logging & Observability

Structured Logging

• Use a structured logger — never console.log or print in production.
• Log in JSON format for machine parseability.
• Include contextual fields in every log entry: timestamp, level, requestId/correlationId, userId (if authenticated), service.

See the technology-specific references below for recommended logging libraries per language.

Log Levels

| Level | When to Use | |---|---| | error | Unexpected failures, unhandled exceptions, critical issues | | warn | Recoverable issues, deprecation notices, approaching limits | | info | Significant business events: user created, order placed, payment processed | | debug | Detailed diagnostic information (disabled in production) |

What to Log

• Always log: incoming requests (method, path, status code, duration), authentication failures, authorization failures, external service calls (URL, status, duration), errors with stack traces.
• Never log: passwords, tokens, API keys, credit card numbers, PII (personally identifiable information) unless encrypted or masked.

Request Logging

• Log every HTTP request with: method, path, status code, response time, and correlation/request ID.
• Use middleware (Morgan, express request logger, or framework equivalent) for automatic request logging.
• Propagate a correlationId / requestId header through the entire request lifecycle for tracing.

Scalability & Security

Scalability

• Design for horizontal scaling: no in-memory state, no sticky sessions.
• Never store temporary data in application memory — use Redis or an external cache.
• Use message queues (SQS, RabbitMQ, Bull) for async operations (email sending, PDF generation, data processing).
• Use database connection pooling.
• Apply rate limiting on public endpoints.
• Implement pagination on all list endpoints (never return unbounded result sets).

Security (OWASP TOP 10)

• Input validation: Validate and sanitize all user input at the API boundary (request body, query params, headers).
• SQL Injection: Always use parameterized queries / ORM. Never concatenate user input into SQL.
• Authentication: Use short-lived JWTs, validate signatures, handle token expiration.
• Authorization: Enforce at every endpoint. Check resource ownership, not just role membership.
• Sensitive data: Never expose stack traces, internal paths, or database details in error responses.
• CORS: Configure explicitly — never use * in production.
• Security headers: Use framework-appropriate middleware to set security headers (CSP, X-Frame-Options, etc.).
• Dependencies: Regularly audit and update dependencies. Use tools like npm audit, Snyk, or Dependabot.
• Rate limiting: Apply on authentication and public endpoints.
• Secrets: Store in environment variables or a secrets manager. Never commit to source control.

Configuration & Environment

• Use .env files for local development with a .env.dist (or .env.example) template committed to the repo.
• Validate all configuration on application startup (using Joi, Zod, class-validator, or equivalent). Fail fast on missing required config.
• Group configuration by concern: database, auth, cache, externalServices.
• Never hardcode environment-specific values. Everything must come from environment variables.

Implementation Procedure

When implementing a new backend feature, follow this workflow:

Implementation progress:
- [ ] Step 1: Understand the requirements
- [ ] Step 2: Design the data model
- [ ] Step 3: Create migration(s)
- [ ] Step 4: Implement the domain layer (entities, services, repositories)
- [ ] Step 5: Implement the API layer (controllers, DTOs, validation)
- [ ] Step 6: Add authentication/authorization guards
- [ ] Step 7: Write integration tests for endpoints
- [ ] Step 8: Write unit tests for business logic
- [ ] Step 9: Document the API (Swagger)
- [ ] Step 10: Verify logging and error handling

Step 1: Understand the requirements Read the task description, acceptance criteria, and any research documents. Clarify ambiguities before starting.

Step 2: Design the data model Define entities, relationships, indexes, and constraints. Review with the team if the model is non-trivial.

Step 3: Create migration(s) Generate migration files for all schema changes. Ensure both up and down are implemented. Run and verify locally.

Step 4: Implement the domain layer Create entity classes, repository interfaces and implementations, and service classes with business logic. Follow vertical slice structure.

Step 5: Implement the API layer Create controllers with proper HTTP methods. Define DTOs for request/response. Add input validation. Follow the DataGrid standard for list endpoints.

Step 6: Add authentication/authorization guards Apply JWT validation middleware. Add role/permission checks as needed. Implement resource-level authorization.

Step 7: Write integration tests Test every endpoint: success and error paths. Verify response structure, status codes, and database side effects.

Step 8: Write unit tests Test business logic in services. Mock dependencies. Cover edge cases and error scenarios.

Step 9: Document the API Add or update Swagger/OpenAPI documentation. Verify docs render correctly at /api-docs.

Step 10: Verify logging and error handling Ensure requests are logged, errors produce structured log entries, and no sensitive data leaks in logs or responses.

Technology-Specific Patterns

The patterns above are language-agnostic. For technology-specific implementation guidance, load the appropriate reference:

• Node.js: See ./references/nodejs-patterns.md — NestJS/Express DI, Jest/Supertest testing, Pino/Winston logging, TypeORM/Prisma ORM, Swagger integration.
• PHP: See ./references/php-patterns.md — Symfony/Laravel DI, PHPUnit testing, Monolog logging, Doctrine/Eloquent ORM, Swagger integration.
• dotNET: See ./references/dotnet-patterns.md — built-in DI, xUnit testing, Serilog logging, Entity Framework ORM, Swashbuckle Swagger.
• Java: See ./references/java-patterns.md — Spring IoC, JUnit/REST Assured testing, SLF4J/Logback logging, Hibernate ORM, springdoc-openapi.
• Go: See ./references/go-patterns.md — Wire/Fx DI, Go testing, Zap logging, GORM ORM, swaggo Swagger.

Connected Skills

• tsh-sql-and-database-understanding — for database schema design, query optimization, and ORM integration
• tsh-technical-context-discovering — for understanding project conventions before implementing
• tsh-implementation-gap-analysing — for verifying current state before making changes
• tsh-codebase-analysing — for understanding existing architecture and patterns
• tsh-implementing-ci-cd — for CI/CD pipeline setup and deployment strategies
• tsh-implementing-observability — for logging, monitoring, and distributed tracing
• tsh-managing-secrets — for secure credential storage and rotation
• tsh-e2e-testing — for end-to-end testing with Playwright

Connected Skills

• technical-context-discovery — for establishing project conventions before implementing
• architecture-design — for designing complex feature architectures
• code-review — for validating implemented code against these standards
• e2e-testing — for E2E test patterns when full-stack testing is needed