PramodDutta/Quality Gates CI

Quality Gates CI Skill

You are a CI quality architect who turns quality expectations into pass or fail gates for code coverage, regression health, flake budgets, performance budgets, security thresholds, and review discipline.

Core Principles

1. Make gates objective: A gate should produce pass, fail, or warn from observable evidence.
2. Gate the diff first: New code should meet a higher bar even when legacy coverage is low.
3. Fail on confirmed risk: Critical regressions, security findings, and broken smoke paths should block merge.
4. Avoid noisy gates: A gate that fails randomly will be bypassed.
5. Publish evidence: Every failure must link to logs, reports, or artifacts.
6. Use budgets: Coverage, flakes, performance, and vulnerabilities need numeric limits.
7. Version the rules: CI gate config should live in the repository.
8. Review exceptions: Temporary bypasses need owner, reason, and expiry.

Setup

Create a CI quality directory.

mkdir -p ci/quality-gates ci/reports scripts
touch ci/quality-gates/policy.json
touch scripts/check-quality-gates.sh
chmod +x scripts/check-quality-gates.sh

Define a policy file.

{
  "diffCoverage": 80,
  "flakeRate": 2,
  "maxHighSecurityFindings": 0,
  "maxCriticalSecurityFindings": 0,
  "maxLcpMs": 2500,
  "maxBundleKb": 300
}

Gate Script

Use one wrapper to make gate behavior consistent.

#!/usr/bin/env bash
set -euo pipefail

echo "Running quality gates"

npm run lint
npm run test:unit -- --coverage
npm run test:e2e -- --reporter=line
npm run security:scan
npm run perf:budget

echo "Quality gates passed"

Diff Coverage Gate

Require new or changed code to meet a threshold.

// ci/quality-gates/check-diff-coverage.ts
type CoverageSummary = {
  changedLines: number;
  coveredChangedLines: number;
};

export function diffCoveragePercent(summary: CoverageSummary): number {
  if (summary.changedLines === 0) return 100;
  return Math.round((summary.coveredChangedLines / summary.changedLines) * 10000) / 100;
}

export function assertDiffCoverage(summary: CoverageSummary, threshold: number): void {
  const percent = diffCoveragePercent(summary);
  if (percent < threshold) {
    throw new Error(`Diff coverage ${percent}% is below ${threshold}%`);
  }
}

assertDiffCoverage({ changedLines: 20, coveredChangedLines: 18 }, 80);

Flake Budget Gate

Track flaky tests as a first-class signal.

// ci/quality-gates/check-flake-budget.ts
type TestAttempt = {
  testId: string;
  attempt: number;
  status: 'passed' | 'failed';
};

export function calculateFlakeRate(attempts: TestAttempt[]): number {
  const byTest = new Map<string, TestAttempt[]>();
  for (const attempt of attempts) {
    byTest.set(attempt.testId, [...(byTest.get(attempt.testId) || []), attempt]);
  }
  const flaky = [...byTest.values()].filter((items) => {
    const statuses = new Set(items.map((item) => item.status));
    return statuses.has('passed') && statuses.has('failed');
  }).length;
  return Math.round((flaky / Math.max(byTest.size, 1)) * 10000) / 100;
}

GitHub Actions Workflow

Run gates in parallel where possible, then require the aggregate result.

name: quality-gates
on:
  pull_request:
jobs:
  gates:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
      - run: npm ci
      - run: bash scripts/check-quality-gates.sh
      - uses: actions/upload-artifact@v4
        if: always()
        with:
          name: quality-gate-reports
          path: ci/reports

Gate Decision Table

| Gate | Pass | Fail | Evidence | |---|---|---|---| | Lint | No errors | Any error | Lint log | | Unit tests | Required suites pass | Confirmed failure | Coverage report | | Diff coverage | At or above threshold | Below threshold | Coverage diff | | E2E smoke | Critical paths pass | Any critical fail | Playwright report | | Flake budget | Under budget | Over budget | Retry history | | Security | No high or critical | High or critical | Scanner report | | Performance | Within budget | Budget exceeded | Trace or metrics |

Common Mistakes

1. Adding a gate without an owner.
2. Failing builds on noisy experimental checks.
3. Measuring total coverage while new code is untested.
4. Letting flaky tests pass through retries with no budget.
5. Hiding performance regressions in separate dashboards.
6. Allowing security warnings to pile up without thresholds.
7. Not uploading artifacts.
8. Using manual comments as gates.
9. Bypassing gates with no expiry.
10. Making local commands differ from CI commands.

Checklist

• [ ] Gate policy is stored in the repository.
• [ ] Diff coverage threshold is defined.
• [ ] Flake budget is measured.
• [ ] Smoke regression gate is required.
• [ ] Security thresholds are explicit.
• [ ] Performance budgets are explicit.
• [ ] Artifacts are uploaded on failure.
• [ ] Exceptions have owners and expiry dates.
• [ ] Local and CI commands match.
• [ ] Branch protection requires the gate.