laurigates/test-quality-analysis

Test Quality Analysis

Expert knowledge for analyzing and improving test quality - detecting test smells, overmocking, insufficient coverage, and other testing anti-patterns.

When to Use This Skill

| Use this skill when... | Use another skill instead when... | |------------------------|----------------------------------| | Reviewing test quality and smells | Writing new unit tests (use vitest-testing) | | Detecting overmocking or flaky tests | Setting up E2E tests (use playwright-testing) | | Analyzing test coverage gaps | Validating tests via mutation (use mutation-testing) | | Improving test maintainability | Generating test data (use property-based-testing) | | Auditing test suites for anti-patterns | Writing Python tests (use python-testing) |

Core Expertise

Test Quality Dimensions

• Correctness: Tests verify the right behavior
• Reliability: Tests are deterministic and not flaky
• Maintainability: Tests are easy to understand and modify
• Performance: Tests run quickly
• Coverage: Tests cover critical code paths
• Isolation: Tests don't depend on external state

Test Smells - Quick Reference

| Smell | Symptom | Fix | |-------|---------|-----| | Overmocking | 3+ mocks per test; mocking pure functions | Mock only I/O boundaries; use real implementations | | Fragile tests | Break on refactor without behavior change | Test public APIs; use semantic selectors | | Flaky tests | Non-deterministic pass/fail | Proper async/await; mock time; ensure isolation | | Test duplication | Copy-pasted setup across tests | Extract to beforeEach(), fixtures, helpers | | Slow tests | Suite > 10s for unit tests | beforeAll() for expensive setup; parallelize | | Poor assertions | toBeDefined(), no assertions, mock-only assertions | Specific matchers; assert outputs and state | | Insufficient coverage | Critical paths untested | 80%+ on business logic; test error paths and boundaries |

Analysis Tools

TypeScript/JavaScript

vitest --coverage                              # Coverage report
vitest --coverage --coverage.thresholds.lines=80  # Threshold check
vitest --reporter=verbose                      # Find slow tests

Python

uv run pytest --cov --cov-report=term-missing  # Coverage with missing lines
uv run pytest --cov --cov-fail-under=80        # Threshold check
uv run pytest --durations=10                   # Find slow tests

Key Anti-Patterns

Testing Implementation vs Behavior

// BAD: Testing how
test('uses correct algorithm', () => {
  const spy = vi.spyOn(Math, 'sqrt')
  calculateDistance({ x: 0, y: 0 }, { x: 3, y: 4 })
  expect(spy).toHaveBeenCalled()
})

// GOOD: Testing what
test('calculates distance correctly', () => {
  const distance = calculateDistance({ x: 0, y: 0 }, { x: 3, y: 4 })
  expect(distance).toBe(5)
})

Weak Assertions

// BAD
expect(users).toBeDefined()     // Too vague
expect(mockAPI).toHaveBeenCalled() // Testing mock, not behavior

// GOOD
expect(user).toMatchObject({
  id: expect.any(Number),
  name: 'John',
  email: '[email protected]',
})

Missing Coverage

// BAD: Only tests happy path
test('applies discount', () => {
  expect(calculateDiscount(100, 'SAVE20')).toBe(80)
})

// GOOD: Tests all paths
describe('calculateDiscount', () => {
  it('applies SAVE20', () => expect(calculateDiscount(100, 'SAVE20')).toBe(80))
  it('applies SAVE50', () => expect(calculateDiscount(100, 'SAVE50')).toBe(50))
  it('invalid coupon', () => expect(calculateDiscount(100, 'INVALID')).toBe(100))
  it('no coupon', () => expect(calculateDiscount(100)).toBe(100))
})

Test Structure (AAA Pattern)

test('user registration flow', async () => {
  // Arrange
  const userData = { email: '[email protected]', password: 'secure123' }
  const mockEmailService = vi.fn()

  // Act
  const user = await registerUser(userData, mockEmailService)

  // Assert
  expect(user).toMatchObject({ email: '[email protected]', emailVerified: false })
  expect(mockEmailService).toHaveBeenCalledWith('[email protected]', expect.any(String))
})

Agentic Optimizations

| Context | Command | |---------|---------| | Coverage check (TS) | vitest --coverage --reporter=dot | | Coverage check (Python) | uv run pytest --cov --cov-fail-under=80 -q | | Find slow tests (TS) | vitest --reporter=verbose | | Find slow tests (Python) | uv run pytest --durations=10 -q | | Missing lines (Python) | uv run pytest --cov --cov-report=term-missing |

For detailed examples, advanced patterns, and best practices, see REFERENCE.md.

See Also

• vitest-testing - TypeScript/JavaScript testing
• python-testing - Python pytest testing
• playwright-testing - E2E testing
• mutation-testing - Validate test effectiveness

References

• Test Smells: https://testsmells.org/
• Test Double Patterns: https://martinfowler.com/bliki/TestDouble.html
• Testing Best Practices: https://kentcdodds.com/blog/common-mistakes-with-react-testing-library