proffesor-for-testing/qe-iterative-loop

QE Iterative Loop

Overview

QE Iterative Loop is a specialized adaptation of the Ralph Wiggum technique for Quality Engineering workflows. It enables autonomous, self-correcting quality cycles where AI agents iterate until quality objectives are achieved - tests pass, coverage targets met, quality gates satisfied, or flaky tests stabilized.

Why QE Benefits from Iteration

Quality Engineering has objective, measurable success criteria:

• Tests either pass or fail (exit code 0 vs non-zero)
• Coverage is quantifiable (78.5% vs 80% target)
• Quality gates have binary outcomes (pass/fail)
• Contract validation has clear schemas

This makes QE ideal for iterative loops - we know exactly when we're done.

Prerequisites

• AQE v3 fleet initialized
• Test framework configured (Jest, Vitest, Pytest, etc.)
• Coverage tooling (c8, istanbul, coverage.py)
• Quality gate definitions

---

Quick Start

Pattern 1: Test Fix Iteration

# Task: Fix all failing tests
/qe-loop "Run npm test and fix all failing tests.
Success: npm test exits with code 0
Output <promise>TESTS_GREEN</promise> when all tests pass."

Pattern 2: Coverage Target Iteration

# Task: Achieve 80% coverage
/qe-loop "Increase test coverage to 80%.
Success: Coverage report shows >= 80%
Output <promise>COVERAGE_MET</promise> when target achieved."

Pattern 3: Quality Gate Iteration

# Task: Pass all quality gates
/qe-loop "Pass all quality gates for deployment.
Gates:
- Unit tests: pass
- Integration tests: pass
- Coverage: >= 80%
- No critical vulnerabilities
- Performance < 200ms P95
Output <promise>QUALITY_GATES_PASSED</promise> when all pass."

---

QE Iteration Patterns

Pattern 1: Test-Fix Iteration Loop

Goal: All tests pass

## QE Test-Fix Loop

### Success Criteria
- `npm test` (or test command) returns exit code 0
- No skipped tests (unless explicitly allowed)
- No pending tests

### Iteration Steps
1. Run full test suite
2. Parse output for failures
3. Analyze first failure:
   - Identify failing test file
   - Understand assertion that failed
   - Check if production code or test is wrong
4. Fix the issue
5. Re-run failed test file only (faster feedback)
6. If file passes, run full suite
7. If all pass -> output <promise>TESTS_GREEN</promise>
8. If failures remain -> continue to next failure

### Safety
- Max iterations: 30
- After 10 iterations: report remaining failures
- Stop if same test fails 5 times (possible design issue)

Pattern 2: Coverage Improvement Loop

Goal: Achieve coverage target

## QE Coverage Loop

### Success Criteria
- Line coverage >= {target}%
- Branch coverage >= {target - 5}% (typically lower target)
- No critical paths uncovered

### Iteration Steps
1. Run tests with coverage: `npm test -- --coverage`
2. Parse coverage report
3. If target met -> output <promise>COVERAGE_MET</promise>
4. Identify uncovered files, sorted by:
   - Critical business logic (highest priority)
   - Lines uncovered (most impact)
   - Complexity (McCabe score)
5. Generate test for highest-impact uncovered code
6. Run tests to verify new test passes
7. Check coverage improvement
8. Continue until target met

### Intelligence Integration
- Store successful test patterns in memory
- Learn from coverage achievements
- Predict best coverage strategies

### Commands
```bash
# Check coverage status (via AQE MCP)
aqe memory get --key "coverage-status" --namespace "coverage"

# Store coverage achievement pattern (via AQE MCP)
aqe memory store \
  --key "coverage-pattern-auth" \
  --value '{"approach": "mock external deps", "improvement": "12%"}' \
  --namespace "coverage-patterns"

### Pattern 3: Quality Gate Compliance Loop

**Goal**: Pass all quality gates

```markdown
## QE Quality Gate Loop

### Gate Definitions
| Gate | Criteria | Priority |
|------|----------|----------|
| unit-tests | All pass | P0 |
| integration-tests | All pass | P0 |
| coverage | >= 80% | P1 |
| lint | No errors | P1 |
| typecheck | No errors | P1 |
| security | No critical/high CVEs | P0 |
| performance | P95 < 200ms | P2 |

### Iteration Strategy
1. Run all gate checks
2. Identify failing gates (sorted by priority)
3. Fix highest-priority failing gate
4. Re-run that gate to verify
5. When gate passes, move to next failing gate
6. When all pass -> output <promise>QUALITY_GATES_PASSED</promise>

### Gate Check Commands
```bash
# Check all gates
npm test && npm run lint && npm run typecheck && npm run coverage && npm audit

# Individual gate checks
npm test                        # unit-tests
npm run test:integration        # integration-tests
npm run coverage               # coverage
npm run lint                   # lint
npx tsc --noEmit               # typecheck
npm audit --audit-level=high   # security
npm run benchmark              # performance

Integration with AQE v3

# Submit quality gate assessment task
aqe quality --runGate true

# Task orchestration for gate compliance
aqe task submit --task "Pass all quality gates" --strategy adaptive

### Pattern 4: Flaky Test Stabilization Loop

**Goal**: Eliminate test flakiness

```markdown
## QE Flaky Test Loop

### Flakiness Detection
1. Run test suite N times (e.g., 5 runs)
2. Identify tests that pass/fail inconsistently
3. Calculate flakiness score: (inconsistent runs / total runs)

### Iteration Steps
1. Run: `for i in {1..5}; do npm test; done`
2. Aggregate results per test
3. Identify flaky tests (passed some, failed some)
4. For each flaky test:
   - Analyze failure modes
   - Common causes:
     - Timing issues (add retries/waits)
     - Shared state (isolate test data)
     - Network calls (mock external services)
     - Random data (use deterministic seeds)
   - Apply appropriate fix
   - Re-run 5 times to verify stability
5. When all tests stable -> output <promise>TESTS_STABLE</promise>

### AQE v3 Flaky Detection
```bash
# Use qe-flaky-hunter agent
Task("Hunt flaky tests", "Detect and stabilize flaky tests", "qe-flaky-hunter")

# Or submit flaky detection task
aqe task submit --type "flaky-detection" --priority "p1"

### Pattern 5: Contract Validation Loop

**Goal**: API contracts aligned

```markdown
## QE Contract Loop

### Success Criteria
- Provider implements all consumer contracts
- No breaking changes detected
- Schema validation passes

### Iteration Steps
1. Run contract tests: `npm run test:contracts`
2. Parse contract violations
3. For each violation:
   - Determine if provider or consumer needs update
   - Update appropriate side
   - Re-run contract tests
4. When all contracts valid -> output <promise>CONTRACTS_VALID</promise>

### AQE v3 Integration
```bash
# Validate contracts
aqe test contract --contractPath "./contracts"

# Or use specialized agent
Task("Validate API contracts", "Check consumer-provider alignment", "qe-contract-validator")

---

## AQE v3 Fleet Integration

### Spawning QE Iteration Agents

```bash
# Initialize AQE fleet for QE iteration
aqe fleet init --topology "hierarchical" --maxAgents 8

# Spawn specialized QE iterators using Task tool
Task("Fix failing tests", "Iterate until all tests pass", "qe-tdd-green", {run_in_background: true})
Task("Improve coverage", "Iterate until 80% coverage", "qe-coverage-analyzer", {run_in_background: true})
Task("Fix security issues", "Iterate until security scan passes", "qe-security-scanner", {run_in_background: true})
Task("Stabilize flaky tests", "Iterate until tests stable", "qe-flaky-hunter", {run_in_background: true})

Memory-Enhanced QE Iteration

# Store iteration patterns for learning (via AQE MCP)
aqe memory store \
  --key "qe-iteration-test-fix" \
  --value '{"approach": "mock external deps", "success_rate": 0.85}' \
  --namespace "qe-patterns"

# Search for relevant QE patterns (via AQE MCP)
aqe memory search \
  --pattern "test-fix-*" \
  --namespace "qe-patterns"

# Record successful iteration completion (via AQE task tracking)
aqe task status --taskId "test-fix-iteration"

QE-Specific Agent Routing

| QE Task | Recommended Agent | Iteration Goal | |---------|-------------------|----------------| | Test fixes | qe-tdd-green | All tests pass | | Coverage gaps | qe-coverage-analyzer | Target coverage met | | Quality gates | qe-quality-gate | All gates pass | | Flaky tests | qe-flaky-hunter | Tests stable | | Contract validation | qe-contract-validator | Contracts aligned | | Security fixes | qe-security-scanner | No vulnerabilities | | Performance | qe-performance-validator | Benchmarks pass |

---

Completion Promises for QE

Standard QE Promises

# Test-related
<promise>TESTS_GREEN</promise>       # All tests pass
<promise>TESTS_STABLE</promise>      # Flaky tests fixed
<promise>TDD_COMPLETE</promise>      # TDD cycle done

# Coverage-related
<promise>COVERAGE_MET</promise>      # Target coverage achieved
<promise>GAPS_FILLED</promise>       # Coverage gaps addressed

# Quality gates
<promise>QUALITY_GATES_PASSED</promise>  # All gates pass
<promise>DEPLOYMENT_READY</promise>      # Ready for deploy

# Contract/API
<promise>CONTRACTS_VALID</promise>   # Contracts aligned
<promise>API_COMPLIANT</promise>     # API matches spec

# Security
<promise>SECURITY_CLEARED</promise>  # No vulnerabilities
<promise>COMPLIANCE_MET</promise>    # Compliance requirements met

# Performance
<promise>PERF_TARGET_MET</promise>   # Benchmarks satisfied

---

Example: Full QE Iteration Workflow

## Complete QE Iteration Task

### Objective
Achieve deployment readiness through iterative quality improvement

### Phase 1: Test Health (Priority)
1. Run `npm test`
2. Fix failing tests iteratively
3. Success: <promise>TESTS_GREEN</promise>

### Phase 2: Coverage (After Phase 1)
1. Run `npm test -- --coverage`
2. Write tests for uncovered critical paths
3. Success: Coverage >= 80% -> <promise>COVERAGE_MET</promise>

### Phase 3: Quality Gates (After Phase 2)
1. Run lint: `npm run lint`
2. Run typecheck: `npx tsc --noEmit`
3. Fix any violations
4. Success: <promise>LINT_PASS</promise> + <promise>TYPES_PASS</promise>

### Phase 4: Security (Parallel with Phase 3)
1. Run `npm audit`
2. Fix critical/high vulnerabilities
3. Success: <promise>SECURITY_CLEARED</promise>

### Phase 5: Integration
1. Run `npm run test:integration`
2. Fix any integration failures
3. Success: <promise>INTEGRATION_PASS</promise>

### Final Gate
When ALL phases complete -> <promise>DEPLOYMENT_READY</promise>

### Safety Limits
- Max iterations per phase: 15
- Total max iterations: 50
- Stuck detection: 5 iterations without progress triggers escalation

---

Troubleshooting

Issue: Tests Keep Failing Same Assertion

Cause: Likely a design issue, not implementation bug

Solution:

1. Stop iteration after 5 attempts on same test
2. Analyze if test expectation is correct
3. Review if production behavior is as designed
4. Escalate to human review if unclear

Issue: Coverage Plateau

Cause: Remaining uncovered code is complex/conditional

Solution:

1. Identify uncovered branches (not just lines)
2. Generate edge case tests
3. Consider if uncovered code is dead code
4. Accept lower target for genuinely untestable code

Issue: Flaky Tests Won't Stabilize

Cause: Deep timing or state issues

Solution:

1. Add explicit waits/retries
2. Mock time-dependent behavior
3. Isolate test environment
4. Consider marking as skip with explanation

---

Related Skills

• iterative-loop - General iteration technique
• qe-test-generation - AI-powered test creation
• qe-coverage-analysis - Coverage gap detection
• qe-quality-assessment - Quality gate management
• qe-chaos-resilience - Resilience iteration testing

Resources

• AQE v3 Documentation - Full v3 reference
• Ralph Wiggum Technique - Original methodology
• Agentic QE MCP Tools - MCP tool reference

---

Origin: Adapted from Ralph Wiggum plugin (anthropics/claude-code) Specialized for: Agentic QE v3 Fleet with 60 QE agents Domains: test-generation, test-execution, coverage-analysis, quality-assessment