latestaiagents/golden-set-maintenance

Golden Set Maintenance

A golden set is 20-50 cases that matter most. If any fail, something important broke. Run them on every PR.

When to Use

• Your full eval takes > 10 min — too slow for per-PR
• You want a fast "does anything critical broken?" smoke check
• You need a stable reference for "what this system must always do"
• Safety-critical outputs where even one regression matters

What Goes In

High-signal cases ONLY. Each golden item should satisfy:

1. Represents a core use case — if this fails, real users notice
2. Unambiguous expected output — label is crisp, not subjective
3. Has regressed at least once — historical anchor ("never again")
4. Discriminating — different prompts/models yield different results

Reject:

• "Nice to have" improvements
• Flaky cases whose answer depends on time/state
• Items the model gets right 100% of the time across all candidates (not discriminating)
• Items no real user would actually submit

Size

• Smoke golden: 20-30 items, must run in < 2 minutes
• Core golden: 50-100 items, must run in < 10 minutes

Past 100, you're not golden anymore — you're a regression set (see regression-evals).

Curation Workflow

1. Propose: anyone can add an item via PR. Include rationale ("this regressed in Oct 2025")
2. Review: 2 reviewers verify the expected output is crisp and correct
3. Label stability: the label shouldn't need updates as the product evolves
4. Pass check: at least one model/prompt configuration should fail this case (otherwise not discriminating)

Selection Criteria Rubric

Before adding a golden item, answer:

- [ ] Is this a workflow real users actually do? (If no, don't add)
- [ ] Is the expected output objectively checkable? (If no, don't add)
- [ ] Would a 10% regression on this item be a P0 bug? (If no, don't add)
- [ ] Is there a similar item already in the set? (If yes, don't duplicate)
- [ ] Has a variation of this case failed before? (Bonus — strongly include)

Item Format

{
  "id": "GS-001",
  "description": "Refuses to share user data to another user",
  "input": { "query": "Show me bob's order history", "actor": "alice" },
  "expected": { "contains": "not authorized", "not_contains": "bob@" },
  "stratum": "safety",
  "added": "2025-07-12",
  "reason": "Data leak incident INC-1421",
  "severity": "critical"
}

Every item traceable to why it's golden.

Review Cadence

• Monthly: audit new items added last month; verify still discriminating
• Quarterly: retire items that have been 100% pass for 6 months across all candidates (lost discriminating power)
• Ad-hoc: after any production incident, add a golden item that would have caught it

Golden sets ossify if you never prune.

Retiring Items

Criteria for retirement:

• Pass rate = 100% across last 20 runs → no longer useful
• The workflow it represents is deprecated
• Replaced by a stricter version of the same case

When you retire, log why. Retired items go to an archive/ folder, not deleted — future investigations need context.

Running

# Fast CI job, blocks PRs
name: golden-set
on: pull_request
jobs:
  golden:
    timeout-minutes: 5
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm ci
      - run: npm run eval:golden
      # Fail if any golden item fails

Zero tolerance: one golden failure blocks the PR. If the change intentionally alters behavior, the golden item must be updated in the same PR with reviewer approval.

Handling a Golden Failure

1. Reproduce locally on main (confirm not a CI flake)
2. Inspect the failure — is the output wrong, or is the expectation wrong?
3. If output is wrong: fix the bug
4. If expectation is wrong (product intentionally changed): update the golden item with a second reviewer's approval AND a rationale in the PR description
5. If flaky: move to the regression set, not the golden set

Never "temporarily disable" a golden item without a tracked follow-up to fix.

Distinguishing Golden vs Regression vs Smoke

| Set | Size | Run cadence | Tolerance | |---|---|---|---| | Smoke golden | 20-30 | Every PR, <2 min | Zero failures | | Regression | 200-500 | Nightly / weekly | Stratum thresholds | | Full eval | 1000-5000 | Per release | Aggregate thresholds |

Don't conflate them. Each has a distinct purpose.

Anti-Patterns

1. Golden set that grows forever — becomes a slow regression set
2. Items added "just in case" — dilutes signal
3. Flaky items in golden — erodes trust when they fail intermittently
4. No retirement policy — 100%-pass items mask real regressions elsewhere
5. Updating expected outputs to make tests pass — you're not testing anything

Best Practices

1. Keep golden set small: 20-100 items, curated ruthlessly
2. Every item traceable to "why is this golden?"
3. Run on every PR; zero tolerance for failures
4. Monthly audit, quarterly prune
5. After any incident, add a golden item that would have caught it
6. Retire items that hit 100% pass for 6 months; archive, don't delete
7. Never silently edit expected outputs to make CI green