LIFEAI/rdc-extract-verifier-rules

rdc:extract-verifier-rules

The self-learning loop. The verifier corpus is the moat (per DECISIONS-LOG.md D-009). This skill is how the corpus grows.

What it does

1. Read enhancement-log.jsonl entries since last run timestamp
2. Cluster entries by fingerprint similarity + reason text
3. For each cluster with N >= 3 occurrences:
   • Generate a candidate regex or AST pattern
   • Test the pattern against the known-good corpus (last 100 successful brochures) for false positives
   • Test the pattern against the failure cluster for recall
   • If false-positive rate = 0 AND recall ≥ 80%, mark as propose
4. For each propose candidate:
   • Open a PR on regen-root adding the rule to verifiers/forbidden-patterns.json
   • Auto-merge if confidence ≥ 95% AND a maintainer's auto-merge flag is enabled
   • Otherwise, await human review
5. Write a markdown summary to .rdc/reports/verifier-{YYYY-MM-DD}.md

Cluster algorithm

Two-stage clustering:

Stage 1 — Fingerprint clustering:

• Each enhancement-log entry has an input_fingerprint (sha256 of the offending JSX block)
• Entries with identical fingerprints cluster trivially (same exact mistake)

Stage 2 — Semantic clustering:

• For unique fingerprints, embed the reason and pattern fields with @xenova/transformers (384-dim, local — no API)
• Cosine similarity ≥ 0.85 → same cluster
• Use HDBSCAN with min_samples=3 to identify real clusters vs. noise

Candidate rule generation

For each cluster, generate one of:

Regex candidate

• Take all pattern strings in the cluster
• Find common substring or template
• Generalize literal values to character classes
• Validate the regex is well-formed and not catastrophic

Example:

Cluster patterns:
  <div className="flex flex-col gap-4">
  <div className="flex flex-col gap-6">
  <div className="flex flex-row items-center">

Generated regex:
  <div[^>]*className="[^"]*\\bflex\\b

AST candidate

• For cluster entries with rich AST context, build an AST query string
• Test with @typescript-eslint/parser

Example:

AST candidate:
  JSXOpeningElement[name.name='div'] > JSXAttribute[name.name='className'] CallExpression[callee.name='clsx']

False-positive check

Run the candidate against the known-good corpus:

• The 12 Phase 1 reference brochures
• All design-partner brochures with final_grade >= 85
• Any brochure tagged corpus:reference in Supabase

If the candidate matches any known-good output → reject the candidate. False positives in the verifier corpus are unacceptable because they break working flows.

Recall check

Run the candidate against the failure cluster:

• Must match ≥ 80% of cluster entries
• Below 80%, the candidate is too specific; widen and retry

PR generation

For each promoted candidate, create a PR on LIFEAI/regen-root:

Title: [verifier-corpus] Add rule FP0{N} — {short description}

Body:
This rule was generated by rdc:extract-verifier-rules on {date}.

**Cluster size:** {N} occurrences over {time span}
**False positive rate:** {rate}
**Recall:** {percent}

**Sample failures:**
{3-5 example enhancement-log entries}

**Generated rule:**
```json
{rule JSON}

Reviewer checklist:

• [ ] Verify the rule doesn't match any known-good brochure
• [ ] Verify the suggested fix is reasonable
• [ ] Confirm severity level
• [ ] Confirm category

Auto-merging: {yes/no based on confidence}

## Auto-merge policy

Auto-merge if **all** are true:
- False-positive rate = 0
- Recall ≥ 95%
- Cluster size ≥ 10
- The rule pattern is a strict subset of an existing pattern (not a fundamentally new category)
- A maintainer's auto-merge flag is enabled in `.rdc/config.json`

Otherwise, await human review.

## Output: nightly report

`.rdc/reports/verifier-{YYYY-MM-DD}.md`:

```markdown
# Verifier Corpus Update — {date}

## Summary

- Enhancement log entries since last run: {N}
- Clusters identified: {C}
- Rules proposed: {R}
- Rules auto-merged: {A}
- Rules awaiting review: {W}

## Top clusters

| Cluster | Size | Pattern | Status |
|---|---|---|---|
| ...

## Open PRs

- #{N}: FP0{X} — {description}
- ...

## False-positive rejections

Candidates that failed the known-good test:
- {summary}

## Corpus growth

- Total rules in forbidden-patterns.json: {before} → {after}
- Total component-allowlist entries: unchanged
- Pagination rules: unchanged

Manual invocation

A maintainer can run this skill manually after a significant brochure run to immediately promote lessons learned:

rdc:extract-verifier-rules --since "2026-05-27" --auto-merge=false

The --auto-merge=false flag forces human review on all proposals regardless of confidence.

Why this matters

The verifier corpus is the asset. Anyone can copy the kit. Anyone can copy the ESLint plugin. Replicating 12 months of customer-tested failure patterns is the moat. This skill is how that moat compounds. Every brochure run feeds it. Every nightly run grows it.

After 12 months at moderate scale, the corpus will contain ~500-2,000 rules, calibrated against real customer documents, with each rule's hit count visible. New competitors entering this category will face a starting position 12 months behind.

That is the point.