etanhey/research-lifecycle

Research Lifecycle — Context Freshness Management

Anti-pattern: R75 says "FTS5 is 96.9% desynced" — but it's been fixed for days. Without lifecycle management, R76 starts from wrong assumptions.

Research context files live at docs.local/claude-web/projects/<project>/. After a sprint implements research findings, these files go STALE. This skill manages the lifecycle so the next research round starts from truth, not assumptions.

---

THE LIFECYCLE

SPRINT COMPLETES (findings implemented)
  │
  ├─ 1. NEWEST: Add sprint results as new context file
  │     (what changed, benchmark numbers, PRs merged)
  │
  ├─ 2. CONDENSED: Compress older context files
  │     (files 01-50 → one condensed file)
  │
  ├─ 3. UPDATED: Refresh description.md with current state
  │     (facts first, narrative second)
  │
  ├─ 4. ARCHIVED: Move implemented research results
  │     (R-numbers whose findings are now in code)
  │
  └─ 5. MIRRORED: Update NotebookLM sources
        (swap stale for updated files)

---

STEP 1: NEWEST — Add Sprint Results

After a sprint that implements research findings, create a new context file:

# File: research-context/NN-sprint-results-{date}.md

## Sprint: {description}
- Date: {date}
- PRs merged: #{N}, #{N}, #{N}

## What Changed
- {specific change 1}: {before state} → {after state}
- {specific change 2}: {before} → {after}

## Benchmark Numbers
- {metric}: {old value} → {new value}
- {metric}: {old} → {new}

## What's Still Open
- {remaining issue 1}
- {remaining issue 2}

Key: Include BEFORE and AFTER states. The researcher needs to know what was the problem AND what's the current state.

Naming: Use sequential numbers (91-, 92-, 93-...) to preserve ordering. Prefix with the topic.

---

STEP 2: CONDENSED — Compress Stale Context

When context files accumulate (>10 files or >50KB total), condense older ones:

BEFORE:
  91-swift-python-root-cause.md     (from 2 weeks ago, findings implemented)
  92-brainbar-ux-current-state.md   (from 1 week ago, partially stale)
  93-inspiration-references.md       (timeless, keep as-is)
  94-search-improvement-collab.md   (from yesterday, still current)

AFTER:
  condensed-pre-R75.md              (91 + 92 merged, updated to current state)
  93-inspiration-references.md       (unchanged — timeless content)
  94-search-improvement-collab.md   (unchanged — still current)
  95-R75-sprint-results.md          (NEW — what the sprint changed)

Condensation rules:

• Merge files about the SAME topic into one
• Update claims to current state (remove "FTS5 is broken" if it's fixed)
• Keep timeless content as-is (references, architecture decisions with rationale)
• Remove redundant context (if 3 files say "BrainBar is Swift", keep it in one place)

---

STEP 3: UPDATED — Refresh description.md

The project description file tells the researcher what the project IS. It must reflect current state.

Location: docs.local/claude-web/projects/<project>/description.md

Refresh checklist:

• [ ] Architecture claims match current code (check recent PRs)
• [ ] Stats are current (chunk count, entity count, test count)
• [ ] Open issues are actually still open (not fixed in latest sprint)
• [ ] "Known limitations" haven't been resolved
• [ ] No claims about features that are stubs (check with /never-fabricate)

Format: Facts first, narrative second. Lead with numbers, follow with explanation.

## Current State (as of {date})
- Chunks: 299,972 (FTS5: 100% synced ← was 3.1% before R75 sprint)
- Entities: 166 (0.055% coverage ← needs improvement)
- Tests: 675 passing
- MCP: BrainBar (Swift daemon), 5 working tools + 3 stubs

## Architecture
{current architecture, verified against code}

---

STEP 4: ARCHIVED — Move Implemented Research

Research results (R-number files) whose findings are fully implemented should move to archived:

# Move completed research
mv docs.local/claude-web/projects/brainlayer/R38-knowledge-graph.md \
   docs.local/claude-web/projects/archived/brainlayer/

Archive criteria:

• ALL findings from the research are implemented (check PRs)
• Benchmark improvements are captured in a sprint results file
• The research is >1 sprint old (keep recent research active for reference)

DON'T archive:

• Research with partially-implemented findings (keep, mark which are done)
• Timeless architectural decisions (keep in main project)
• The LATEST research result (always keep the most recent R-number)

---

STEP 5: MIRRORED — Update NotebookLM Sources

If the project has a NotebookLM notebook (see /gemini-research):

1. Delete stale sources from the notebook:

   source_delete(source_id=stale_source, confirm=True)
   

2. Add updated files:

   source_add(notebook_id, source_type="file", file_path="updated-context.md", wait=True)
   

3. Update notebook instructions:

   chat_configure(notebook_id, goal="custom",
     custom_prompt="Context files have been updated as of {date}. Previous findings about {X} are now implemented.")
   

Remember NP3: .py/.json/.swift/.yaml files must be renamed to .txt for NotebookLM upload.

---

WHEN TO TRIGGER

| Trigger | Action | |---------|--------| | Sprint merged PRs that address research findings | Run full lifecycle (Steps 1-5) | | Before writing a new R-number research prompt | Run Steps 2-4 (condense, update, archive) | | Nightly/project maintenance runs | Check if context files are stale (Step 3 at minimum) | | Context file count > 10 | Run Step 2 (condense) | | NotebookLM notebook exists | Run Step 5 (mirror) |

---

STALENESS DETECTION

A context file is STALE when:

1. It describes a problem that's been fixed — e.g., "FTS5 is 96.9% desynced" after PR fixing it
2. Its benchmark numbers are outdated — e.g., "675 tests" when there are now 720
3. Its architecture claims are wrong — e.g., "Python MCP" when it's now Swift BrainBar
4. It references PRs as "open" that are merged — check via gh pr view
5. It's >2 sprints old and never updated — age alone signals staleness

Quick staleness check:

# Find context files older than 7 days with no updates
find docs.local/claude-web/projects/PROJECT/research-context/ -name "*.md" -mtime +7

---

INTEGRATION

| Skill | How Research Lifecycle Uses It | |-------|------------------------------| | Claude/Gemini research runs | Create research results → lifecycle manages them afterward | | /gemini-research | NotebookLM file type restrictions apply to mirroring (Step 5) | | Nightly/project maintenance | Should check research context staleness during sweeps | | /never-fabricate | Verify claims in description.md against actual code before updating | | /orc W5 | Claude Web research prompts consume these context files |

---

EXAMPLE: Post-R75 Lifecycle

R75 sprint fixed FTS5 sync, implemented detect_entities_in_prompt, improved search.

1. NEWEST: Create 95-R75-sprint-results.md with: FTS5 3.1%→100%, entity detection implemented, search hybrid mode live
2. CONDENSED: Merge 91-swift-python-root-cause.md + 92-brainbar-ux-current-state.md → condensed-pre-R75.md (update "FTS5 broken" → "FTS5 fixed in PR #198")
3. UPDATED: Refresh description.md — chunk count, entity count, test count, remove "FTS5 broken" from known issues
4. ARCHIVED: Move R38-knowledge-graph.md (fully implemented) to archived/brainlayer/
5. MIRRORED: Delete old NLM sources, upload updated files