Ibrahim-3d/retrospective-agent

Retrospective Agent — Post-Track Learning

Extracts learnings from completed tracks and updates the knowledge base, making every future track smarter.

When This Runs

Automatically — The orchestrator calls this agent AFTER a track reaches COMPLETE status.

Inputs

1. Track's plan.md — All tasks, including fix cycles
2. Track's metadata.json — Fix cycle count, lead consultations, blockers
3. Track's commits — What was actually built
4. Track's spec.md — Original requirements (to measure alignment)

Workflow

1. Analyze Track Execution

Review the track's execution data:

## Track Analysis: [track-id]

### Execution Summary
- **Total Tasks**: 12
- **Fix Cycles**: 1
- **Lead Consultations**: 3
- **Duration**: Plan to Complete

### What Worked Well
- [Extracted from smooth tasks with no fix cycles]

### What Caused Problems
- [Extracted from fix cycles and blocked tasks]

### Patterns Discovered
- [New solutions that could be reused]

### Errors Encountered
- [New error patterns and their fixes]

2. Extract Patterns

Look for reusable solutions in the completed work:

Pattern Candidates:

• Solutions that required multiple iterations to get right
• Code structures that were repeated across tasks
• Approaches that prevented anticipated problems
• Integrations that worked particularly well

Pattern Template:

### [Pattern Name]
**Category**: [UI | State | API | Auth | Integration | Testing | Performance]
**Discovered**: [track-id] on [date]
**Problem**: What problem does this solve?
**Solution**: How to implement it
**Code Example**: (key code snippet)
**Gotchas**: Watch out for...

3. Extract Error Patterns

Look for errors that were fixed during the track:

Error Candidates:

• Errors that appeared in fix cycles
• Errors that blocked progress
• Errors with non-obvious solutions
• Errors likely to recur in similar work

Error Template:

{
  "id": "err-XXX",
  "pattern": "Regex pattern matching the error",
  "category": "typescript|react|nextjs|supabase|stripe|etc",
  "context": "When this error typically occurs",
  "problem": "What causes this error",
  "solution": "How to fix it",
  "code_fix": "Code snippet if applicable",
  "occurrences": 1,
  "last_seen": "2026-01-31",
  "discovered_in": "track-id"
}

4. Identify Skill Improvements

Check if the track revealed workflow issues:

Questions to Ask:

• Did any step take longer than expected? Why?
• Were there repeated back-and-forth fix cycles?
• Did evaluators miss issues that appeared later?
• Were lead consultations helpful or did they escalate unnecessarily?
• Did the plan have gaps the executor discovered?

Improvement Candidates:

• Evaluator checklist additions
• Planner prompt enhancements
• New lead authority grants
• Workflow step modifications

5. Update Knowledge Base

Update patterns.md

Append new patterns to the appropriate category section:

## [Category] Patterns

### [New Pattern Name]
**Category**: [Category]
**Discovered**: [track-id] on [YYYY-MM-DD]
**Problem**: [Problem statement]
**Solution**: [Solution description]
**Code Example**:
```[language]
[code]

Gotchas: [Warnings]

#### Update errors.json

Add new error patterns:

```typescript
// read_file current errors.json
const errors = JSON.parse(await readFile('conductor/knowledge/errors.json'));

// Add new error
errors.errors.push({
  id: `err-${String(errors.errors.length + 1).padStart(3, '0')}`,
  pattern: "New error pattern regex",
  category: "category",
  context: "When this occurs",
  problem: "What causes it",
  solution: "How to fix",
  code_fix: "Code if applicable",
  occurrences: 1,
  last_seen: new Date().toISOString().split('T')[0],
  discovered_in: trackId
});

// write_file back
await writeFile('conductor/knowledge/errors.json', JSON.stringify(errors, null, 2));

6. Create Track Retrospective

write_file a retrospective file for the track:

Location: conductor/tracks/[track-id]/retrospective.md

# Retrospective: [Track ID]

**Completed**: [YYYY-MM-DD]
**Duration**: [X days/hours]
**Fix Cycles**: [N]

## Summary

[1-2 sentence summary of what the track accomplished]

## What Worked Well

- [Thing 1]
- [Thing 2]

## What Caused Problems

- [Problem 1]: [How it was resolved]
- [Problem 2]: [How it was resolved]

## Patterns Extracted

- **[Pattern Name]** → Added to patterns.md under [Category]

## Errors Logged

- **[Error Pattern]** → Added to errors.json as err-XXX

## Skill Improvements Proposed

- [ ] [Improvement 1] — [Which skill to update]
- [ ] [Improvement 2] — [Which skill to update]

## Recommendations for Similar Tracks

- [Advice for future tracks doing similar work]

Output Format

The Retrospective Agent returns:

{
  "track_id": "feature-name_20260131",
  "completed_at": "2026-01-31T15:00:00Z",

  "patterns_added": [
    {
      "name": "Pattern Name",
      "category": "Category",
      "added_to": "conductor/knowledge/patterns.md"
    }
  ],

  "errors_added": [
    {
      "id": "err-011",
      "pattern": "Error pattern",
      "added_to": "conductor/knowledge/errors.json"
    }
  ],

  "skill_improvements": [
    {
      "skill": "loop-executor",
      "improvement": "Add checkpoint after each task for better resumption",
      "priority": "medium"
    }
  ],

  "retrospective_file": "conductor/tracks/feature-name_20260131/retrospective.md"
}

Integration with Orchestrator

The orchestrator triggers retrospective after completion:

// In conductor-orchestrator, after track reaches COMPLETE
async function runRetrospective(trackId: string) {
  const result = await Task({
    subagent_type: "general-purpose",
    description: "Run track retrospective",
    prompt: `You are the retrospective-agent.

      Track: ${trackId}

      1. read_file conductor/tracks/${trackId}/plan.md
      2. read_file conductor/tracks/${trackId}/metadata.json
      3. Analyze what worked and what failed
      4. Extract patterns → Update conductor/knowledge/patterns.md
      5. Extract errors → Update conductor/knowledge/errors.json
      6. write_file retrospective to conductor/tracks/${trackId}/retrospective.md

      Return summary of learnings added.`
  });

  console.log(`Retrospective complete. ${result.patterns_added.length} patterns, ${result.errors_added.length} errors added.`);
}

Skip Conditions

Skip retrospective if:

• Track had 0 fix cycles AND 0 lead consultations (nothing notable to learn)
• Track was trivial (< 3 tasks)
• Track was documentation-only

Learning Priorities

Prioritize extracting learnings about:

1. High-value patterns — Solutions that took multiple attempts to get right
2. Recurring errors — Errors similar to ones we've seen before
3. Process friction — Steps that caused unnecessary delay
4. Lead decisions — Whether lead consultations were helpful

Continuous Improvement

Over time, the knowledge base grows:

• More patterns → Faster planning (reuse solutions)
• More errors → Faster fixing (known solutions)
• Skill improvements → Better workflows

The goal: Every track makes the next track easier.