pjt222/repair-damage

Schaeden reparieren

Implementieren regenerative recovery for systems that have sustained structural damage — whether from incidents, failed migrations, accumulated neglect, or external disruption. Uses biological wound-healing as a framework: triage, stabilization, scaffolding, progressive rebuild, and scar tissue management.

Wann verwenden

• A system has suffered an incident and needs structured recovery beyond "just fix it"
• A failed transformation (see adapt-architecture) left das System in a damaged intermediate state
• Accumulated technical debt has caused partial system failure
• Organizational damage (team departures, knowledge loss, morale collapse) needs structured repair
• Post-defense recovery (see defend-colony) when the colony sustained damage
• A system is functional but degraded, and the degradation is worsening

Eingaben

• Erforderlich: Description of the damage (what broke, when, how severely)
• Erforderlich: Current system state (what's still working, what's not)
• Optional: Root cause (if known — may not be clear yet)
• Optional: Pre-damage system state (for comparison)
• Optional: Available repair resources (time, people, budget)
• Optional: Urgency (is das System actively degrading or stable-but-damaged?)

Vorgehensweise

Schritt 1: Triage — Bewerten and Classify Wounds

Rapidly assess all damage and classify by severity and urgency.

1. Catalog every known point of damage:
   • What specific component, function, or capability is affected?
   • Is the damage complete (non-functional) or partial (degraded)?
   • Is the damage spreading (affecting adjacent components) or contained?
2. Classify each wound:

Wound Classification:
┌──────────┬──────────────────────┬────────────────────────────────────┐
│ Class    │ Severity             │ Response                           │
├──────────┼──────────────────────┼────────────────────────────────────┤
│ Critical │ Core function lost,  │ Immediate: stop bleeding, activate │
│          │ data at risk,        │ backup, redirect traffic, page     │
│          │ actively spreading   │ on-call team                       │
├──────────┼──────────────────────┼────────────────────────────────────┤
│ Serious  │ Important function   │ Urgent: fix within hours/days,     │
│          │ degraded, no spread  │ workarounds acceptable short-term  │
├──────────┼──────────────────────┼────────────────────────────────────┤
│ Moderate │ Non-critical function│ Scheduled: fix within sprint,      │
│          │ affected, contained  │ prioritize against other work      │
├──────────┼──────────────────────┼────────────────────────────────────┤
│ Minor    │ Cosmetic or edge     │ Backlog: fix when convenient,      │
│          │ case, no user impact │ may self-resolve                   │
└──────────┴──────────────────────┴────────────────────────────────────┘

3. Priorisieren repair order:
   • Critical wounds first (stop the bleeding)
   • Then serious wounds (restore important function)
   • Moderate and minor wounds can wait for scheduled repair
4. Pruefen auf wound interaction:
   • Do any wounds amplify each other? (A is worse because B is also broken)
   • Would fixing one wound automatisch fix others? (shared root cause)
   • Would fixing one wound make another worse? (competing repair strategies)

Erwartet: A complete wound inventory classified by severity, with a prioritized repair order that accounts for wound interactions.

Bei Fehler: If triage takes too long (das System is actively degrading), skip detailed classification and focus on: "What is the single most critical thing to stabilize?" Beheben that first, then return to full triage.

Schritt 2: Emergency Stabilization

Stoppen the damage from spreading vor beginning repair.

1. Enthalten the wound:
   • Isolate damaged components (circuit breakers, network segmentation, traffic rerouting)
   • Verhindern cascade: disable non-essential features that depend on damaged components
   • Preserve evidence: take snapshots, save logs, capture the current state vor any changes
2. Anwenden emergency patches:
   • These sind nicht permanent fixes — they're tourniquets
   • Acceptable emergency measures:
     • Redirect traffic to a healthy replica
     • Deaktivieren the damaged feature entirely
     • Anwenden a known-working configuration from backup
     • Scale up healthy components to absorb redirected load
   • Unacceptable emergency measures:
     • Modifying code ohne testing (creates new wounds)
     • Deleting data to "reset" das Problem (destroys recovery options)
     • Hiding the damage (disabling alerts, suppressing errors)
3. Verifizieren stabilization:
   • Is the damage still spreading? If yes, containment failed — try a broader isolation
   • Is das System functional (possibly degraded)? If yes, proceed to repair
   • Are emergency patches holding? If yes, you have time for deliberate repair

Erwartet: The system is stable (not actively degrading) even if degraded. Damage is contained and not spreading. Evidence is preserved for root cause analysis.

Bei Fehler: If stabilization fails (damage continues spreading despite containment), escalate to full system fallback: activate disaster recovery, switch to backup system, or gracefully degrade to minimal viable operation. Stabilization that takes too long becomes the disaster.

Schritt 3: Erstellen Repair Scaffolding

Construct the temporary structures that support the repair process.

1. Einrichten a repair environment:
   • Branch or copy the damaged system for repair work
   • Sicherstellen repair changes kann tested vor applying to production
   • Erstellen a rollback plan fuer jede repair step
2. Erstellen diagnostic infrastructure:
   • Enhanced monitoring on damaged areas (detect regression sofort)
   • Logging that captures the repair process (what was changed, when, why)
   • Comparison tools: vor-damage state vs. current vs. nach-repair
3. Entwerfen the repair sequence:
   • Fuer jede wound (in priority order from triage): a. Root cause identification (why did this break?) b. Repair approach (fix the cause, not just the symptom) c. Verification method (how to confirm the repair worked) d. Regression check (did the repair break anything else?)
4. Identifizieren scar tissue risk:
   • Repairs done under pressure often introduce scar tissue (workarounds, special cases, technical debt)
   • Planen for scar tissue management (Step 5) from the start

Erwartet: A repair environment with diagnostic capability, a sequenced repair plan, and awareness of scar tissue risk.

Bei Fehler: If setting up a proper repair environment is too slow (system urgency demands immediate production changes), apply changes directly but with extreme discipline: one change at a time, tested by the available means, rolled back if it doesn't help.

Schritt 4: Ausfuehren Progressive Rebuild

Repair damage systematically, verifying each fix vor proceeding.

1. Fuer jede wound (in triage priority order): a. Identifizieren root cause:
   • Is this a code bug? Configuration error? Data corruption? Dependency failure?
   • Is this a symptom of a deeper structural problem?
   • Would fixing the root cause also address other wounds? b. Implementieren the repair:
   • Beheben the root cause, not just the symptom
   • If the root cause can't be fixed sofort, implement a deliberate workaround and document it
   • Keep repairs minimal — fix what's broken, don't refactor the neighborhood c. Verifizieren the repair:
   • Does the specific damaged function work korrekt now?
   • Does the repair pass automated tests?
   • Is das System's overall health improved or mindestens unchanged? d. Pruefen auf regression:
   • Did this repair break anything else?
   • Are emergency patches from Step 2 still needed, or can some be removed?
2. After all critical and serious wounds are repaired:
   • Entfernen emergency patches that are no longer needed
   • Wiederherstellen disabled features
   • Zurueckgeben traffic to normal routing
3. Planen moderate and minor wound repairs:
   • These enter the normal development workflow
   • Verfolgen them to completion (don't let them become "accepted" damage)

Erwartet: Critical and serious wounds are repaired with verified fixes. Emergency patches are removed. The system is restored to functional operation.

Bei Fehler: If a repair attempt fails or causes regression, roll back to the previous state and reassess. If multiple repair attempts fail for the same wound, the damage kann too deep for local repair — consider whether the affected component needs full replacement anstatt repair (see dissolve-form).

Schritt 5: Verwalten Scar Tissue and Strengthen

Address the workarounds and shortcuts introduced waehrend emergency repair, and strengthen gegen recurrence.

1. Inventory scar tissue:
   • Emergency patches that became permanent
   • Workarounds that were never replaced with proper fixes
   • Special cases added to handle damage-related Grenzfaelle
   • Disabled features that were never re-enabled
2. Fuer jede piece of scar tissue, decide:
   • Remove: the workaround is no longer needed (damage is fully repaired)
   • Replace: the workaround addresses a real need but sollte implemented ordnungsgemaess
   • Accept: the workaround is the most practical long-term solution (rare, document why)
3. Strengthen gegen recurrence:
   • Root cause analysis: why did this damage occur?
   • Prevention: what would have prevented it? (monitoring, testing, architecture change)
   • Detection: how could we detect this faster next time? (alerts, health checks)
   • Recovery: how could we recover faster? (runbooks, backup procedures, automation)
4. Aktualisieren immune memory:
   • Hinzufuegen the incident pattern to monitoring and alerting (see defend-colony immune memory)
   • Aktualisieren runbooks with the repair procedure that worked
   • Teilen learnings across das Team/organization

Erwartet: Scar tissue is managed (removed, replaced, or accepted with documentation). The system ist nicht only repaired but more resilient than vor the damage. Learnings are captured for future incidents.

Bei Fehler: If scar tissue management is deprioritized ("it works, don't touch it"), schedule it explicitly. Unmanaged scar tissue accumulates and eventually contributes to the next incident. If the root cause can't be identified, strengthen detection and recovery speed as compensating controls.

Validierung

• [ ] All damage is inventoried and classified by severity
• [ ] Emergency stabilization stopped the spread of damage
• [ ] Evidence is preserved for root cause analysis
• [ ] Critical and serious wounds are repaired with verified fixes
• [ ] Emergency patches are removed nach proper repair
• [ ] Scar tissue is inventoried and managed (removed, replaced, or documented)
• [ ] Root cause analysis identifies prevention and detection improvements
• [ ] System resilience is improved verglichen mit pre-damage state

Haeufige Stolperfallen

• Repairing ohne stabilizing: Attempting to fix the root cause while das System is actively bleeding. Stabilisieren first, then repair. Tourniquets vor surgery
• Permanent emergency patches: Emergency measures that become the permanent solution create compounding technical debt. Always follow up with proper repair
• Root cause assumption: Assuming the root cause is known ohne investigation. Many "obvious" causes are symptoms of deeper issues. Investigate vor committing to a repair strategy
• Repair-induced damage: Rushing repairs ohne testing creates new wounds. One verified fix per iteration — never batch untested changes
• Ignoring scar tissue: "It works now" ist nicht the same as "it's healthy." Scar tissue from hasty repairs is the seed of the next incident

Verwandte Skills

• assess-form — damage assessment shares methodology with form assessment
• adapt-architecture — architectural adaptation kann needed if damage reveals structural weakness
• dissolve-form — for components too damaged to repair; dissolve and rebuild
• defend-colony — defense triggers repair; post-incident recovery feeds back into defense
• shift-camouflage — surface adaptation can mask damage while repair proceeds (with caution)
• conduct-post-mortem — structured post-incident analysis complements root cause identification
• write-incident-runbook — repair procedures sollte captured as runbooks for future incidents