bolivian-peru/generation-timeline

Generation Timeline — Time-Travel Debugging

Correlate NixOS generations + audit ledger + journal logs to debug issues across time.

"What changed?" Workflow

When a user reports "something broke":

1. Get the generation timeline

shell_exec({ command: "nixos-rebuild list-generations 2>/dev/null | tail -10" })

2. Get the audit ledger for the same period

event_log({ limit: 30 })

3. Correlate with journal errors

journal_logs({ priority: "err", since: "24 hours ago", lines: 50 })

4. Check memory for related context

memory_recall({ query: "config change rebuild generation", timeframe: "7d" })

5. Build the timeline

Present a unified view:

Timeline — Last 48 Hours:

Feb 19 09:30  Gen 48 applied (nixpkgs update: OpenSSL 3.1→3.2)
Feb 19 09:47  ERROR: app returned first 502 (17 min after gen 48)
Feb 19 09:48  WARNING: 12 more 502s in next minute
Feb 19 09:52  agentd detected service degradation (audit #134)
Feb 19 10:00  You: "something broke"

Root cause: Gen 48 updated OpenSSL, breaking TLS cipher
used by your app.

Options:
(a) Rollback to Gen 47 (reverts everything since)
(b) Pin OpenSSL 3.1 for your app only (overlay)
(c) Update app TLS config (best long-term fix)

"Explain my server" Workflow

When asked "what does this server do" or "explain the setup":

shell_exec({ command: "cat /etc/nixos/configuration.nix" })
shell_exec({ command: "nixos-rebuild list-generations | head -5" })
shell_exec({ command: "systemctl list-units --type=service --state=running --no-pager" })

Generate a plain-English overview:

Your Server — osModa (Hetzner CX22)

Services running:
  - agentd (osModa system daemon) — since boot
  - OpenClaw Gateway (AI agent interface) — port 18789
  - SSH (key-auth only) — port 22
  - nginx (reverse proxy) — ports 80, 443

Architecture:
  Internet → nginx (443) → OpenClaw (18789)
  SSH → direct access (22)

NixOS: Generation 50, built Feb 20
Resources: 2 CPU, 7.6GB RAM, 80GB disk (38% used)
Uptime: 12 hours

History: 50 generations over 18 days.
Key changes stored in audit ledger (134 entries).

"Fork my server" Workflow

When asked to create a staging copy or export the config:

shell_exec({ command: "cat /etc/nixos/configuration.nix" })
shell_exec({ command: "cat /etc/nixos/hardware-configuration.nix" })

Generate a self-contained flake that reproduces this machine:

• Include all service configs
• EXCLUDE secrets, API keys, database contents
• Include a README explaining how to deploy

Critical Rule

The audit ledger is the source of truth for "why" something was done. NixOS generations are the source of truth for "what" the system state is. Journal logs are the source of truth for "when" things happened.

Always correlate all three when debugging.