gtapps/smoke-test

Smoke Test

Validate the post-hatch setup. Produces a structured report with PASS/WARN/FAIL per check.

Plan

1. Initialize counters

Track passed, warnings, failures counts. Collect output lines for the final report.

2. Config validation

• Read .claude-code-hermit/config.json
  • If missing or invalid JSON: FAIL config.json missing or invalid
  • If valid: PASS config.json parsed
• Read ${CLAUDE_PLUGIN_ROOT}/.claude-plugin/plugin.json to get the current plugin version
• Compare config._hermit_versions["claude-code-hermit"] with the plugin version
  • Match: PASS version matches plugin (X.Y.Z)
  • Mismatch: WARN config version X.Y.Z != plugin X.Y.Z — run /claude-code-hermit:hermit-evolve
• Check required top-level keys exist: agent_name, channels, env, heartbeat
  • Each missing key: FAIL missing required key: <key>

3. OPERATOR.md sanity

• Check if .claude-code-hermit/OPERATOR.md exists
  • Missing: WARN OPERATOR.md missing — run hatch or create manually
  • Empty: WARN OPERATOR.md is empty — fill in project context
• If exists and non-empty:
  • Check for {{ placeholder patterns (NOT <!-- which are legitimate HTML comments)
    • Found: WARN OPERATOR.md contains unfilled {{ placeholders
    • Not found: PASS OPERATOR.md readable

4. Scheduled check references

• Read scheduled_checks array from config.json
• For each entry:
  • Extract the plugin name and skill name from the skill field (format: /plugin:skill-name or plugin:skill-name)
  • Check if the skill directory exists at ${CLAUDE_PLUGIN_ROOT}/../<plugin>/skills/<skill-name>/SKILL.md
  • Exists: PASS scheduled_checks[N]: <id> resolves
  • Missing: WARN scheduled_checks[N]: skill not found at <path> — may be installed globally (sibling directory layout is not guaranteed for marketplace installs)

5. Routine validation (static)

• Read routines array from config.json
• For each routine:
  • Check required keys (id, schedule, skill, enabled): FAIL per missing key
  • If time or days keys are present: FAIL routine "<id>" uses legacy "time"/"days" — migrate to "schedule" (5-field cron)
  • Validate schedule is a 5-field cron expression (space-separated, 5 fields: min hour dom month dow): FAIL if malformed
  • Validate skill contains : (plugin:skill-name format): FAIL with expected format
  • PASS routine "<id>" valid (<schedule>, <skill>)
• Check for duplicate routine IDs: WARN per duplicate

6. State file validation

Template defaults: ${CLAUDE_PLUGIN_ROOT}/state-templates/ — alert-state.json.template, micro-proposals.json.template.

For each file: parse JSON. If missing or unparseable, rewrite from the corresponding template. If parseable but wrong shape, repair in place — backfill missing keys from template defaults, overwrite keys that exist with the wrong type, preserve all other existing data. Emit one WARN per file that needed repair (noting what was fixed), or PASS if valid.

• .claude-code-hermit/state/alert-state.json — object with: alerts (object), self_eval (object), total_ticks (number), last_digest_date (any, presence required)

  • For each missing or wrong-type key: overwrite with template default (e.g. alerts: {}, self_eval: {}, total_ticks: 0, last_digest_date: null) → WARN alert-state.json repaired: <keys>

• .claude-code-hermit/state/micro-proposals.json — object with pending key (array)

  • Missing pending key (and active key present): migrate — read active value; if non-null, set pending: [<active value>]; if null, set pending: []; remove active key → WARN micro-proposals.json migrated: active → pending
  • Missing pending key (and no active key): add "pending": [] → WARN micro-proposals.json missing pending key — backfilled
  • pending is not an array: overwrite with [] → WARN micro-proposals.json pending is not an array — reset

7. Channel test (optional)

• Check if any channels are configured and enabled in config.json
• If no channels: skip silently
• If channels configured:
  • For each enabled channel, attempt to send a test message via the channel's MCP tool:
    • Discord: use the Discord MCP send tool
    • Telegram: use the Telegram MCP send tool
  • Message content: "Smoke test — channel is working." (include agent name if configured)
  • Success: PASS <channel> test message sent
  • Failure: FAIL <channel> test message failed: <error>
  • If the MCP tool is not available (plugin not installed): WARN <channel> plugin not available for test

8. Print report

Output each check result as exactly one line:

PASS  config.json parsed and version matches plugin (0.3.4)
WARN  OPERATOR.md contains unfilled {{ placeholders
WARN  scheduled_checks[0]: skill not found at ../claude-code-setup/skills/claude-automation-recommender — may be installed globally
PASS  routine "heartbeat-restart" valid (0 4 * * *, claude-code-hermit:heartbeat start)
PASS  routine "morning" valid (0 9 * * *, claude-code-hermit:reflect)
FAIL  routine "bad" uses legacy "time" field — migrate to "schedule" (5-field cron)

Summary line at end:

Smoke test: 4 passed, 1 warning, 1 failed

For each FAIL, include a remediation hint on the same line or the line after.