fox3000wang/autonomous-agent-harness

Autonomous Agent Harness

Turn Claude Code into a persistent, self-directing agent system using only native features and MCP servers.

When to Activate

• User wants an agent that runs continuously or on a schedule
• Setting up automated workflows that trigger periodically
• Building a personal AI assistant that remembers context across sessions
• User says "run this every day", "check on this regularly", "keep monitoring"
• Wants to replicate functionality from Hermes, AutoGPT, or similar autonomous agent frameworks
• Needs computer use combined with scheduled execution

Architecture

┌──────────────────────────────────────────────────────────────┐
│                    Claude Code Runtime                        │
│                                                              │
│  ┌──────────┐  ┌──────────┐  ┌──────────┐  ┌─────────────┐ │
│  │  Crons   │  │ Dispatch │  │ Memory   │  │ Computer    │ │
│  │ Schedule │  │ Remote   │  │ Store    │  │ Use         │ │
│  │ Tasks    │  │ Agents   │  │          │  │             │ │
│  └────┬─────┘  └────┬─────┘  └────┬─────┘  └──────┬──────┘ │
│       │              │             │                │        │
│       ▼              ▼             ▼                ▼        │
│  ┌──────────────────────────────────────────────────────┐    │
│  │              ECC Skill + Agent Layer                  │    │
│  │                                                      │    │
│  │  skills/     agents/     commands/     hooks/        │    │
│  └──────────────────────────────────────────────────────┘    │
│       │              │             │                │        │
│       ▼              ▼             ▼                ▼        │
│  ┌──────────────────────────────────────────────────────┐    │
│  │              MCP Server Layer                        │    │
│  │                                                      │    │
│  │  memory    github    exa    supabase    browser-use  │    │
│  └──────────────────────────────────────────────────────┘    │
└──────────────────────────────────────────────────────────────┘

Core Components

1. Persistent Memory

Use Claude Code's built-in memory system enhanced with MCP memory server for structured data.

Built-in memory (~/.claude/projects/*/memory/):

• User preferences, feedback, project context
• Stored as markdown files with frontmatter
• Automatically loaded at session start

MCP memory server (structured knowledge graph):

• Entities, relations, observations
• Queryable graph structure
• Cross-session persistence

Memory patterns:

# Short-term: current session context
Use TodoWrite for in-session task tracking

# Medium-term: project memory files
Write to ~/.claude/projects/*/memory/ for cross-session recall

# Long-term: MCP knowledge graph
Use mcp__memory__create_entities for permanent structured data
Use mcp__memory__create_relations for relationship mapping
Use mcp__memory__add_observations for new facts about known entities

2. Scheduled Operations (Crons)

Use Claude Code's scheduled tasks to create recurring agent operations.

Setting up a cron:

# Via MCP tool
mcp__scheduled-tasks__create_scheduled_task({
  name: "daily-pr-review",
  schedule: "0 9 * * 1-5",  # 9 AM weekdays
  prompt: "Review all open PRs in affaan-m/everything-claude-code. For each: check CI status, review changes, flag issues. Post summary to memory.",
  project_dir: "/path/to/repo"
})

# Via claude -p (programmatic mode)
echo "Review open PRs and summarize" | claude -p --project /path/to/repo

Useful cron patterns:

| Pattern | Schedule | Use Case | |---------|----------|----------| | Daily standup | 0 9 * * 1-5 | Review PRs, issues, deploy status | | Weekly review | 0 10 * * 1 | Code quality metrics, test coverage | | Hourly monitor | 0 * * * * | Production health, error rate checks | | Nightly build | 0 2 * * * | Run full test suite, security scan | | Pre-meeting | */30 * * * * | Prepare context for upcoming meetings |

3. Dispatch / Remote Agents

Trigger Claude Code agents remotely for event-driven workflows.

Dispatch patterns:

# Trigger from CI/CD
curl -X POST "https://api.anthropic.com/dispatch" \
  -H "Authorization: Bearer $ANTHROPIC_API_KEY" \
  -d '{"prompt": "Build failed on main. Diagnose and fix.", "project": "/repo"}'

# Trigger from webhook
# GitHub webhook → dispatch → Claude agent → fix → PR

# Trigger from another agent
claude -p "Analyze the output of the security scan and create issues for findings"

4. Computer Use

Leverage Claude's computer-use MCP for physical world interaction.

Capabilities:

• Browser automation (navigate, click, fill forms, screenshot)
• Desktop control (open apps, type, mouse control)
• File system operations beyond CLI

Use cases within the harness:

• Automated testing of web UIs
• Form filling and data entry
• Screenshot-based monitoring
• Multi-app workflows

5. Task Queue

Manage a persistent queue of tasks that survive session boundaries.

Implementation:

# Task persistence via memory
Write task queue to ~/.claude/projects/*/memory/task-queue.md

# Task format
---
name: task-queue
type: project
description: Persistent task queue for autonomous operation
---

## Active Tasks
- [ ] PR #123: Review and approve if CI green
- [ ] Monitor deploy: check /health every 30 min for 2 hours
- [ ] Research: Find 5 leads in AI tooling space

## Completed
- [x] Daily standup: reviewed 3 PRs, 2 issues

Replacing Hermes

| Hermes Component | ECC Equivalent | How | |------------------|---------------|-----| | Gateway/Router | Claude Code dispatch + crons | Scheduled tasks trigger agent sessions | | Memory System | Claude memory + MCP memory server | Built-in persistence + knowledge graph | | Tool Registry | MCP servers | Dynamically loaded tool providers | | Orchestration | ECC skills + agents | Skill definitions direct agent behavior | | Computer Use | computer-use MCP | Native browser and desktop control | | Context Manager | Session management + memory | ECC 2.0 session lifecycle | | Task Queue | Memory-persisted task list | TodoWrite + memory files |

Setup Guide

Step 1: Configure MCP Servers

Ensure these are in ~/.claude.json:

{
  "mcpServers": {
    "memory": {
      "command": "npx",
      "args": ["-y", "@anthropic/memory-mcp-server"]
    },
    "scheduled-tasks": {
      "command": "npx",
      "args": ["-y", "@anthropic/scheduled-tasks-mcp-server"]
    },
    "computer-use": {
      "command": "npx",
      "args": ["-y", "@anthropic/computer-use-mcp-server"]
    }
  }
}

Step 2: Create Base Crons

# Daily morning briefing
claude -p "Create a scheduled task: every weekday at 9am, review my GitHub notifications, open PRs, and calendar. Write a morning briefing to memory."

# Continuous learning
claude -p "Create a scheduled task: every Sunday at 8pm, extract patterns from this week's sessions and update the learned skills."

Step 3: Initialize Memory Graph

# Bootstrap your identity and context
claude -p "Create memory entities for: me (user profile), my projects, my key contacts. Add observations about current priorities."

Step 4: Enable Computer Use (Optional)

Grant computer-use MCP the necessary permissions for browser and desktop control.

Example Workflows

Autonomous PR Reviewer

Cron: every 30 min during work hours
1. Check for new PRs on watched repos
2. For each new PR:
   - Pull branch locally
   - Run tests
   - Review changes with code-reviewer agent
   - Post review comments via GitHub MCP
3. Update memory with review status

Personal Research Agent

Cron: daily at 6 AM
1. Check saved search queries in memory
2. Run Exa searches for each query
3. Summarize new findings
4. Compare against yesterday's results
5. Write digest to memory
6. Flag high-priority items for morning review

Meeting Prep Agent

Trigger: 30 min before each calendar event
1. Read calendar event details
2. Search memory for context on attendees
3. Pull recent email/Slack threads with attendees
4. Prepare talking points and agenda suggestions
5. Write prep doc to memory

Constraints

• Cron tasks run in isolated sessions — they don't share context with interactive sessions unless through memory.
• Computer use requires explicit permission grants. Don't assume access.
• Remote dispatch may have rate limits. Design crons with appropriate intervals.
• Memory files should be kept concise. Archive old data rather than letting files grow unbounded.
• Always verify that scheduled tasks completed successfully. Add error handling to cron prompts.