ADu2021/darwin-godel-evolution

Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agents

Core Concept

The Darwin Godel Machine enables autonomous, continuous self-improvement in AI agents without requiring human-designed fixed architectures. The approach combines evolutionary principles with language models to create agents that can modify their own code, test improvements empirically, and maintain an archive of increasingly capable variants.

Rather than static agent design, this framework treats agent improvement as an ongoing evolutionary process where LLMs generate code mutations, benchmark systems evaluate them against task metrics, and successful variants populate an evolving archive. The system automatically discovers beneficial modifications (tool improvements, context management strategies) without explicit programming.

Architecture Overview

• Evolutionary Archive: Maintain a population of agent code variants with tracked performance across benchmarks
• Mutation Generation: Use LLMs to propose code modifications by sampling from the archive and generating variants
• Empirical Evaluation: Test each candidate against established benchmarks (e.g., SWE-bench, Polyglot)
• Selection Pressure: Retain high-performing variants and seed future mutations from the archive
• Feedback Loop: Iteratively improve agent capabilities through code-level evolution
• Foundation Model Integration: Leverage LLM reasoning to propose semantically meaningful changes

Implementation

The following steps outline how to implement a self-improving agent system using evolutionary principles:

1. Initialize agent archive - Create baseline agent implementations and store them with their benchmark scores
2. Sample candidates for mutation - Select agents from the archive, biased toward high-performing variants
3. Generate code mutations - Use an LLM to propose modifications (new tools, strategy changes, parameter adjustments)
4. Implement and test - Apply mutations to create new agent variants and evaluate them on benchmarks
5. Update archive - Add successful variants to the population; discard or deprioritize failing mutations
6. Repeat cycle - Continue mutation and evaluation until performance plateaus or convergence criteria are met

import json
from typing import Dict, List, Tuple
from dataclasses import dataclass
from anthropic import Anthropic

@dataclass
class AgentVariant:
    code: str
    benchmark_scores: Dict[str, float]
    generation: int

class DarwinMachine:
    def __init__(self, client: Anthropic, benchmarks: List[str]):
        self.client = client
        self.benchmarks = benchmarks
        self.archive: List[AgentVariant] = []
        self.generation = 0

    def add_baseline(self, code: str, scores: Dict[str, float]):
        """Add initial agent to archive."""
        variant = AgentVariant(code=code, benchmark_scores=scores, generation=0)
        self.archive.append(variant)

    def select_parent(self) -> AgentVariant:
        """Select agent from archive, biased toward high performance."""
        scores = [sum(v.benchmark_scores.values()) for v in self.archive]
        total = sum(scores)
        weights = [s / total for s in scores]
        import random
        return random.choices(self.archive, weights=weights, k=1)[0]

    def mutate_code(self, parent_code: str) -> str:
        """Generate code mutations using Claude."""
        message = self.client.messages.create(
            model="claude-opus-4.6",
            max_tokens=2000,
            messages=[{
                "role": "user",
                "content": f"""You are evolving an AI agent. Here's the current agent code:

{parent_code}

Propose ONE specific, meaningful improvement to this agent. Consider:
- Adding new tools or improving existing ones
- Better context management strategies
- Improved error handling or recovery
- More efficient reasoning patterns

Return ONLY the modified code, no explanations."""
            }]
        )
        return message.content[0].text

    def evaluate_variant(self, code: str) -> Dict[str, float]:
        """Simulate benchmark evaluation."""
        # In practice, this would execute the agent and measure performance
        # For this example, return random scores
        import random
        return {bench: random.uniform(0, 100) for bench in self.benchmarks}

    def evolve(self, generations: int = 5):
        """Run evolutionary loop."""
        for gen in range(generations):
            self.generation = gen
            parent = self.select_parent()
            mutated_code = self.mutate_code(parent.code)
            scores = self.evaluate_variant(mutated_code)

            variant = AgentVariant(code=mutated_code, benchmark_scores=scores,
                                  generation=gen)
            self.archive.append(variant)

            best = max(self.archive, key=lambda v: sum(v.benchmark_scores.values()))
            print(f"Gen {gen}: New variant added. Best score: {sum(best.benchmark_scores.values()):.1f}")

Practical Guidance

Hyperparameters to tune:

• Archive size (10-100 variants): Larger archives provide more diversity but slower evolution; smaller archives converge faster but may miss promising paths
• Mutation selection bias (exponential, linear): Control how much selective pressure is applied to favor high performers
• Benchmark evaluation frequency: Evaluate every mutation (thorough) vs. sampling (faster but riskier)
• Timeout per agent (seconds): Prevent infinite loops in generated code; balance between comprehensive testing and efficiency

When to use:

• Optimizing agent architectures when design space is large and unclear
• Discovering novel tool combinations or strategy improvements
• Long-horizon improvement tasks where humans can't easily design optimal solutions
• Research on open-ended learning and agent capabilities

When NOT to use:

• Time-critical applications where evolution overhead is unaffordable
• Domains with small benchmark suites (overfitting to specific tasks likely)
• Safety-critical systems where mutation could introduce unpredictable failures
• Tasks with well-established optimal solutions already known

Common pitfalls:

• Poor benchmark diversity: Using only SWE-bench can lead to agents optimized narrowly for that task
• Evaluation noise: Stochastic benchmarks require multiple runs per variant to avoid false positives
• Code bloat: Mutations accumulate complexity without refactoring; periodic code simplification is essential
• Archive staleness: Old high-performers can dominate selection; periodically restart from diverse sources

Reference

The paper demonstrates substantial performance gains on software engineering tasks: SWE-bench improved from 20.0% to 50.0%, Polyglot from 14.2% to 30.7%. Notably, the system discovered beneficial modifications (enhanced code editing, better context management) without explicit design.

Original paper: "Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agents" (arxiv.org/abs/2505.22954)