lebsral/dspy-infer-rules

Extracting Decision Rules with dspy.InferRules

Guide the user through using dspy.InferRules to discover explicit, human-readable rules from labeled examples and inject them into program instructions.

What is dspy.InferRules

dspy.InferRules is a DSPy optimizer that analyzes your training examples and extracts natural-language rules describing the decision patterns it finds. These rules are then appended to the instructions of each predictor in your program. The result is a compiled program whose prompts contain explicit, interpretable decision logic -- not opaque few-shot examples.

It inherits from BootstrapFewShot, so it first bootstraps demonstrations and then goes further by inducing rules from those demonstrations.

Key properties:

• Extracts human-readable rules -- the discovered logic is plain English, not weights or embeddings
• Builds on BootstrapFewShot -- bootstraps demonstrations first, then induces rules from them
• Generates multiple candidates -- creates several rule-enhanced programs and picks the best one on a validation set
• Enhances instructions -- appends discovered rules directly to each predictor's signature instructions
• Gracefully handles context limits -- iteratively removes examples if they exceed the LM's context window

When to use InferRules

Use dspy.InferRules when:

• You have labeled examples and want to understand the patterns behind them
• Interpretability matters -- you need to explain decisions to stakeholders or auditors
• Your task has consistent, describable rules (classification, routing, moderation, triage)
• You want to improve a program's instructions without manually writing rules
• You need a compiled program that works without few-shot demonstrations at inference time

Do not use InferRules when:

• You have very few examples (fewer than ~20) -- rules need enough data to generalize
• The task has no consistent patterns (creative writing, open-ended generation)
• You want to tune few-shot examples only -- use dspy.BootstrapFewShot instead
• You want full prompt + demo optimization -- use dspy.MIPROv2 instead
• You need weight tuning -- use dspy.BootstrapFinetune

Basic usage

Three things are needed: a DSPy program, a metric function, and a training set.

import dspy

dspy.configure(lm=dspy.LM("openai/gpt-4o-mini"))  # or any LiteLLM-supported provider

# 1. Define a program
classify = dspy.ChainOfThought("text -> label")

# 2. Define a metric
def exact_match(example, pred, trace=None):
    return pred.label.strip().lower() == example.label.strip().lower()

# 3. Prepare training data
trainset = [
    dspy.Example(text="Server is down again", label="urgent").with_inputs("text"),
    dspy.Example(text="Update my billing info", label="normal").with_inputs("text"),
    dspy.Example(text="Site is completely broken", label="urgent").with_inputs("text"),
    dspy.Example(text="How do I change my password?", label="normal").with_inputs("text"),
    # ... more labeled examples
]

# 4. Compile with InferRules
optimizer = dspy.InferRules(metric=exact_match, num_rules=10)
compiled = optimizer.compile(classify, trainset=trainset)

# 5. Use the compiled program -- instructions now contain discovered rules
result = compiled(text="Database connection pool exhausted")
print(result.label)

After compilation, inspect the rules that were injected:

# View the enhanced instructions for each predictor
for name, predictor in compiled.named_predictors():
    print(f"Predictor: {name}")
    print(f"Instructions: {predictor.signature.instructions}")
    print()

How InferRules extracts rules

The compilation process has five stages:

1. Data splitting -- Splits trainset 50/50 into training and validation sets (unless you provide valset separately)
2. Bootstrap demonstrations -- Runs the parent BootstrapFewShot.compile() to collect successful input-output demonstrations
3. Rule induction -- For each predictor, feeds the bootstrapped demonstrations into a RulesInductionProgram that generates natural-language rules describing the patterns
4. Candidate generation -- Repeats the rule induction num_candidates times with different samples to produce diverse rule sets
5. Validation and selection -- Scores each candidate program on the validation set using your metric and returns the highest-scoring one

The induced rules look like plain English statements, for example:

"If the text mentions system failures, outages, or data loss, classify as urgent." "If the text is a routine account or billing question, classify as normal."

These rules are appended to the predictor's existing instructions, giving the LM explicit decision logic to follow.

Constructor parameters

dspy.InferRules(
    num_candidates=10,   # Number of candidate programs to evaluate
    num_rules=10,        # Number of rules to induce per predictor
    num_threads=None,     # Thread count for parallel evaluation
    teacher_settings=None,  # Config for the teacher model
    metric=...,          # Evaluation metric (required, via kwargs)
    max_errors=...,      # Max allowed errors during evaluation (optional, via kwargs)
)

| Parameter | Type | Default | Description | |-----------|------|---------|-------------| | num_candidates | int | 10 | Number of candidate rule-enhanced programs to generate. More candidates increase the chance of finding better rules but cost more LM calls | | num_rules | int | 10 | Number of rules to induce per predictor. More rules capture finer patterns but risk overfitting or exceeding context limits | | num_threads | int | None | Number of threads for parallel evaluation. None uses the default | | teacher_settings | dict | None | Configuration for the teacher model used during bootstrapping | | metric | Callable | -- | Evaluation function (example, prediction, trace) -> float. Passed via kwargs | | max_errors | int | -- | Maximum errors allowed before stopping evaluation. Passed via kwargs |

The compile method

compiled_program = optimizer.compile(
    student,              # Your DSPy program to optimize (required)
    trainset=trainset,    # Training examples (required)
    valset=None,          # Validation examples (optional -- auto-split if not provided)
)

If valset is not provided, compile automatically splits trainset 50/50 into training and validation sets. Providing your own valset gives you more control over evaluation.

Interpretability benefits

InferRules stands apart from other optimizers because its output is human-readable:

| Optimizer | Output | Interpretable? | |-----------|--------|----------------| | BootstrapFewShot | Few-shot examples in the prompt | Somewhat -- you can read the examples | | MIPROv2 | Optimized instructions + few-shot | Partially -- instructions are readable but auto-generated | | BootstrapFinetune | Updated model weights | No -- weights are opaque | | InferRules | Explicit natural-language rules | Yes -- you can read, audit, and edit the rules |

This makes InferRules a good fit for:

• Regulated industries where you must explain how decisions are made
• Debugging -- read the rules to understand what the optimizer learned
• Human-in-the-loop refinement -- edit or remove rules that are wrong before deploying
• Documentation -- the rules serve as a specification of your system's behavior

Tuning num_candidates and num_rules

num_candidates controls how many different rule sets are generated and compared:

| Value | Use case | |-------|----------| | 3-5 | Quick iteration, prototyping | | 10 (default) | Good balance of quality and cost | | 15-20 | High-stakes applications, when you need the best possible rules |

num_rules controls how many rules are induced per predictor:

| Value | Use case | |-------|----------| | 3-5 | Simple binary tasks (spam/not-spam) | | 10 (default) | Multi-class tasks, moderate complexity | | 15-20 | Tasks with many edge cases or subtle distinctions |

More rules is not always better. Too many rules can overwhelm the LM's context or introduce contradictions. Start with the defaults and adjust based on validation scores.

Providing a separate validation set

For more control, provide your own validation set:

optimizer = dspy.InferRules(metric=exact_match, num_rules=10, num_candidates=10)
compiled = optimizer.compile(
    classify,
    trainset=train_examples,
    valset=val_examples,
)

This is recommended when:

• Your dataset has a natural train/val split
• You want to ensure specific edge cases appear in validation
• You want a larger training set for rule induction (the 50/50 auto-split may leave too few training examples)

Saving and loading compiled programs

# Save the compiled program (includes the discovered rules in instructions)
compiled.save("compiled_with_rules.json")

# Load it later
from your_module import YourProgram
loaded = YourProgram()
loaded.load("compiled_with_rules.json")

# The loaded program has the same enhanced instructions
result = loaded(text="New input here")

Gotchas

1. Claude skips inspecting the discovered rules. After optimizer.compile(), always print the enhanced instructions with predictor.signature.instructions. InferRules can generate incorrect or contradictory rules. Read them, edit or remove bad ones before deploying.
2. Claude sets num_rules too high. More rules is not always better. Too many rules overwhelm the LM's context window or introduce contradictions. Start with 10 (the default) and only increase if validation scores improve. Reduce if you see contradictory behavior.
3. Claude does not compare against BootstrapFewShot. InferRules adds complexity. Research shows it sometimes matches or underperforms the baseline — a 2025 study found InferRules achieved the same 87% accuracy as the unoptimized prompt on a code generation task. Always compare against plain BootstrapFewShot; if few-shot examples alone match InferRules accuracy, use the simpler approach.
4. Claude forgets that InferRules splits trainset 50/50 automatically. If you have 40 examples and do not pass valset, InferRules uses only 20 for training — often too few for good rule induction. Pass valset explicitly to control the split.
5. Claude uses InferRules for open-ended or creative tasks. InferRules works best on tasks with consistent, describable patterns (classification, routing, triage). For creative writing or open-ended generation where there are no consistent rules to discover, it adds noise. Use BootstrapFewShot or MIPROv2 instead.

Cross-references

Install any skill: npx skills add lebsral/DSPy-Programming-not-prompting-LMs-skills --skill <name>

• Bootstrapping few-shot examples as the foundation -- see /ai-improving-accuracy
• Full prompt optimization with MIPROv2 -- see /ai-improving-accuracy
• Evaluating your program to measure rule quality -- see /dspy-evaluate
• Data preparation for training and validation sets -- see /dspy-data
• Signatures and instructions that InferRules modifies -- see /dspy-signatures
• For worked examples, see examples.md
• Install /ai-do if you do not have it — it routes any AI problem to the right skill and is the fastest way to work: npx skills add lebsral/DSPy-Programming-not-prompting-LMs-skills --skill ai-do

Additional resources

• dspy.InferRules API docs
• DSPy optimizer selection guide
• For constructor signatures and method reference, see reference.md
• For worked examples (ticket classification, content moderation), see examples.md