lebsral/ai-choosing-architecture

Choose the Right DSPy Architecture

When NOT to use this skill

• Already know what module to use — go to the matching /dspy-* skill
• Fixing errors in existing code — use /ai-fixing-errors
• Learning a specific module — use the matching /dspy-* skill
• Need a project plan, not an architecture decision — use /ai-planning

---

Step 1: Answer 3 questions

Before recommending anything, get answers to these three questions from the user (or infer them from context):

1. What goes in and what comes out? Input type and format, output type and format.
2. Does the AI need external tools? Search, APIs, databases, calculators, code execution?
3. How complex is the reasoning? Simple mapping, moderate analysis, complex multi-step logic?

---

Step 2: Pick the module

Walk the decision tree:

Does it need tools?
├── Yes: Does it need to write and run code?
│   ├── Yes → CodeAct
│   └── No → ReAct
└── No: How complex is the reasoning?
    ├── Simple (direct mapping) → Predict
    ├── Moderate (needs explanation) → ChainOfThought
    ├── Complex (math/computation) → ProgramOfThought
    └── Very complex (compare approaches) → MultiChainComparison

Module tradeoff summary:

| Module | Accuracy | Latency | Cost | Best for | |---|---|---|---|---| | Predict | Baseline | 1x | 1x | Simple classification, extraction, formatting | | ChainOfThought | +10-30% | 1.5-2x | 1.5-2x | Most tasks — default choice when unsure | | ProgramOfThought | +20-40% on math | 2-3x | 2-3x | Math, computation, data manipulation | | ReAct | Varies | 3-10x | 3-10x | Tasks requiring external information or actions | | CodeAct | Varies | 3-10x | 3-10x | Tasks requiring code generation and execution | | MultiChainComparison | +5-15% | 3-5x | 3-5x | When you need the best possible single answer | | BestOfN | +5-10% | Nx | Nx | When you have a reward function and acceptance threshold |

For the full module list including Refine, RLM, and Parallel, see reference.md.

---

Step 3: Single module vs pipeline

Use this table to decide whether one module is enough or a pipeline is warranted:

| Signal | Single module | Pipeline | |---|---|---| | Input maps directly to output | Yes | -- | | Task has distinct phases (classify then generate) | -- | Yes | | Different parts need different LM capabilities | -- | Yes | | Need to validate intermediate results | -- | Yes | | Simple input-output with clear signature | Yes | -- | | Need to combine retrieval + generation | -- | Yes |

Rule of thumb: start with a single module. Add pipeline stages only when you have measured a quality gap that a single module cannot close.

Verification: After implementing the chosen architecture, run dspy.Evaluate(devset, metric=your_metric) on 20-50 examples to confirm the module choice was correct before optimizing.

---

Step 4: Architecture-to-optimizer pairing

| Architecture | First optimizer | Best optimizer | Why | |---|---|---|---| | Single Predict | BootstrapFewShot | MIPROv2 | Simple, fast to optimize | | Single ChainOfThought | BootstrapFewShot | MIPROv2 | Reasoning benefits from good demos | | ReAct agent | BootstrapFewShot | BootstrapFewShot | Agents are hard to optimize, start simple | | Multi-module pipeline | BootstrapFewShot | MIPROv2 | End-to-end optimization tunes all stages | | Pipeline with fine-tuning | BootstrapFinetune | BetterTogether | Weight tuning for max quality |

---

Step 5: Generate the recommendation

Output the recommendation in this format:

## Architecture Recommendation

**Module:** dspy.ChainOfThought (or whatever was chosen)
**Why:** [1-2 sentences tying the module to the task]
**Skeleton:**
[minimal code showing the module or pipeline structure]

**Optimizer path:**
1. Start with BootstrapFewShot (quick baseline)
2. Move to MIPROv2 if accuracy needs to improve

**Alternative considered:** [what else was considered and why it was not chosen]

---

Skeleton code templates

1. Single Predict (simplest)

import dspy

class MyTask(dspy.Signature):
    """One sentence describing the task."""
    input_text: str = dspy.InputField()
    output_label: str = dspy.OutputField()

predictor = dspy.Predict(MyTask)
result = predictor(input_text="...")
print(result.output_label)

2. Single ChainOfThought (default choice)

import dspy

class MyTask(dspy.Signature):
    """One sentence describing the task."""
    question: str = dspy.InputField()
    answer: str = dspy.OutputField()

cot = dspy.ChainOfThought(MyTask)
result = cot(question="...")
print(result.answer)

3. ReAct with tools

import dspy

def search(query: str) -> str:
    """Search external knowledge base."""
    ...

def lookup(term: str) -> str:
    """Look up a term in a database."""
    ...

class MyAgentTask(dspy.Signature):
    """Answer questions using search and lookup tools."""
    question: str = dspy.InputField()
    answer: str = dspy.OutputField()

agent = dspy.ReAct(MyAgentTask, tools=[search, lookup])
result = agent(question="...")
print(result.answer)

4. Two-stage pipeline (classify then generate)

import dspy

class Classify(dspy.Signature):
    """Classify the input into a category."""
    text: str = dspy.InputField()
    category: str = dspy.OutputField()

class Generate(dspy.Signature):
    """Generate a response given the category and original text."""
    text: str = dspy.InputField()
    category: str = dspy.InputField()
    response: str = dspy.OutputField()

class ClassifyThenGenerate(dspy.Module):
    def __init__(self):
        self.classify = dspy.Predict(Classify)
        self.generate = dspy.ChainOfThought(Generate)

    def forward(self, text: str) -> dspy.Prediction:
        category = self.classify(text=text).category
        response = self.generate(text=text, category=category).response
        return dspy.Prediction(category=category, response=response)

5. Three-stage RAG pipeline (retrieve, reason, generate)

import dspy

retriever = dspy.Retrieve(k=3)

class Reason(dspy.Signature):
    """Given context passages, identify the key facts relevant to the question."""
    question: str = dspy.InputField()
    context: list[str] = dspy.InputField()
    key_facts: str = dspy.OutputField()

class Answer(dspy.Signature):
    """Answer the question using the identified key facts."""
    question: str = dspy.InputField()
    key_facts: str = dspy.InputField()
    answer: str = dspy.OutputField()

class RAGPipeline(dspy.Module):
    def __init__(self):
        self.retrieve = retriever
        self.reason = dspy.ChainOfThought(Reason)
        self.answer = dspy.ChainOfThought(Answer)

    def forward(self, question: str) -> dspy.Prediction:
        passages = self.retrieve(question).passages
        key_facts = self.reason(question=question, context=passages).key_facts
        answer = self.answer(question=question, key_facts=key_facts).answer
        return dspy.Prediction(answer=answer, passages=passages)

---

Gotchas

1. Defaulting to ChainOfThought for everything. Predict is better for simple classification or extraction where reasoning adds noise, not signal. If the correct output is a fixed label from a known set, CoT can hallucinate reasoning that leads it astray.

2. Using ReAct when a pipeline suffices. ReAct is for tasks that need dynamic tool selection at runtime. If you know the steps upfront (e.g., always retrieve then answer), use a pipeline — it is cheaper, faster, and easier to optimize.

3. Over-engineering with MultiChainComparison. MCC runs 3-5x the cost of a single pass. Only reach for it after measuring that single-pass accuracy is insufficient for your use case.

4. Building a pipeline before proving a single module works. Always start with the simplest module that could work. Measure it on your eval set. Add pipeline stages only when you have a specific, measured quality gap.

5. Ignoring cost implications early. A ReAct agent with 10 tool calls costs roughly 10x a single Predict call. Factor cost and latency into architecture decisions before you build, not after.

---

Cross-references

Install any skill: npx skills add lebsral/DSPy-Programming-not-prompting-LMs-skills --skill <name>

• For full module comparison tables and complete code templates, see reference.md
• For worked architecture decisions with real examples, see examples.md
• Ready to build? Use the matching /dspy-* skill for your chosen module
• Need to implement a pipeline? Use /ai-building-pipelines
• Want to plan the full project? Use /ai-planning
• Need to review existing code? Use /ai-auditing-code
• Install /ai-do if you do not have it — it routes any AI problem to the right skill and is the fastest way to work: npx skills add lebsral/DSPy-Programming-not-prompting-LMs-skills --skill ai-do