lebsral/dspy-labeled-few-shot

Hand-Picked Demonstrations with dspy.LabeledFewShot

Guide the user through using dspy.LabeledFewShot -- the simplest DSPy optimizer. It takes labeled examples you provide and attaches them as few-shot demonstrations to your program's predictors. No bootstrapping, no metric, no LM calls during optimization.

What is LabeledFewShot

dspy.LabeledFewShot is an optimizer that takes a set of labeled training examples and injects them directly as few-shot demonstrations into every predictor in your DSPy program.

• No metric required -- unlike other optimizers, it does not evaluate or filter examples
• No LM calls during compilation -- it just copies your examples into the prompt
• Deterministic -- uses a fixed random seed (0) for reproducible example selection
• Fast -- compilation is instant because there is no search or bootstrapping step

Under the hood, compile() creates a copy of your program, iterates over each predictor, and assigns up to k examples from your training set as that predictor's demos.

When to use LabeledFewShot

| Use LabeledFewShot when... | Use something else when... | |---|---| | You have hand-curated, high-quality examples | You want the optimizer to discover good examples (BootstrapFewShot) | | You want a quick baseline before trying fancier optimizers | You need instruction tuning too (MIPROv2) | | You need full control over which demonstrations the LM sees | You have enough data to let DSPy search (BootstrapFewShotWithRandomSearch) | | Your task is simple enough that a few good examples suffice | Quality requires filtering examples by a metric | | You want deterministic, reproducible behavior | You want the optimizer to explore different combinations |

Rule of thumb: Use LabeledFewShot as your first optimization step. If accuracy is not high enough, upgrade to BootstrapFewShot which evaluates examples against a metric and keeps only the ones that work.

API reference

Constructor

dspy.LabeledFewShot(k=16)

| Parameter | Type | Default | Description | |-----------|------|---------|-------------| | k | int | 16 | Maximum number of demonstration examples to include per predictor |

compile()

optimizer.compile(student, *, trainset, sample=True)

| Parameter | Type | Default | Description | |-----------|------|---------|-------------| | student | dspy.Module | required | The DSPy program to optimize | | trainset | list[dspy.Example] | required | Labeled examples to use as demonstrations | | sample | bool | True | True = randomly sample k examples; False = take the first k sequentially |

Returns: A copy of student with demonstrations attached to each predictor.

If trainset is empty, the student is returned unmodified.

Basic usage

import dspy
from typing import Literal

# Configure any LM provider
lm = dspy.LM("openai/gpt-4o-mini")  # or "anthropic/claude-sonnet-4-5-20250929", etc.
dspy.configure(lm=lm)

# 1. Define your signature
class ClassifyIntent(dspy.Signature):
    """Classify the user message into an intent category."""
    message: str = dspy.InputField(desc="User message")
    intent: Literal["question", "complaint", "praise", "request"] = dspy.OutputField()

# 2. Build your program
classify = dspy.Predict(ClassifyIntent)

# 3. Create hand-picked training examples
trainset = [
    dspy.Example(message="How do I reset my password?", intent="question").with_inputs("message"),
    dspy.Example(message="This is broken and I want a refund", intent="complaint").with_inputs("message"),
    dspy.Example(message="Your team was incredibly helpful!", intent="praise").with_inputs("message"),
    dspy.Example(message="Please update my billing address", intent="request").with_inputs("message"),
    dspy.Example(message="What formats do you export to?", intent="question").with_inputs("message"),
    dspy.Example(message="The app crashes every time I open it", intent="complaint").with_inputs("message"),
]

# 4. Compile with LabeledFewShot
optimizer = dspy.LabeledFewShot(k=4)
optimized = optimizer.compile(classify, trainset=trainset)

# 5. Use the optimized program -- it now includes few-shot demos in every call
result = optimized(message="Can you send me last month's invoice?")
print(result.intent)  # request

How example selection works

When sample=True (the default):

• DSPy randomly selects k examples from trainset using a fixed seed (0)
• Every predictor in your program gets the same set of demos
• The selection is reproducible across runs because of the fixed seed

When sample=False:

• DSPy takes the first k examples from trainset in order
• Use this when the order of your examples matters or you want exact control

If your trainset has fewer than k examples, all examples are used.

Choosing k

The k parameter controls how many demonstrations appear in the prompt.

• Smaller k (2-4): Lower token cost, faster inference. Good when your examples are diverse and high-quality.
• Larger k (8-16): More context for the LM. Good when the task has many edge cases or subtle distinctions.
• Default (16): A reasonable starting point. Reduce if you hit token limits or want faster responses.

Keep in mind that each demonstration adds tokens to every LM call. For long input/output fields, use a smaller k to stay within context limits.

Using sample=False for ordered examples

When you want precise control over which examples appear, disable sampling:

# Place your best, most representative examples first
trainset = [
    dspy.Example(message="What's your return policy?", intent="question").with_inputs("message"),
    dspy.Example(message="This product is defective", intent="complaint").with_inputs("message"),
    dspy.Example(message="Love the new feature!", intent="praise").with_inputs("message"),
    dspy.Example(message="Please cancel my subscription", intent="request").with_inputs("message"),
    # ... more examples, ordered by importance
]

optimizer = dspy.LabeledFewShot(k=4)
optimized = optimizer.compile(classify, trainset=trainset, sample=False)
# The first 4 examples are used as demos, in order

Saving and loading an optimized program

After compilation, save the optimized program so you can reuse it without recompiling:

# Save
optimized.save("intent_classifier.json")

# Load later
loaded = dspy.Predict(ClassifyIntent)
loaded.load("intent_classifier.json")
result = loaded(message="How do I upgrade my plan?")

Multi-predictor programs

LabeledFewShot attaches demos to every predictor in your program. This works with multi-step pipelines too:

class SupportRouter(dspy.Module):
    def __init__(self):
        self.classify = dspy.Predict(ClassifyIntent)
        self.respond = dspy.ChainOfThought("message, intent -> response")

    def forward(self, message):
        intent = self.classify(message=message).intent
        return self.respond(message=message, intent=intent)

router = SupportRouter()

# Both self.classify and self.respond get demos from the same trainset
optimizer = dspy.LabeledFewShot(k=3)
optimized_router = optimizer.compile(router, trainset=trainset)

Note: every predictor receives demos from the same trainset. If your predictors have different signatures, make sure your training examples include all fields needed across all predictors, or consider compiling predictors separately.

When to upgrade to BootstrapFewShot

LabeledFewShot is a great starting point, but it has limitations:

1. No quality filtering -- it uses your examples as-is, even if some are misleading or ambiguous
2. No metric evaluation -- it cannot tell which examples actually help the LM perform better
3. Same demos for all predictors -- it does not tailor demonstrations per predictor

dspy.BootstrapFewShot addresses all three. It runs your program on each training example, evaluates with a metric, and keeps only the demonstrations that led to correct outputs. The upgrade is straightforward:

# Before: LabeledFewShot (no metric needed)
optimizer = dspy.LabeledFewShot(k=4)
optimized = optimizer.compile(program, trainset=trainset)

# After: BootstrapFewShot (needs a metric)
def metric(example, prediction, trace=None):
    return prediction.intent == example.intent

optimizer = dspy.BootstrapFewShot(metric=metric, max_bootstrapped_demos=4)
optimized = optimizer.compile(program, trainset=trainset)

Gotchas

1. Claude forgets .with_inputs() on training examples. Without .with_inputs("field_name"), DSPy does not know which fields are inputs vs labels. The demonstrations appear malformed in the prompt — the LM sees all fields as input, which confuses it. Always call .with_inputs() on every dspy.Example in your trainset.
2. Claude uses LabeledFewShot when the user needs metric-driven selection. LabeledFewShot uses examples as-is with no quality filtering. If the user mentions "accuracy is low" or "some examples are noisy," recommend BootstrapFewShot instead — it evaluates examples against a metric and keeps only the ones that help.
3. Claude sets k larger than the trainset without explaining the behavior. When k exceeds len(trainset), DSPy silently uses all available examples. This is fine, but Claude should tell the user: "You have 5 examples and k=16, so all 5 will be used as demos."
4. Claude creates separate trainsets for multi-predictor programs. LabeledFewShot assigns the same demos to every predictor. If predictors have different signatures, the examples need all fields across all signatures, or the user should compile predictors separately. Claude sometimes splits the trainset incorrectly — explain the shared-demo behavior.

Additional resources

• dspy.LabeledFewShot API docs
• For API details, see reference.md
• For worked examples, see examples.md

Cross-references

Install any skill: npx skills add lebsral/DSPy-Programming-not-prompting-LMs-skills --skill <name>

• Creating training examples (dspy.Example, with_inputs, datasets) -- see /dspy-data
• Defining signatures (inline and class-based, typed fields) -- see /dspy-signatures
• BootstrapFewShot for metric-driven demo selection -- see /ai-improving-accuracy
• Evaluating your program to measure if LabeledFewShot is enough -- see /dspy-evaluate
• Building modules with multiple predictors -- see /dspy-modules
• For worked examples, see examples.md
• Install /ai-do if you do not have it — it routes any AI problem to the right skill and is the fastest way to work: npx skills add lebsral/DSPy-Programming-not-prompting-LMs-skills --skill ai-do