qte77/designing-mas-plugins

Designing MAS Plugins

Target: $ARGUMENTS

When to Use

Trigger this skill when:

• Designing agent plugins or evaluation components
• Planning pipeline architecture
• Architecting new metrics or evaluation tiers
• Refactoring engines into plugin patterns

Core Principles

Plugins follow six principles. For worked code examples of each, see references/core-principles-with-examples.md.

1. Stateless Reducer — evaluate(context) -> result as a pure function; no side effects, no shared state
2. Own Context Window — plugin manages its own context; no global state access
3. Structured Outputs — all data uses validated models, no raw dicts
4. Own Control Flow — plugin handles its own errors and timeouts
5. Compact Errors — structured partial results, not exceptions
6. Single Responsibility — one metric or tier per plugin

Plugin Design Checklist

Before implementing a plugin, verify:

• [ ] Stateless: No class attributes, no global state
• [ ] Own Context: All inputs via evaluate() parameter
• [ ] Typed I/O: Validated models for inputs and outputs
• [ ] Own Errors: Returns error results, doesn't raise
• [ ] Own Timeout: Respects configured timeout
• [ ] Single Responsibility: One metric or tier
• [ ] Explicit Context: Filters output for next stage
• [ ] Env Config: All config via env vars / settings
• [ ] Observable: Emits structured logs for debugging
• [ ] Graceful Degradation: Partial results on failures

Anti-Patterns

• Shared State: self.cache = {} (breaks stateless)
• Raw Dicts: return {"score": 0.5} (use models)
• Raising Exceptions: raise ValueError() (return error)
• Global Access: config.get_global() (use settings)
• Implicit Context: Passing entire result to next tier
• Multiple Responsibilities: One plugin, 3 metrics

Implementation Template

See references/plugin-implementation-template.md for the full EvaluatorPlugin abstract base class and a worked MyPlugin example with typed context/result models, error handling, and next-tier context filtering.

Testing Strategy

See references/plugin-testing-strategy.md for isolation test patterns — happy path and structured-error-handling tests using mocked context.

References

• references/mas-design-principles.md — foundational design principles (existing)
• references/core-principles-with-examples.md — code examples for each of the six core principles
• references/plugin-implementation-template.md — full EvaluatorPlugin + MyPlugin template
• references/plugin-testing-strategy.md — isolation test patterns

Further Reading

• 12-Factor Agents
• Anthropic Harnesses
• PydanticAI Patterns
• NIST AI RMF 1.0
• ISO/IEC 42001:2023 — AI Management System