lebsral/ai-auditing-code
skills/ai-auditing-code/SKILL.md
Review DSPy code for correctness and best practices. Use when you want a code review of your DSPy program, need to check if your AI code follows best practices, want to find anti-patterns in your DSPy usage, or need a quality audit of your AI implementation. Also use for DSPy code review, is my DSPy code correct, review my AI code, best practices check, DSPy anti-patterns, code quality audit, am I using DSPy right, sanity check my AI code, peer review my DSPy program, does this follow DSPy conventions.
$ install --global
skillsauthnpx skillsauth add lebsral/dspy-programming-not-prompting-lms-skills ai-auditing-codeInstall this skill globally with one command. Works with Claude Code, Cursor, and Windsurf.
Security Scan Results
3 of 9 scanners reported clean
Some scanners were skipped, did not run, or reported a non-clean status. Review each row below.
3
Scanners Passed
9
Scanners in report
Clean
TrivyContainer and dependency vulnerability scanner
95%
Clean
SemgrepStatic code analysis for vulnerabilities
95%
Clean
mcp-scan (Snyk)Model Context Protocol security validation
95%
Skipped
Snyk (dep)Open source security scanning
50%
Skipped
Socket.devSupply chain security analysis
50%
Skipped
VirusTotalMulti-engine malware detection
50%
Skipped
CrowdStrikeAdvanced threat intelligence
50%
Skipped
OSV-ScannerOpen Source Vulnerability database check
50%
Skipped
OWASP Dep-Check
50%
Last scanned: May 5, 2026, 7:54 AM249.2s5 files scanned
SKILL.md
- name:
- ai-auditing-code
- description:
- Review DSPy code for correctness and best practices. Use when you want a code review of your DSPy program, need to check if your AI code follows best practices, want to find anti-patterns in your DSPy usage, or need a quality audit of your AI implementation. Also use for DSPy code review, is my DSPy code correct, review my AI code, best practices check, DSPy anti-patterns, code quality audit, am I using DSPy right, sanity check my AI code, peer review my DSPy program, does this follow DSPy conventions.
Audit Your DSPy Code
When NOT to use this skill:
- Code is crashing or throwing errors -> use
/ai-fixing-errorsfirst - Need to measure or improve accuracy -> use
/ai-improving-accuracy - Have not built anything yet -> use
/ai-kickoffor/ai-choosing-architecture - Want to learn a specific DSPy API -> use the matching
/dspy-*skill
Step 1: Ask 2 questions
Before reading any code, ask:
- Point me to the code — which files contain your DSPy code?
- What are your concerns? — general review, a specific worry, or a pre-launch check?
Do not proceed until you have answers to both.
Step 2: Read the code
Read all DSPy-related files the user points you to. Build a mental model of:
- Signatures defined and their field names/types
- Modules defined and how they compose
- How
forward()methods pass data between sub-modules - Where data is loaded and how examples are constructed
- Whether a metric exists and how it is implemented
- Whether an optimizer is used and how it is called
Step 3: Run the 7-category audit
For each category, check the items in reference.md. Mark each finding:
- CRITICAL — causes silent failures, wrong results, or data loss (e.g., missing
with_inputs(), metric always returnsTrue) - WARNING — suboptimal but works (e.g.,
PredictwhereChainOfThoughtwould help, no error handling) - INFO — style or convention suggestions (e.g., naming, file organization)
Category 1: Signature Design
Check that field names are descriptive (not output, result, text), types are correct, complex outputs use Pydantic models, and ambiguous fields have a desc= argument.
Category 2: Module Composition
Check that modules are registered in __init__, forward() passes sub-module outputs as typed fields (not strings), and there is no raw string manipulation of LM outputs.
Category 3: Data Pipeline
Check that with_inputs() is called on every dspy.Example, a train/dev split exists, example field names match the signature, and data loading is reproducible (no random shuffles without a seed).
Category 4: Metric Design
Check that the metric handles edge cases (empty strings, None), returns a float between 0 and 1 or a bool, accepts the trace parameter, and can be tested independently of the module.
Category 5: Optimizer Usage
Check that the right optimizer is chosen for the dataset size, the trainset is large enough, the metric is passed correctly, and the optimized program is saved with program.save().
Category 6: Production Readiness
Check that LM calls are wrapped in error handling, timeouts are set, there is a fallback for LM failures, and costs have been estimated before deployment.
Category 7: Anti-patterns
Check for f-string prompt construction instead of signatures, direct lm() calls instead of using modules, hardcoded prompt strings alongside DSPy code, and mixed raw API calls with DSPy modules.
Step 4: Generate the findings report
After completing the audit, produce this report:
## DSPy Code Audit: [Project/Module Name]
### Summary
- X findings: Y critical, Z warnings, W info
- Overall assessment: [Ready for production / Needs fixes before production / Needs significant rework]
### Critical Findings
1. **[Category] — [Issue]**
- File: path/to/file.py:line
- Problem: ...
- Fix: ...
- Code:
Before: <code>
After: <code>
### Warnings
1. **[Category] — [Issue]**
- File: path/to/file.py:line
- Problem: ...
- Fix: ...
### Info
1. **[Category] — [Suggestion]**
- ...
### Recommended Next Steps
1. Fix all critical findings
2. Address warnings in priority order
3. Run /ai-improving-accuracy to measure baseline quality after fixes
Step 5: Offer to fix
After presenting the report, ask:
"Would you like me to apply the critical and warning fixes directly to your code?"
If yes, make the fixes. Do not silently apply fixes during the audit itself.
Gotchas
-
Do not rewrite code during the audit. Audit first, present findings, then fix on request. Silent refactoring while reviewing confuses users about what changed and why.
-
Do not rate everything CRITICAL. Reserve CRITICAL for things that cause silent failures, wrong results, or data loss. Suboptimal patterns are WARNING. Style issues are INFO.
-
Do not audit accuracy instead of code. This skill reviews code structure and patterns — not whether the AI produces correct answers. For accuracy measurement, send the user to
/ai-improving-accuracy. -
Respect domain context. A simple
Predictmodule may be entirely correct for a simple task. Do not recommendChainOfThoughteverywhere or assume complexity is always better. -
Do not suggest MIPROv2 for every finding. Not every issue requires an optimizer-level fix. Many issues are plain code bugs that need to be corrected before any optimization makes sense.
Additional resources
- Full 7-category checklist with code examples: reference.md
- Worked audit examples (ticket classifier, RAG pipeline, content generator): examples.md
Cross-references
Install any skill:
npx skills add lebsral/DSPy-Programming-not-prompting-LMs-skills --skill <name>
- Code is crashing? Fix it first with
/ai-fixing-errors - Measure accuracy after fixing — see
/ai-improving-accuracy - Plan your AI feature — see
/ai-planning - Pick the right DSPy pattern — see
/ai-choosing-architecture - Signature design patterns — see
/dspy-signatures - Module composition — see
/dspy-modules - Optimizer selection — see
/dspy-optimizers - Install
/ai-doif you do not have it — it routes any AI problem to the right skill and is the fastest way to work:npx skills add lebsral/DSPy-Programming-not-prompting-LMs-skills --skill ai-do
Related Skills
lebsral/ai-watching-optimization
tools
VerifiedTrustedCommunity
See what is happening during optimizer.compile() instead of waiting blind. Use when you want to watch optimization progress, see scores as they come in, know if your optimizer is working, check if optimization is stuck, understand why optimization is taking too long, get live progress during compile, monitor convergence, detect overfitting during optimization, interpret optimization results, or pick the right tool for watching optimization. Also used for optimizer progress bar, is my optimizer doing anything, optimization seems stuck, how long will optimization take, watch GEPA run, watch MIPROv2 run, live optimization dashboard, optimizer not improving, scores not going up, optimization taking forever, see what optimizer is doing, debug slow optimization, optimization visibility, optimizer metrics, track compile progress, optimization observability.
6SKILL.mdUpdated May 31, 2026
lebsral/dspy-miprov2
testing
VerifiedTrustedCommunity
Use when you want the highest-quality prompt optimization DSPy offers — jointly optimizes instructions and few-shot demos, with auto=light/medium/heavy presets. Common scenarios - you want the best possible accuracy from prompt optimization, jointly tuning instructions and few-shot demonstrations, using auto presets for different compute budgets, or when COPRO or BootstrapFewShot alone are not reaching your accuracy target. Related - ai-improving-accuracy, dspy-copro, dspy-bootstrap-few-shot. Also used for dspy.MIPROv2, best DSPy optimizer, highest quality optimization, auto=light medium heavy, joint instruction and demo optimization, most powerful prompt optimizer, MIPROv2 vs COPRO vs BootstrapFewShot, which optimizer should I use, state of the art prompt optimization, when to use MIPROv2, optimize both instructions and examples, heavy optimization for production, best optimizer for accuracy.
6SKILL.mdUpdated Apr 27, 2026
lebsral/dspy-langwatch
testing
VerifiedTrustedCommunity
Use LangWatch for DSPy auto-tracing and real-time optimizer progress. Use when you want to set up LangWatch, langwatch.dspy.init, auto-tracing DSPy, real-time optimization dashboard, optimizer progress tracking, app.langwatch.ai, or DSPy optimizer dashboard. Also used for langwatch setup, pip install langwatch, langwatch trace, optimizer progress, real-time optimization, watch optimizer run, LangWatch self-hosted, langwatch docker, langwatch vs langtrace, langwatch autotrack_dspy.
6SKILL.mdUpdated Apr 27, 2026
lebsral/dspy-gepa
data-ai
VerifiedTrustedCommunity
Use when you want to optimize instructions without few-shot examples — a lightweight alternative to COPRO when you do not have or do not want to use demonstrations. Common scenarios - optimizing instructions when you do not have or do not want to use few-shot demonstrations, lightweight instruction search as a first step, tasks where examples in the prompt confuse the model, or when you want fast instruction optimization without the cost of COPRO. Related - ai-improving-accuracy, dspy-copro, dspy-miprov2. Also used for dspy.GEPA, instruction optimization without demos, lightweight prompt optimization, optimize instructions only, no few-shot examples needed, GEPA vs COPRO, quick instruction search, when demonstrations hurt performance, zero-shot optimization, instruction-only optimizer, simplest instruction tuner, fast prompt optimization, skip few-shot and just tune instructions, optimize Pydantic field descriptions, GEPA structured output, GEPA does not optimize field desc.
6SKILL.mdUpdated Apr 27, 2026