8Dionysus/aoa-eval-design
skills/core/engineering/aoa-eval-design/SKILL.md
--- name: aoa-eval-design scope: core status: scaffold summary: Design a local eval suite or report after selection fails, using deterministic checks first and leaving central proof acceptance to aoa-evals. invocation_mode: explicit-preferred technique_dependencies: - AOA-T-0015 - AOA-T-0017 - AOA-T-0007 --- # aoa-eval-design ## Intent Use this skill to design a bounded local eval suite, prompt set, report, or validator plan after existing surfaces have been inspected and found insuffic
$ install --global
skillsauthnpx skillsauth add 8Dionysus/aoa-skills aoa-eval-designInstall this skill globally with one command. Works with Claude Code, Cursor, and Windsurf.
Security Scan Results
3 of 9 scanners reported clean
Some scanners were skipped, did not run, or reported a non-clean status. Review each row below.
3
Scanners Passed
9
Scanners in report
Clean
TrivyContainer and dependency vulnerability scanner
95%
Clean
SemgrepStatic code analysis for vulnerabilities
95%
Clean
mcp-scan (Snyk)Model Context Protocol security validation
95%
Skipped
Snyk (dep)Open source security scanning
50%
Skipped
Socket.devSupply chain security analysis
50%
Skipped
VirusTotalMulti-engine malware detection
50%
Skipped
CrowdStrikeAdvanced threat intelligence
50%
Skipped
OSV-ScannerOpen Source Vulnerability database check
50%
Skipped
OWASP Dep-Check
50%
Last scanned: Jul 14, 2026, 4:26 AM390.2s4 files scanned
SKILL.md
- name:
- aoa-eval-design
- scope:
- core
- status:
- scaffold
- summary:
- Design a local eval suite or report after selection fails, using deterministic checks first and leaving central proof acceptance to aoa-evals.
- invocation_mode:
- explicit-preferred
aoa-eval-design
Intent
Use this skill to design a bounded local eval suite, prompt set, report, or validator plan after existing surfaces have been inspected and found insufficient.
Trigger boundary
Use this skill when:
- a local eval needs design from a clear failure mode or missing invariant
- deterministic tests, validators, or scripts should be planned before rubric or transcript graders
- positive, negative, collision, or regression cases need to be shaped for a local repo
- the output is a draft suite or report, not central proof acceptance
- the parent has classified design after selection, but the explicit invariant
or behavior, target repository, local owner path, or rejected existing
surfaces is unavailable inside the active evidence boundary; select this
child only to stop with
blocked_missing_inputrather than design generically
Do not use this skill when:
- an existing eval should simply run now
- the right next artifact is only an intake pressure packet
- the invariant is unknown and discovery is still needed
- the user asks for central
aoa-evalsproof doctrine changes
Inputs
- selected failure mode, target behavior, invariant, or trigger boundary
- local repo tests, scripts, validators, fixtures, generated surfaces, and
evals/port - evidence refs and rejected existing evals
- central
aoa-evalsstandards when relevant
Outputs
- draft suite structure, report plan, fixture set, or validator design
- positive, negative, and regression cases
- deterministic checks first, rubric or trace review only when needed
- proof limit and owner handoff
Procedure
- if the design route is selected but the explicit invariant or behavior,
target repository, local owner path, or rejected existing surfaces is
unavailable inside the active evidence boundary and no permitted packet or
source read can supply it, stop with
blocked_missing_input; do not relabel missing input asdeferred_owner_boundary - restate the behavior or invariant to evaluate
- choose the lowest-level deterministic checks that can constrain it
- add positive and negative examples; add collision cases for trigger or router behavior
- add trace or rubric review only where deterministic checks cannot cover the behavior
- place the draft in the local eval port or report surface named by the repo
- keep central acceptance language out of the draft
- hand off to
aoa-eval-applywhen a runnable check exists
Contracts
- design starts after selection fails, not before
- local design remains local until central review accepts it
- deterministic checks are preferred for coding-agent behavior
- transcript and rubric grading are supporting layers when objective checks are insufficient
- missing design context inside the active evidence boundary ends as
blocked_missing_input, not as a generic draft or owner-boundary deferral
Risks and anti-patterns
- designing a suite around vague success criteria
- using only positive examples
- creating rubric-only evals where a script or test can check the behavior
- making one local report sound like system-wide proof
- relabelling absent design inputs as
deferred_owner_boundary
Verification
- confirm the target invariant or behavior is explicit
- confirm positive, negative, and regression cases exist or are deliberately deferred
- confirm deterministic checks were considered first
- confirm local owner path and proof limit
- confirm absent design inputs stopped as
blocked_missing_inputwithout being relabelled asdeferred_owner_boundary - confirm next runnable command or review route
Technique traceability
Manifest-backed techniques:
- AOA-T-0015 from
8Dionysus/aoa-techniquesat1a7d146957108ecefc24219c7d56357c5a4a2c2cusing pathtechniques/proof/skill-support/contract-test-design/TECHNIQUE.mdand sections: Intent, Inputs, Outputs, Core procedure, Contracts, Risks, Validation - AOA-T-0017 from
8Dionysus/aoa-techniquesat1a7d146957108ecefc24219c7d56357c5a4a2c2cusing pathtechniques/proof/skill-support/property-invariants/TECHNIQUE.mdand sections: Intent, Inputs, Outputs, Core procedure, Contracts, Risks, Validation - AOA-T-0007 from
8Dionysus/aoa-techniquesat1a7d146957108ecefc24219c7d56357c5a4a2c2cusing pathtechniques/proof/evaluation-chain/signal-first-gate-promotion/TECHNIQUE.mdand sections: Intent, Inputs, Outputs, Core procedure, Contracts, Risks, Validation
Adaptation points
- local fixture format
- local suite/report naming
- central review handoff
Related Skills
8Dionysus/aoa-knowledge-stewardship
development
VerifiedTrustedCommunity
Sanitize private technical material into a public-safe owner-bounded derivative, or resolve authority among authored, generated, runtime, and installed sources. Use for governed incidents, logs, configs, diagnostics, or conflicting source roles. Do not use for ordinary editing, memo/session work, direct publication, or durable-memory authority.
4SKILL.mdUpdated Jul 16, 2026
8Dionysus/aoa-session-recovery
tools
VerifiedTrustedCommunity
Diagnose a reviewed recurring session/workflow failure, or carry one bounded owner repair through checkpoint, rollback, and real health verification. Use for reviewed contamination, drift, repeated route/tool failure, or an established diagnosis ready for repair. Do not use on live evidence, for vague self-improvement, or to call a proposed or merely executed change verified.
4SKILL.mdUpdated Jul 16, 2026
8Dionysus/aoa-session-harvest
tools
VerifiedTrustedCommunity
Use only when a literal ref to a closed/reviewed session packet is supplied, to extract, classify, propose promotion or branching, or identify an automation opportunity. Missing, live, or unreviewed evidence must fail closed without inventing a ref. Natural closeout preservation belongs to aoa-memo-writeback. Do not use for generic summaries, progression, direct owner writes, scheduling, mutation, or telemetry authority.
4SKILL.mdUpdated Jul 16, 2026
8Dionysus/aoa-verification
testing
VerifiedTrustedCommunity
Turn one stable technical truth into bounded evidence through contract, coverage-audit, or property mode. Use to define a named consumer seam, audit what existing checks really constrain, or express a broad invariant after manual cases establish meaning. Compose with aoa-eval when an evaluation surface must first be found or applied. Do not use for undefined behavior, generic test plans, or green checks as whole-system proof.
4SKILL.mdUpdated Jul 16, 2026