synaptiai/workflow-validation

Workflow Validation

You validate FlowWorkflow YAMLs at two levels: schema conformance + cross-reference correctness.

Iron Law

A workflow that schema-validates but references a non-existent skill is a bug waiting to fire at runtime. Cross-reference validation catches this at author-time so it never reaches the user.

Inputs

The invoking command MUST pass:

1. Workflow id — start-issue | review-pr | address-pr | merge-pr | release | debug | design (or a future custom workflow). Maps to plugins/flow/workflows/<id>.workflow.yaml.

Outputs

Structured JSON report on stdout. Each violation entry includes source_file (the YAML path that failed) and example (a snippet showing the corrected form) so the caller can render actionable error messages without consulting the schema separately:

{
  "workflow_id": "start-issue",
  "source_file": "plugins/flow/workflows/start-issue.workflow.yaml",
  "schema_valid": true,
  "cross_reference_errors": [
    {
      "type": "missing_skill",
      "name": "specifcation-capture",
      "source_file": "plugins/flow/workflows/start-issue.workflow.yaml",
      "example": "required_skills:\n  - specification-capture  # correct spelling"
    }
  ],
  "tier3_violations": [],
  "native_slash_violations": [],
  "overall": "pass"
}

Exit code: 0 if overall: pass; 1 if any cross-reference, recursion, native-slash, or tier-3 violation; 2 if schema invalid.

Per-violation field contract:

• source_file — the YAML path (project-local override takes precedence over plugin default; whichever was actually loaded).
• example — a YAML snippet showing the corrected shape for this specific violation. The renderer in commands/workflow.md:validate prints source_file + example below each error.

Workflow

Step 1: Schema validation (with v3.0.x deprecation migration)

the previous documentation showed both an in-memory migration shim AND a python3 -m jsonschema -i <file> <schema> invocation. The CLI re-reads the file from disk, bypassing the in-memory shim entirely. Any project-local workflow with the legacy completion_gate.requires field would fail schema validation despite the documented "accept legacy through v3.0.x" contract. The shim and the validator must run in the same Python process. The canonical invocation is:

python3 - "$WORKFLOW_PATH" "$SCHEMA_PATH" <<'PYEOF'
import sys, yaml, json, jsonschema
sys.path[:] = [p for p in sys.path if p not in ("", ".")]

workflow_path = sys.argv[1]
schema_path = sys.argv[2]

with open(workflow_path, "r", encoding="utf-8") as f:
    wf = yaml.safe_load(f)

# Pre-validation migration (v3.0.x deprecation shim).
gate = (wf or {}).get("completion_gate") or {}
if "requires" in gate and "documented_requirements" not in gate:
    gate["documented_requirements"] = gate.pop("requires")
    wf["completion_gate"] = gate
    print(
        f"WARN: {workflow_path}: completion_gate.requires is deprecated — "
        f"rename to completion_gate.documented_requirements (will be required in v3.1)",
        file=sys.stderr,
    )
elif "requires" in gate and "documented_requirements" in gate:
    # both fields present — drop legacy,
    # emit WARN. Previously the shim silently skipped this case and the schema
    # rejected the YAML with an opaque `additionalProperties` error.
    del gate["requires"]
    wf["completion_gate"] = gate
    print(
        f"WARN: {workflow_path}: both completion_gate.requires (legacy) and "
        f".documented_requirements present — dropping legacy field. Remove from source.",
        file=sys.stderr,
    )

with open(schema_path, "r", encoding="utf-8") as f:
    schema = json.load(f)
try:
    jsonschema.validate(wf, schema)
    print("schema_valid: true")
except jsonschema.ValidationError as e:
    print(f"schema_valid: false\nerror: {e.message}")
    sys.exit(2)
PYEOF

If schema validation fails, populate cross_reference_errors with the first error and set overall: schema_invalid. Skip remaining steps.

Step 2: Required-skills cross-reference

For each entry in required_skills[]:

[ -f "plugins/flow/skills/${skill_name}/SKILL.md" ]

Missing → cross_reference_errors.append({"type": "missing_skill", "name": skill_name, "source_file": <loaded YAML path>, "example": "required_skills:\n - <correct-skill-name>\n# check plugins/flow/skills/ for existing skill directory names"}).

Step 3: Required-agents cross-reference

For each entry in required_agents[]:

[ -f "plugins/flow/agents/${agent_name}.md" ]

Missing → cross_reference_errors.append({"type": "missing_agent", "name": agent_name, "source_file": <loaded YAML path>, "example": "required_agents:\n - <correct-agent-name>\n# check plugins/flow/agents/*.md for available agents"}).

Step 4: Activity-level skill/agent cross-reference

Walk phases[].activities[]. For each activity with a skill field, repeat the existence check from Step 2. For each agent field, repeat Step 3.

Step 5: Tier classification check

In tier_classification, verify:

• merge is confirm (Tier 3 — Iron Law, hard fail)
• release is confirm (Tier 3 — Iron Law, hard fail)
• tag_push is confirm when present (Iron Law, hard fail)

Any of merge, release, or tag_push set to autonomous or journal → tier3_violations.append({"action": ..., "value": ..., "expected": "confirm", "source_file": <loaded YAML path>, "example": "tier_classification:\n merge: confirm # Tier 3 must be confirm — non-negotiable per the Iron Law"}). Hard fail (exit 1). This matches trigger-policy/SKILL.md Step 2 — the project's "No Irreversible Actions Without Approval" boundary is non-negotiable; a workflow that downgrades Tier 3 is broken by construction, not "legitimately overriding."

Step 6: No-native-slash check

Grep the entire workflow YAML for /goal\b, /loop\b, /schedule\b, /routine\b outside of description fields. Any match that looks like an invoked dependency (e.g., command: /goal foo) → native_slash_violations.append(...). Hard fail.

Step 7: Completion gate documentation check

completion_gate.documented_requirements is advisory documentation, not an enforced check vocabulary — commands enforce their own gates (goal lifecycle, finding ledger, quality commands). This step only verifies the field is present and non-empty (already enforced by the schema's minItems: 1). No cross-reference to activity evidence fields is performed; the labels are free-text by design.

If a future version reintroduces enforcement, the registry of valid labels would live here. For v3.0.x, this step is a no-op beyond the schema check.

Step 8: Overall verdict

| Condition | overall | |---|---| | schema fails | schema_invalid (exit 2) | | any native_slash_violations | native_slash_present (exit 1) | | any tier3_violations | tier3_invalid (exit 1 — Iron Law) | | any cross_reference_errors | cross_reference_failed (exit 1) | | else | pass (exit 0) |

Anti-patterns

• ❌ Treating schema-valid as cross-reference-valid. Schema only catches shape; references catch identity.
• ❌ Treating Tier 3 downgrade as a soft warning. The "No Irreversible Actions Without Approval" boundary is non-negotiable.
• ❌ Accepting workflows that reference native /goal. Per the v3 non-goals — plugins cannot invoke native slash commands.

Reuse map

• plugins/flow/schemas/v1/workflow.schema.json — schema this skill validates against.
• plugins/flow/workflows/*.workflow.yaml — workflows this skill validates.
• plugins/flow/commands/workflow.md — the /flow:workflow dispatcher that invokes this skill.