bigeasyfreeman/slop-judge

Slop Judge

Use this only after stop-slop regex and pattern checks do not hard-fail.

The judge exists to close the output-side quality loop. It is not a prompt rewriter. It scores the produced output against a concrete rubric and returns a numeric result that a gate can compare to a threshold.

Inputs

{
  "mode": "general | outreach | product_output | agent_output",
  "regex_screen": "pass",
  "content": "string",
  "audience": "internal | external",
  "rubric": [
    "Concrete criterion such as specificity, actionability, schema validity, policy fit, or non-hallucination"
  ],
  "threshold": 0.7,
  "baseline_score": 0.82,
  "regression_tolerance": 0.03
}

Output

{
  "verdict": "pass | revise",
  "score": 0.0,
  "threshold": 0.7,
  "criterion_scores": [
    {
      "criterion": "string",
      "score": 0.0,
      "reason": "string"
    }
  ],
  "regression_delta": 0.0,
  "rationale": "string",
  "signals": ["generic_filler", "passive_evasion", "tautology"],
  "new_eval_case_candidate": {
    "source": "council_rejection | human_edit | runtime_failure | production_sample | other",
    "input": "optional string",
    "bad_output": "string",
    "expected_quality": "string",
    "metric": "rubric_score",
    "threshold": 0.7
  }
}

Rules

• pass means score >= threshold and any baseline regression is within regression_tolerance.
• revise means the output scored below threshold, regressed beyond tolerance, or remains padded, evasive, generic, structurally invalid, or self-referential even though deterministic checks did not catch it.
• Score each rubric criterion from 0 to 1. The top-level score is the average unless the input names a stricter metric.
• If the output fails because the benchmark is missing, return revise with signals: ["missing_benchmark"].
• When revising, include new_eval_case_candidate so feedback-loop can promote the failure into the saved suite after approval.
• Limit signals to observed issues. Do not invent brand guidance that is not in the source skill.