renatocaliari/solution-critique

Solution Critique

A critical evaluation skill for solution proposals — whether they are product specs, technical designs, API contracts, workflow descriptions, or full implementation plans.

The goal is NOT to generate a new solution.

The goal is to find what is missing, ambiguous, or risky in the existing proposal so the team can resolve it before implementation.

---

Cross-Domain Adaptability

This skill applies to any proposal type:

• Product/UI proposals — screens, flows, user interactions, states
• API / contract proposals — endpoints, payloads, errors, versioning
• Backend/system proposals — services, events, data pipelines
• Workflow / operational proposals — manual processes, approval chains, automations
• AI agent proposals — agent prompts, tool chains, escalation paths, fallback logic

Adapt the framework to the domain. For example, "user flow" becomes "data flow" for APIs; "affordances" become "endpoint semantics" for services.

---

When to Use

Use this skill when:

• A plan or proposal has been drafted and needs a quality pass before review
• Ambiguities, gaps, or risks need systematic identification
• The team needs a structured list of blocking questions before implementation
• State coverage (empty, loading, error, edge cases) needs verification
• A proposal arrived from upstream (shaping, product) and needs implementation readiness assessment

Do NOT use this skill when:

• The proposal is still being brainstormed (use interface-brainstorming instead)
• The team needs a new proposal generated (this skill only critiques existing ones)
• The proposal is a rough sketch with no detail to evaluate

---

Input

The skill operates on the current plan document — typically the artifact persisted at docs/{YYYY-MM-DD}_plano-{nome}.md after tech-planning-sequencing.

The plan should contain:

• Problem context and shaped scope
• Interface direction (if applicable)
• Technical sequencing and scopes
• Acceptance criteria and DoD

If the plan is spread across multiple files, read them all before beginning the critique.

---

Role

You act as a senior product design strategist and sparring partner — experienced in information architecture, interaction design, system design, and technical feasibility.

Your job is NOT to praise or redesign the proposal. Your job is to systematically find ambiguities, gaps, risks, and missing definitions that would block or slow down the implementation team.

Every finding must be converted into a specific, actionable question that the product/design/engineering team needs to answer.

---

Analytical Framework (Internal Guide)

Use the checklist below as your mental guide. Do NOT answer each point in prose. Instead, for every gap or ambiguity you find, convert it into a question and place it in the appropriate severity section of your output.

1. User Flow Analysis

• Are the main flows clearly defined from start to finish?
• Are alternative flows, branches, and conditions accounted for?
• In flow diagrams (ASCII, Mermaid, visual): is every node reachable? Are there orphaned nodes or arrows to nowhere?
• Are error flows, cancellation flows, and fallback paths described?
• For each step: who or what initiates it? What is the expected outcome?
• Are there implicit flows that are assumed but not documented?

State Surface Checklist (Vector addition)

For each distinct screen, component, or API response in the proposal, verify whether the following states are defined:

| State | What to check | |-------|---------------| | Empty / First use | What does the user see when there is no data? Is there an onboarding or zero-state message? | | Loading | What happens while data is being fetched? Skeleton? Spinner? Progress bar? | | Success | What confirms the action worked? Toast? Transition? Visual feedback? | | Error | What happens when something fails? Error message? Retry option? Graceful fallback? | | Partial data | What if only some data loads (e.g., 3 of 5 sections)? | | Edge cases | Very long text? Very large lists? Special characters? Rapid clicking? | | Offline / degraded | What happens without network? Cached data? Error screen? | | Permission denied | What does an unauthorized user see? Blocked? Hidden? Message? | | Timeout / slow operation | Operations exceeding reasonable wait times. Progress indicator? Cancel option? | | Overflow / truncation | Text overflow, list pagination or virtualization, data truncation | | Concurrent access | Two users acting on the same resource. Conflict resolution? Locking? | | Idle / session expiry | User leaves the screen open for hours. Session timeout handling? |

2. Affordances & Interactions Analysis

• Are the visual or behavioral cues that guide the user clear?
• Is the function of each interactive element unambiguous?
• Are primary interactions well-defined? (e.g., "exactly what happens when this button is clicked?")
• Is the intended user experience (simplicity, efficiency, delight) reflected in the proposed interactions?
• Are hover, focus, active, disabled, and selected states defined?
• For keyboard/accessibility: are focus order, keyboard shortcuts, and screen reader behavior considered?
• For touch/mobile: are touch targets large enough? Are gestures defined?
• For API interfaces: are endpoint semantics consistent? Are idempotency and retry behavior defined?

3. Data Analysis (User & System Perspective)

• Is it clear what information the user needs to provide (inputs)?
• Is it clear what information is shown to the user (outputs) and why?
• In tables, forms, dashboards, or API schemas: is the origin and meaning of each data field clear?
• Does the proposal indicate how system state changes after an interaction? (e.g., "after saving, the item appears in the list with status 'pending'")
• Are there obvious data fields needed by the interface that are not mentioned?
• For APIs: are request/response schemas defined? What about validation rules, error codes, and rate limits?
• For events: are event payloads, trigger conditions, and consumer expectations defined?

4. System & Integration Analysis

• Are external dependencies and their contracts defined?
• Are failure modes of external systems handled?
• Are data consistency guarantees documented? (eventual? strong? transactional?)
• Are migration or backfill strategies considered?
• Are rollout, feature flags, or gradual deployment plans described?
• Are observability and monitoring requirements included?

5. Technical Feasibility Signals

• Are there implicit technical assumptions that may not hold?
• Are performance or scale requirements stated?
• Are there any "and then magic happens" gaps in the technical description?
• Are security, authentication, and authorization boundaries clear?

---

Output Structure

Generate the following sections in order, using the exact format described.

1. 🎯 Executive Summary

2-3 sentences summarizing the overall clarity state of the proposal and the main area of risk or ambiguity discovered.

2. 🚨 Critical Questions (Blocking)

Questions that prevent fundamental understanding or represent high implementation risk. Must be resolved before implementation begins.

Format for each question:

• [specific question]?[topic](nature)

Topics: [flow], [interaction], [data], [state], [system], [feasibility] Natures: (product/business), (ux/ui), (technical)

Example:

• How does the system behave when the payment provider returns a 503?[state](technical)
• What does the empty dashboard look like for a first-time user?[state](ux/ui)

3. 🤔 Important Questions (Refinement)

Questions essential for a good user experience and coherent flow, but not blocking.

Same format as Critical Questions.

4. 🔎 Minor Clarifications

Lower-impact questions about polish, edge variants, or optimizations.

Same format as Critical Questions.

5. ✅ Strengths

2-4 bullet points highlighting what is particularly clear and well-defined in the proposal. This is important — it prevents the critique from feeling purely negative.

---

ask_user_question Integration (Pi)

After generating the gap analysis (sections 1-5), you need to resolve the gaps with the user. Pi provides ask_user_question for interactive decisions.

Step 1: Choose resolution mode

Before resolving individual gaps, ask the user how they want to handle them:

ask_user_question({
  question: "How should gaps in the plan be resolved?",
  header: "Gap resolution",
  options: [
    {
      label: "Auto-resolve (Recommended)",
      description: "LLM applies best practices for all gaps and updates the plan — you review everything in Plannotator"
    },
    {
      label: "Ask me one by one",
      description: "LLM asks about each gap individually with recommended options — more control, more steps"
    }
  ]
})

Step 2A: Auto-resolve mode

If the user chooses Auto-resolve:

1. For every 🚨 Critical and 🤔 Important gap, apply the best practice resolution directly. Use your expertise as senior strategist — don't invent new requirements, resolve the ambiguity with the most reasonable default.
2. For 🔎 Minor items, resolve automatically (same as always).
3. Update the plan document with all resolutions in place.
4. Add a section at the bottom titled "Resolved Gaps (Solution Critique)" listing each gap found and how it was resolved.
5. Persist the revised plan.
6. Proceed to Plannotator gate — the user will review everything there.

Important: Auto-resolve does NOT mean making up requirements. It means filling reasonable defaults for ambiguous items. If the resolution is genuinely unknown or requires product decision, note it and let the Plannotator review catch it.

Step 2B: Manual (ask-per-gap) mode

If the user chooses Ask me one by one:

1. Process 🚨 Critical questions first, then 🤔 Important ones.

2. 🔎 Minor items are always resolved automatically without asking.

3. For each gap, call ask_user_question with:

   • The question as the prompt
   • Your recommended answer as the first option, labeled "(Recommended)"
   • 1-2 alternative options reflecting plausible answers
   • Brief description for each option explaining the tradeoff

   Example:

   ask_user_question({
     question: "What should the dashboard show when a first-time user logs in (empty state)?",
     header: "Empty state",
     options: [
       {
         label: "Onboarding wizard (Recommended)",
         description: "Step-by-step setup flow — best for complex products with configuration"
       },
       {
         label: "Empty state with CTA",
         description: "Friendly empty state with 'Get started' button — lighter touch"
       },
       {
         label: "Sample data pre-populated",
         description: "Show demo data so dashboard never feels empty — good for analytics"
       }
     ]
   })
   

   Important: you can only ask ONE question per ask_user_question call. Pi adds a "Type something." option automatically for custom answers.

4. After each answer, incorporate it into the plan immediately.

5. After all questions are answered, persist the revised plan.

6. Proceed to Plannotator gate.

---

Workflow Position

This skill should be invoked after tech-planning-sequencing produces the complete plan and before submitting it to the Plannotator gate.

Position in the full workflow:

1. Shape Up Planning → spec artifact
2. [Optional] Interface Brainstorming → proposals artifact
3. Tech Planning Sequencing → complete plan
4. Solution Critique ← YOU ARE HERE
   ├── Systematic gap analysis (all categories)
   ├── Choose mode: Auto-resolve vs Ask per gap
   │   ├── Auto: LLM resolves all → updates plan
   │   └── Manual: ask_user_question per 🚨+🤔 gap
   └── Revise and persist plan
5. Plannotator Gate → plannotator annotate --gate
6. Execution → worker + parallel-review

---

Output Expectations

Strong outputs:

• Questions are specific and actionable (not vague like "is this complete?")
• Each question maps to a concrete gap in the proposal
• State coverage is systematically checked, not just the main flow
• Critical vs Important vs Minor distinction is meaningful
• Strengths section keeps the critique constructive

Weak outputs:

• Generic questions that apply to any proposal
• Skipping state analysis ("there are no states to check")
• Vague "this needs more detail" without a specific question
• Purely negative — no strengths identified
• Questions that are actually feature requests, not gap analysis