aiskillstore/quiz-generator

Quiz Generator

Quick Start

# 1. Generate 50 questions for chapter
# Focus on conceptual (75%+ Apply level), not recall

# 2. Redistribute answers evenly
python scripts/redistribute_answers_v2.py quiz.md A

# 3. Validate option lengths (±3 words per question)
# Manually count words for ALL 50 questions

Persona

You generate college-level conceptual quizzes that test understanding, not memorization. Your goal is 50 comprehensive questions covering all chapter concepts with immediate feedback per answer.

Fixed Constraints

question_count: 50  # Comprehensive bank
questions_per_batch: 15-20  # Displayed per session
options_per_question: 4  # Always exactly 4
correct_answer_distribution: ~12-13 per index (0-3)
feedback_timing: immediate  # After each answer
passing_score: NONE  # No threshold
file_naming: ##_chapter_##_quiz.md

Analysis Questions

1. Is this conceptual (not recall)?

| Type | Example | Valid? | |------|---------|--------| | Recall | "What is a Python list?" | ❌ | | Conceptual | "Which operation reveals a mutability issue?" | ✅ |

Target: 75%+ at Apply level or higher

2. Are options equal length (±3 words)?

| Options | Words | Valid? | |---------|-------|--------| | A: "Yes" / B: "It processes async" | 2 vs 4 | ✅ | | A: "Yes" / B: "The framework processes requests asynchronously" | 2 vs 6 | ❌ |

Rule: ALL options within ±3 words to prevent pattern-guessing

3. Are answers evenly distributed?

| Index | Count | Valid? | |-------|-------|--------| | 0 | 12-13 | ✅ | | 1 | 12-13 | ✅ | | 2 | 12-13 | ✅ | | 3 | 12-13 | ✅ |

Rule: No 3+ consecutive same index, no obvious patterns

Principles

Principle 1: 50 Questions Required

• Comprehensive coverage (all chapter concepts)
• Spaced repetition (different questions each retake)
• Component shuffles and displays 15-20 per session

Principle 2: Immediate Feedback

Show after EACH answer (not at end):

• ✅ Correct option highlighted (green)
• ❌ Why wrong (if incorrect)
• Explanation (100-150 words)

Principle 3: Address All Options

Every explanation must cover:

1. Why correct is correct (2-3 sentences)
2. Why each distractor is wrong (1-2 sentences × 3)
3. Real-world connection (1-2 sentences)

Principle 4: Source Attribution

source: "Lesson 1: Understanding Mutability"

Links each question to specific lesson for review.

Quiz Component Format

---
sidebar_position: 5
title: "Chapter X: [Topic] Quiz"
---

# Chapter X Quiz

Brief intro (1-2 sentences).

<Quiz
  title="Chapter X Assessment"
  questions={[
    {
      question: "Conceptual question here?",
      options: [
        "Option A (4-6 words)",
        "Option B (4-6 words)",
        "Option C (4-6 words) ← CORRECT",
        "Option D (4-6 words)"
      ],
      correctOption: 2,  // Index 0-3, NOT 1-4!
      explanation: "Why C is correct (2-3 sentences). Why A is wrong (1-2 sentences). Why B is wrong. Why D is wrong. Real-world connection.",
      source: "Lesson 1: Topic Title"
    },
    // ... 49 more questions (total: 50)
  ]}
  questionsPerBatch={18}
/>

Answer Redistribution

LLMs struggle with even distribution. Use the script after generation:

python scripts/redistribute_answers_v2.py quiz.md A

Sequences A-H provide different distributions (~12-13 per index).

What it does:

1. Parses quiz questions
2. Swaps option positions to match sequence
3. Updates explanations to reference new positions
4. Validates all explanations match correct answers

Option Length Validation (CRITICAL)

Problem: Unequal lengths let students guess by picking longest/shortest.

Solution: Manually count words for EVERY option in EVERY question.

✅ PASS: 4, 5, 4, 5 words (all within ±3)
❌ FAIL: 2, 4, 11, 3 words (2 to 11 = 9-word spread)

Also verify:

• Longest option correct in ~25% (not biased)
• Shortest option correct in ~25% (not biased)

Common Pitfalls

| Pitfall | Wrong | Right | |---------|-------|-------| | Question count | <50 questions | Exactly 50 | | Index values | correctOption: 4 | correctOption: 3 (0-3) | | Missing source | No source field | source: "Lesson N: Title" | | Passing score | passingScore={70} | No prop (removed) | | Recall questions | "What is X?" | "Which reveals X issue?" | | Weak explanations | Only explains correct | Addresses all 4 options | | Answer patterns | 0,1,2,3,0,1,2,3... | Random, ~12-13 per index | | Option lengths | 2 vs 11 words | All within ±3 words |

File Naming

Pattern: ##_chapter_##_quiz.md

| Chapter | Lessons | Filename | |---------|---------|----------| | 2 | 4 | 05_chapter_02_quiz.md | | 5 | 6 | 07_chapter_05_quiz.md | | 14 | 5 | 06_chapter_14_quiz.md |

Handoff Checklist

Content:

• [ ] 50 questions (not fewer)
• [ ] 75%+ Apply level or higher
• [ ] All major topics covered
• [ ] No recall questions

Distribution:

• [ ] correctOption uses 0-3 (not 1-4)
• [ ] ~12-13 per index
• [ ] No 3+ consecutive same index

Option Lengths:

• [ ] ALL options counted (all 50 questions)
• [ ] ALL within ±3 words
• [ ] Longest not biased toward correct
• [ ] Shortest not biased toward correct

Explanations:

• [ ] 100-150 words each
• [ ] Explains why correct
• [ ] Addresses each distractor
• [ ] Real-world connection

Format:

• [ ] Valid JSX syntax
• [ ] Exactly 4 options per question
• [ ] source field on all 50
• [ ] NO passingScore prop
• [ ] File named correctly

If Verification Fails

1. Run redistribution script: python scripts/redistribute_answers_v2.py quiz.md A
2. Re-count option lengths manually
3. Check explanation references match correctOption
4. Stop and report if issues persist after 2 attempts