evalstate/pr-writing-review

PR Writing Review

Extract editorial feedback from GitHub PRs to learn from review improvements.

Prerequisites

• GitHub CLI: gh installed
• Authenticated gh session: gh auth status should show you’re logged in
  • For private repos, your token needs appropriate scopes (typically repo).
• Python: 3.12+
• uv (recommended): https://github.com/astral-sh/uv

Division of Labor

| Tool | Responsibility | | ----------------- | ----------------------------------------------------------------------- | | Python script | API calls, parsing, file tracking across renames, structured extraction | | LLM analysis | Pattern recognition, paragraph comparison, style lesson synthesis |

Quick Start

> **All paths are relative to the directory containing this SKILL.md file.**
> Before running any script, first `cd` to that directory or use the full path.

# Get suggestions and feedback
uv run scripts/extract_pr_reviews.py <pr_url>

# Get full first→final comparison for deep analysis
uv run scripts/extract_pr_reviews.py <pr_url> --diff

# Same as above, but cap each FIRST/FINAL dump to 2k chars for LLM prompting
uv run scripts/extract_pr_reviews.py <pr_url> --diff --max-file-chars 2000

Workflow

Step 1: Extract with --diff

uv run scripts/extract_pr_reviews.py https://github.com/org/repo/pull/123 --diff

This outputs:

1. Explicit Suggestions — exact before/after text from suggestion blocks (supports multiple suggestion blocks per comment)
2. Reviewer Feedback — plain text comments (the "why" behind changes)
3. File Evolution — first draft and final version of each text file

Tip: add --max-file-chars 2000 to keep each FIRST/FINAL dump lightweight, or pair --diff with --no-files if you only need the suggestion/feedback summaries.

Step 2: Analyze the Output

With the script output, perform this analysis:

A. Catalog the Explicit Suggestions

Create a table of mechanical fixes:

| Pattern | Original | Fixed | | -------------- | ------------------ | ------------------ | | Grammar | "Its easier" | "It's easier" | | Filler removal | "using this way" | "this way" | | Capitalization | "Image Generation" | "image generation" |

B. Map Feedback to Changes

For each reviewer feedback comment:

1. Find the relevant section in FIRST DRAFT
2. Find the same section in FINAL VERSION
3. Document what changed and why

Example:

Feedback: "would be nice to end more enthusiastically"

First draft: "...it's simple to add new tools to Claude and use them straight away."

Final: "...Let us know what you find and create in the comments below!"

Lesson: End blog posts with a call-to-action

C. Paragraph-by-Paragraph Comparison

Compare FIRST DRAFT to FINAL VERSION section by section:

• What was added?
• What was removed?
• What was reworded?
• What structural changes were made?

D. Synthesize Style Patterns

Group findings into categories:

| Category | Patterns Found | | ------------- | ------------------------------------------------ | | Clarity | Passive→active, shorter sentences, remove filler | | Precision | Vague→specific, "Create"→"Generate" | | Tone | Added enthusiasm, call-to-action endings | | Structure | Added transitions, better section flow | | Grammar | its/it's, subject-verb agreement | | Content | Added links, examples, context |

Script Options

📁 All paths are relative to the directory containing this SKILL.md file.

| Flag | Output | Use Case | | -------------------- | -------------------------------------------------------------------------------- | ----------------------------------------------------------- | | (none) | Suggestions + feedback | Quick review of what reviewers said | | --diff | Adds FIRST/FINAL file dumps to the default output | Deep analysis of how the author responded | | --max-file-chars N | Truncates each FIRST/FINAL block to N chars (appends ...[truncated X chars]) | Keep prompts within LLM token limits | | --no-files | Suppresses FIRST/FINAL dumps even when --diff is set | When you only need explicit suggestions + reviewer feedback | | --json | Raw JSON (includes file_evolutions when --diff without --no-files) | Programmatic processing |

Input formats: pass either a full PR URL, owner/repo PR_NUMBER, or owner repo PR_NUMBER.

Output Structure

Default Output

• Writing Suggestions: Grouped by reviewer, shows original→suggested text (fenced blocks) along with any reviewer note and a permalink back to GitHub
• Reviewer Feedback: Plain comments without code suggestions, each tagged with its GitHub link

With --diff

• Explicit Suggestions: Compact before/after pairs, reviewer notes, and GitHub permalinks in one place
• Reviewer Feedback: Numbered list of requests (same as default view)
• File Evolution: FIRST DRAFT and FINAL VERSION for each .md/.txt/.rst/.mdx file; add --max-file-chars to truncate each block with a visible ...[truncated X chars] indicator

Handling File Renames

The script traces files through renames by:

1. Checking each commit for rename operations
2. Building a path history (e.g., claudeimages.md → claude-images.md → claude-and-mcp.md)
3. Fetching content using the correct path for each commit

Example Analysis Output

After running the script and performing LLM analysis, produce a summary like:

## Style Lessons from PR #123

### Mechanical Fixes

- Fix grammar: "Its" → "It's" (contraction)
- Lowercase generic terms: "Image Generation" → "image generation"
- Remove filler: "the output quality of" → "the quality of"

### Reviewer-Driven Changes

- **"end more enthusiastically"** → Added call-to-action in conclusion
- **"emphasize these are SoTA"** → Changed "latest" to "state-of-the-art"
- **"add blurb about MCP Server"** → Added explanatory paragraph

### Structural Improvements

- Added transition sentence between sections
- Simplified setup instructions (3 sentences → 1)
- Added new bullet point for model flexibility

Limitations

• Only extracts inline PR review comments (not issue comments or the PR description)
• Extremely long files can still be heavy; when that happens, lower --max-file-chars or pass --no-files to keep outputs prompt-friendly

{{currentDate}} {{env}}