a-organvm/essay-publishing-pipeline

Essay Publishing Pipeline

Move written content from draft through editing, formatting, and multi-platform distribution.

Pipeline Architecture

Draft → Edit → Format → Metadata → Build → Publish → Distribute
  ↑                                                      │
  └──────────── Feedback Loop ───────────────────────────┘

Stage Definitions

| Stage | Input | Output | Tools | |-------|-------|--------|-------| | Draft | Ideas, notes | Raw markdown | Editor, voice notes | | Edit | Raw markdown | Polished markdown | Linter, peer review | | Format | Polished markdown | Structured content | Frontmatter, templates | | Metadata | Structured content | Enriched content | Tags, categories, SEO | | Build | Enriched content | HTML/PDF output | SSG, Pandoc | | Publish | Built output | Live content | Deploy, CMS API | | Distribute | Published URL | Cross-posts | RSS, social, newsletter |

Content Structure

Markdown with Frontmatter

---
title: "On the Architecture of Automated Systems"
subtitle: "Why eight organs beat one monolith"
author: "Author Name"
date: 2026-03-20
updated: 2026-03-20
status: published
tags: [architecture, automation, organvm]
category: systems-thinking
series: "Orchestration Essays"
series_order: 3
abstract: >
  A 2000-word exploration of why modular organ-based
  architecture outperforms monolithic automation.
canonical_url: "https://example.com/essays/architecture-of-automated-systems"
---

# On the Architecture of Automated Systems

Opening paragraph that hooks the reader...

Essay Taxonomy

| Field | Purpose | Example | |-------|---------|---------| | status | Workflow state | draft, review, published, archived | | tags | Topic classification | [architecture, automation] | | category | Primary category | systems-thinking | | series | Multi-part grouping | "Orchestration Essays" | | canonical_url | SEO canonical | Primary publication URL | | abstract | Summary for feeds/cards | 1-2 sentence summary |

Markdown Processing

Conversion Pipeline

# Markdown → HTML with Pandoc
pandoc essay.md \
  --from markdown+yaml_metadata_block \
  --to html5 \
  --template template.html \
  --highlight-style tango \
  --toc \
  --toc-depth=2 \
  --output essay.html

# Markdown → PDF
pandoc essay.md \
  --pdf-engine=weasyprint \
  --css style.css \
  --output essay.pdf

Static Site Generator Integration

# Build script for essay collection
from pathlib import Path
import yaml
import markdown

def build_essays(source_dir: str, output_dir: str):
    essays = []
    for md_file in sorted(Path(source_dir).glob("*.md")):
        text = md_file.read_text()
        frontmatter, content = text.split("---\n", 2)[1:]
        meta = yaml.safe_load(frontmatter)
        if meta.get("status") != "published":
            continue
        html = markdown.markdown(content, extensions=["fenced_code", "tables", "toc"])
        essays.append({"meta": meta, "html": html, "slug": md_file.stem})

    for essay in essays:
        output = Path(output_dir) / f"{essay['slug']}/index.html"
        output.parent.mkdir(parents=True, exist_ok=True)
        output.write_text(render_template(essay))

    build_index(essays, output_dir)
    build_rss_feed(essays, output_dir)

RSS/Atom Feed Generation

from datetime import datetime
import xml.etree.ElementTree as ET

def build_rss_feed(essays: list[dict], output_dir: str):
    rss = ET.Element("rss", version="2.0")
    channel = ET.SubElement(rss, "channel")
    ET.SubElement(channel, "title").text = "Essay Collection"
    ET.SubElement(channel, "link").text = "https://example.com/essays"
    ET.SubElement(channel, "description").text = "Long-form writing on systems and culture"

    for essay in essays[:20]:
        item = ET.SubElement(channel, "item")
        ET.SubElement(item, "title").text = essay["meta"]["title"]
        ET.SubElement(item, "link").text = f"https://example.com/essays/{essay['slug']}"
        ET.SubElement(item, "description").text = essay["meta"].get("abstract", "")
        ET.SubElement(item, "pubDate").text = essay["meta"]["date"].strftime(
            "%a, %d %b %Y 00:00:00 GMT"
        )

    tree = ET.ElementTree(rss)
    tree.write(f"{output_dir}/feed.xml", xml_declaration=True, encoding="utf-8")

Cross-Posting Strategy

POSSE (Publish on Own Site, Syndicate Elsewhere)

Own site (canonical) → Medium → Dev.to → LinkedIn → Newsletter
                    ↗          ↗
            RSS feed triggers automation

Cross-Post Formatting

| Platform | Format | Limits | Notes | |----------|--------|--------|-------| | Own site | Full HTML | None | Canonical URL | | Medium | Markdown import | None | Set canonical URL | | Dev.to | Markdown + frontmatter | None | Use API for automation | | LinkedIn | Plain text + link | 3000 chars | Excerpt + link to full | | Newsletter | HTML email | Images inline | Adapt layout for email |

Automated Cross-Posting

async def cross_post(essay: dict):
    canonical = essay["meta"]["canonical_url"]

    # Dev.to
    await devto_api.create_article(
        title=essay["meta"]["title"],
        body_markdown=essay["content"],
        canonical_url=canonical,
        tags=essay["meta"]["tags"][:4],
        published=True,
    )

    # Newsletter
    await newsletter_api.create_campaign(
        subject=essay["meta"]["title"],
        html=render_email_template(essay),
    )

Quality Gates

Pre-Publish Checklist

• [ ] Spell check and grammar review
• [ ] Links verified (no 404s)
• [ ] Images optimized and alt-text present
• [ ] Frontmatter complete (title, date, tags, abstract)
• [ ] Canonical URL set
• [ ] Open Graph / social card metadata present
• [ ] RSS feed validates
• [ ] Mobile rendering verified

SEO Metadata

<meta property="og:title" content="Essay Title">
<meta property="og:description" content="Abstract text">
<meta property="og:type" content="article">
<meta property="og:url" content="https://example.com/essays/slug">
<meta name="twitter:card" content="summary_large_image">

Anti-Patterns

• Publishing without canonical URL — Leads to duplicate content SEO penalties
• Manual cross-posting — Automate with APIs and RSS triggers
• No RSS feed — Essential for syndication and discoverability
• Draft content leaking — Always check status field before building
• No versioning — Track updated date for revised essays
• Platform-first publishing — Always publish on own domain first (POSSE)