lyndonkl/fetch-pubmed-recent

fetch-pubmed-recent

Fetch PubMed records for a date window + keyword set. Normalize the output to the same shape as fetch-preprint-recent so a calling agent can dedupe across sources.

Workflow

- [ ] Step 1: Detect transport — prefer PubMed MCP if available, else E-utilities via WebFetch
- [ ] Step 2: Build the keyword query with explicit date bounds
- [ ] Step 3: Execute search → list of PMIDs (page if > 100)
- [ ] Step 4: Fetch metadata + abstracts for each PMID batch
- [ ] Step 5: Normalize records to canonical shape
- [ ] Step 6: Return matched records + summary

Step 1 — Detect transport

At session start the runtime exposes any connected MCP servers. Check whether tools matching mcp__*PubMed*__search_articles and mcp__*PubMed*__get_article_metadata are available.

• MCP available: use it. The MCP wraps the same E-utilities but handles pagination and field extraction. Pass the keyword query as the search string with the date filter appended (see Step 2).
• MCP not available: fall back to direct E-utilities calls via WebFetch. Endpoints:
  • Search: https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi?db=pubmed&term={query}&retmode=json&retmax=200
  • Summary: https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esummary.fcgi?db=pubmed&id={pmids_csv}&retmode=json
  • Abstract: https://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?db=pubmed&id={pmids_csv}&rettype=abstract&retmode=xml

Both transports return the same downstream shape. The skill caller should not need to care which was used (record transport_used in the return summary so debugging is possible).

Step 2 — Build the date-bounded query

PubMed accepts boolean keyword expressions plus a date filter. The canonical date filter for "papers indexed in PubMed during this window" is:

("YYYY/MM/DD"[PDAT] : "YYYY/MM/DD"[PDAT])

PDAT is the publication date — the most stable choice for a weekly digest. (EDAT is the entry-into-PubMed date, useful if you want "newly indexed even if old"; MHDA is the MeSH-indexed date. Default PDAT.)

For each watchlist keyword, expand into a sub-query:

• Plain term: "protein language model"[Title/Abstract]
• Term + MeSH: ("Alzheimer Disease"[MeSH] OR "alzheimer's disease"[Title/Abstract]) — only when the keyword has a clean MeSH match the caller has confirmed; do not auto-MeSH-expand or you'll widen the net unpredictably.

Combine sub-queries with OR, then AND the date filter:

(("term1"[Title/Abstract]) OR ("term2"[Title/Abstract]) OR ...) AND ("2026/05/04"[PDAT] : "2026/05/10"[PDAT])

Watch out for special characters — escape () inside terms by quoting the term.

Step 3 — Execute search

Call the search tool. Expected output is a list of PMIDs and a count.

If count > 200, page: rerun the search with retstart=200, retstart=400, etc. Cap at 1,000 PMIDs (5 pages). If a 7-day window matches more than 1,000 records, the keyword set is too broad — surface this and ask the calling agent to tighten.

Step 4 — Fetch metadata + abstracts

Batch the PMIDs (200 per esummary call). Pull:

• PMID
• Title
• Authors (full list — keep in order; et al. truncation is the digest's job, not this skill's)
• Journal name (fulljournalname)
• Publication date (pubdate, sortpubdate)
• DOI (articleids[].idtype == 'doi')
• Abstract (separate efetch call with rettype=abstract since esummary doesn't return abstracts)

Abstract fetch is the slow part. Batch 100 PMIDs at a time. If abstract fetch fails for a record, keep it but with abstract: null and a warnings flag.

Step 5 — Normalize

Same canonical shape as fetch-preprint-recent:

{
  "id": "PMID:39123456",                                  // PMID-prefixed for source clarity
  "title": "...",
  "authors": ["Smith J", "Doe A", ...],
  "abstract": "...",
  "date": "2026-05-07",                                   // YYYY-MM-DD parsed from pubdate
  "server": "pubmed",
  "journal": "Nature Biotechnology",
  "doi": "10.1038/s41587-026-12345-6",                    // if present
  "url": "https://pubmed.ncbi.nlm.nih.gov/39123456/",
  "matched_keywords": ["protein language model"]
}

URL pattern: https://pubmed.ncbi.nlm.nih.gov/{pmid}/ — the canonical PubMed link. Optional secondary URL via DOI: https://doi.org/{doi} for the publisher's page.

Step 6 — Return

{
  "server": "pubmed",
  "transport_used": "mcp" | "eutils",
  "window": "2026-05-04/2026-05-10",
  "query": "(...full query string...)",
  "fetched_total": 47,
  "matched_total": 47,                       // PubMed filter is server-side, so fetched == matched
  "fetch_errors": [],
  "records": [ ... ]
}

Cache the raw search + metadata responses to .cache/{YYYY-WW}-pubmed.json.

Common Patterns

Pattern A — Standard weekly scan: PDAT-bounded query with all watchlist keywords OR'd together. The default.

Pattern B — Newly-indexed reach-back: when the user wants "papers PubMed indexed this week, even if published earlier" (useful for older preprints just appearing in PubMed), swap PDAT for EDAT. Document the choice in the digest.

Pattern C — High-precision MeSH-only search: when keyword matching is producing too much noise, ask the user for the exact MeSH headings they care about and search term[MeSH] only. Higher precision, lower recall.

Pattern D — Author-specific watch: outside this skill's default scope but easy — add (Smith J[Author] OR Doe A[Author]) as an additional OR clause to the keyword group.

Guardrails

1. Always include the date bound. A keyword-only PubMed query returns the entire PubMed history for the term. Forgetting PDAT is the bug that turns a 7-paper digest into a 70,000-paper data dump.
2. Don't auto-expand keywords to MeSH. PubMed's automatic-term-mapping silently expands "Alzheimer" to a MeSH tree; that's the right call for a clinician but the wrong call for a curated keyword digest. Quote keywords with [Title/Abstract] to disable expansion unless MeSH expansion is explicitly requested.
3. Don't trust the first abstract API call to succeed. efetch with rettype=abstract occasionally times out for batches > 100. On failure, halve the batch and retry; only after two full halvings do you give up on a record.
4. Don't strip the PMID: prefix from id. It distinguishes PubMed records from preprint DOIs at a glance — important during cross-source dedupe in the caller.
5. Don't dedupe against bioRxiv/medRxiv inside this skill. Cross-source dedupe is the caller's job.
6. Respect the rate limit. NCBI allows 3 requests/second without an API key, 10/second with one. Pace accordingly. If a key is in the env (NCBI_API_KEY), include it as &api_key=....
7. Don't claim a record is "in PubMed" just because esearch returned its PMID. Some PMIDs are MEDLINE-status Publisher (in-process, not yet fully indexed). Fine to include but note status if available.

Quick Reference

| Field | E-utilities (fallback) | MCP (preferred) | | -------------- | ------------------------------------------------------------------------ | ------------------------------------- | | Search | esearch.fcgi?db=pubmed&term={q}&retmode=json | search_articles(query=...) | | Metadata | esummary.fcgi?db=pubmed&id={pmids}&retmode=json | get_article_metadata(pmids=...) | | Abstract | efetch.fcgi?db=pubmed&id={pmids}&rettype=abstract&retmode=xml | included in metadata when MCP returns it | | Date filter | ("YYYY/MM/DD"[PDAT] : "YYYY/MM/DD"[PDAT]) | same — pass inside query string | | Page size | 200 (search), 100-200 (summary), 100 (abstract) | MCP server's choice | | Auth | optional api_key query param (3 → 10 rps) | per MCP server config | | Canonical URL | https://pubmed.ncbi.nlm.nih.gov/{pmid}/ | |