dpla/dpla-verify-and-notify
docs/ai-context/archive/cursor-skills/dpla-verify-and-notify/SKILL.md
Verify ingest outcomes and send failure or status notifications to Slack or [email protected]. Use when the user asks to verify the ingest, check if it succeeded, notify about a failure, or post to tech-alerts.
$ install --global
skillsauthnpx skillsauth add dpla/ingestion3 dpla-verify-and-notifyInstall this skill globally with one command. Works with Claude Code, Cursor, and Windsurf.
Security Scan Results
3 of 9 scanners reported clean
Some scanners were skipped, did not run, or reported a non-clean status. Review each row below.
3
Scanners Passed
9
Scanners in report
Clean
TrivyContainer and dependency vulnerability scanner
95%
Clean
SemgrepStatic code analysis for vulnerabilities
95%
Clean
mcp-scan (Snyk)Model Context Protocol security validation
95%
Skipped
Snyk (dep)Open source security scanning
50%
Skipped
Socket.devSupply chain security analysis
50%
Skipped
VirusTotalMulti-engine malware detection
50%
Skipped
CrowdStrikeAdvanced threat intelligence
50%
Skipped
OSV-ScannerOpen Source Vulnerability database check
50%
Skipped
OWASP Dep-Check
50%
Last scanned: May 26, 2026, 3:38 AM151.7s1 file scanned
SKILL.md
- name:
- dpla-verify-and-notify
- description:
- Verify ingest outcomes and send failure or status notifications to Slack or [email protected]. Use when the user asks to verify the ingest, check if it succeeded, notify about a failure, or post to tech-alerts.
DPLA Verify and Notify
Purpose
After a run or when something fails, verify pipeline output and ensure the right people are notified (Slack #tech-alerts or email [email protected]).
When to Use
- "Verify the ingest"
- "Check if it succeeded"
- "Notify about the failure"
- "Post to tech-alerts"
- "Send failure alert"
- "Who do we notify when ingest fails?"
Environment: For commands that run the orchestrator or scripts, ensure source .env has been run in the shell so JAVA_HOME, SLACK_WEBHOOK, etc. are set.
Verification
Pipeline success markers
Each step writes _SUCCESS when complete:
# Harvest
ls $DPLA_DATA/<hub>/harvest/<timestamped-dir>/_SUCCESS
# Mapping
ls $DPLA_DATA/<hub>/mapping/<timestamped-dir>/_SUCCESS
# Enrichment
ls $DPLA_DATA/<hub>/enrichment/<timestamped-dir>/_SUCCESS
# JSONL
ls $DPLA_DATA/<hub>/jsonl/<timestamped-dir>/_SUCCESS
Record counts
cat $DPLA_DATA/<hub>/harvest/<timestamped-dir>/_MANIFEST
cat $DPLA_DATA/<hub>/mapping/<timestamped-dir>/_SUMMARY
Escalation reports (orchestrator runs)
When the orchestrator has failures, it writes:
data/escalations/failures-<run_id>.md— human-readabledata/escalations/failures-<run_id>.json— machine-readable
ls data/escalations/
cat data/escalations/failures-*.md | tail -1
What to Report (failures)
| Stage | What to report | |-------|----------------| | Harvest | Feed unreachable, timeout, or harvest step failed | | Mapping/remap | Mapping or IngestRemap failed (non-zero exit, no output) | | Sync | S3 sync failed or was blocked | | Anomaly | Anomaly detection halted sync (critical threshold) |
Include: hub name, stage that failed, one-line error or path to logs/escalation report.
Where to Notify
- Slack #tech-alerts: Preferred when
SLACK_WEBHOOKis set. Post a short message: hub, stage, error or link to report. - Email [email protected]: Use when Slack is unavailable or as backup. Same content: hub, stage, error or attach/link failure report.
When using the orchestrator, it posts failure alerts automatically; when running scripts manually, you (or the agent) must post or email.
Test notifications
To verify webhooks without running an ingest:
source .env
./venv/bin/python -m scheduler.orchestrator.main --dry-run-notify
Messages are sent with a [TEST] prefix.
Reference
- Full notification policy: AGENTS.md (sections on notifications and #tech-alerts)
Related Skills
dpla/dpla-hub-info
data-ai
VerifiedTrustedCommunity
Show key i3.conf config for a hub (provider, harvest.type, harvest.endpoint, schedule, email, setlist). Use when user asks for hub config, harvest type/endpoint, who gets emails, schedule months, or OAI setlist details.
35SKILL.mdUpdated Apr 16, 2026
dpla/dpla-community-webs-ingest
development
VerifiedTrustedCommunity
Run Community Webs ingest. Use when the user says harvest community-webs, run community-webs ingest, export community webs, or process community webs DB.
35SKILL.mdUpdated Apr 15, 2026
dpla/dpla-staged-report
business
VerifiedTrustedCommunity
Report which hubs have new JSONL staged in S3 for a given month, and optionally post the report to Slack. Use when user asks what hubs are staged/ready for indexing, /ingest staged, or what changed this month in S3.
33SKILL.mdUpdated May 26, 2026
dpla/dpla-script-workflow
development
VerifiedTrustedCommunity
Follow the project workflow when adding or modifying shell or Python scripts. Use when the user asks to add a script, create a new script, modify a script, or write a script for a task. Ensures POSIX bash, common.sh, documentation in SCRIPTS.md, and tests are created or updated and run.
33SKILL.mdUpdated May 26, 2026