mathematic-inc/migrate-to-earl

Migrate to Earl

Converts existing raw CLI/API calls in a codebase to Earl templates. Works one provider at a time to keep changes reviewable and reversible.

Scope Constraints (Read First)

These rules prevent context window exhaustion and partial-migration messes:

1. One provider per run. If multiple providers are found, ask which to migrate first.
2. Cap at 10 call sites. If more than 10 are found, ask which 10 to prioritize. Flag the rest with # TODO: migrate to earl comments and note them in the final report.
3. Commit checkpoint. Pause after template creation, before rewriting call sites. Let the user commit the template before the rewrite begins.
4. Flag complex pipelines. Multi-pipe commands (curl | jq | xargs) are not rewritten. Add # TODO: migrate to earl (complex pipeline — review manually) and move on.

---

Phase 1: Scan All Providers

Read references/cli-to-earl-mapping.md (or fetch from https://raw.githubusercontent.com/mathematic-inc/earl/main/skills/development/migrate-to-earl/references/cli-to-earl-mapping.md) for the full list of patterns and files to scan.

Grep across all relevant files for every pattern in the mapping table. Even on repeated invocations, always do a full scan first.

Present a summary before asking the user to pick a provider:

"Found API/CLI calls to:

• GitHub (12 sites in 4 files)
• Stripe (8 sites in 2 files)
• Slack (3 sites in 1 file)

Which provider would you like to migrate first?"

If only one provider is found, proceed with it directly.

Note the remaining providers in the report so the user knows what's left for follow-up runs.

---

Phase 2: Import Pre-built or Create Custom

Check if a pre-built template exists for the chosen provider (see references/cli-to-earl-mapping.md).

If pre-built exists:

Check earl templates list first — if the provider is already imported, skip the import command below and go directly to showing available commands.

earl templates import https://raw.githubusercontent.com/mathematic-inc/earl/main/examples/<provider>.hcl
earl templates list

Show the user which commands are available in the imported template.

If no pre-built exists:

Invoke create-template for the provider. The create-template skill includes its own human review, validation, and secrets steps — do not duplicate those steps here.

---

Phase 3: Map Call Sites

For each found call site, determine the corresponding Earl command:

earl templates list --json

Match each raw call to an Earl command:

• curl -X GET https://api.github.com/repos/... → earl call github.get_repo
• gh issue create ... → earl call github.create_issue
• stripe customers list → earl call stripe.list_customers

For calls that don't map to any existing command, invoke create-template to add it.

---

Phase 4: Set Secrets

Skip this phase only if Phase 2 took the "no pre-built" path and invoked create-template directly (i.e., no bare earl templates import was run). In that case, create-template already ran secrets setup (its own Phase 7 and Phase 8) and confirmed secrets are set. Proceed to Phase 5.

Do NOT skip if Phase 2 ran a bare earl templates import — even if Phase 3 also invoked create-template for an unmapped command. The Phase 3 create-template only set secrets for the new command it created, not for the Phase 2 pre-built import. Phase 4 must still run for the pre-built provider's secrets.

Check annotations.secrets in the template file for required secret keys. For pre-built imports, read the imported file at ~/.config/earl/templates/<provider>.hcl (macOS/Linux) or %APPDATA%\earl\templates\<provider>.hcl (Windows) to find them.

Print the checklist of required secrets for the imported templates:

Before replacing call sites, set the required secrets in your terminal:

  earl secrets set <provider>.<key>

Tell me when you're done and I'll verify before we proceed.

On macOS: First run of earl secrets set may show a system keychain access dialog. Click "Always Allow" to avoid repeated prompts.

After the user confirms, verify:

earl secrets list

Check that all required keys appear. Re-print missing ones if needed.

---

Phase 5: Verify Earl Commands Work

Run a test call for each mapped command before touching any call sites:

earl call --yes --json <provider>.<command> --<param> <representative_value>

Important: If a command has annotations.mode = "write", the test call will create/modify/ delete real data. If the template defines environments (check the environments block for valid names), use --env <name> to select a non-production environment for the test call. Otherwise, use read-only commands for verification where possible, or use a test/sandbox account. Warn the user before running any write-mode test calls.

If any call fails, resolve it (via troubleshoot-earl if needed) before proceeding.

---

Phase 6: Checkpoint — Confirm Before Rewriting

Show the user:

1. The list of call sites that will be rewritten
2. The Earl equivalents they'll be replaced with
3. The call sites flagged for manual review (complex pipelines)

Ask explicitly:

"I'm about to rewrite these X call sites. This will modify your source files. Should I proceed? (You should commit your current changes first if you haven't.)"

Do not rewrite anything until the user approves.

---

Phase 7: Replace Call Sites

For each approved call site, rewrite to the Earl equivalent:

# Before:
curl -H "Authorization: Bearer $GITHUB_TOKEN" https://api.github.com/repos/$OWNER/$REPO

# After:
earl call --yes --json github.get_repo --owner $OWNER --repo $REPO

For source files (Python, JavaScript, etc.): curl calls typically appear inside subprocess invocations — replace the subprocess call with the Earl equivalent for that language:

# Before (Python):
subprocess.run(["curl", "-H", f"Authorization: Bearer {token}", url])

# After (Python):
subprocess.run(["earl", "call", "--yes", "--json", "github.get_repo", "--owner", owner, "--repo", repo])

// Before (Node.js):
execSync(`curl -H "Authorization: Bearer ${token}" ${url}`);

// After (Node.js):
execSync(
  `earl call --yes --json github.get_repo --owner ${owner} --repo ${repo}`,
);

For flagged complex pipelines, add a comment but leave the original:

# TODO: migrate to earl (complex pipeline — review manually)
curl ... | jq ... | xargs ...

---

Phase 8: Final Validation

earl templates validate

---

Phase 9: Report

Summarize what was done:

• Which provider was migrated
• How many call sites were rewritten
• Which were flagged for manual review
• Which secrets were set
• What providers remain for follow-up runs

---

Next Steps

• To migrate another provider: invoke migrate-to-earl again
• To create a template for an unmatched service: invoke create-template
• To enforce Earl usage at the platform level: invoke secure-agent
• If a migrated call fails: invoke troubleshoot-earl