cutec-chris/skill-creator

Skill Creator

Create, edit, improve, or audit PawLia AgentSkills. Manage credentials.

A skill is a sub-agent with its own LLM session. The dispatcher reads the skill's description to decide whether to invoke it; the SkillRunner then loads the full SKILL.md body, injects credentials as env vars, and hands control to the sub-agent with bash + other tools.

Anatomy

skill-name/
├── SKILL.md        # required — frontmatter + instructions
├── scripts/        # optional — executable code (python/bash/node)
├── references/     # optional — docs the agent reads while working
├── assets/         # optional — templates/boilerplate used IN the output
└── harness.sh      # optional — smoke-test (also .py / .mjs); run via `creator.py test`

Three loading levels: frontmatter always in context (~100 tokens, triggers the skill) → SKILL.md body loaded when triggered (keep <500 lines) → bundled resources loaded on demand (scripts execute without entering context).

No README / CHANGELOG / test-suites / setup guides. Only files the agent needs.

workflow.yaml, if present, was LLM-compiled from SKILL.md — never hand-write it. Compile with creator.py compile --name <name> after substantive SKILL.md edits.

For design patterns, read references/patterns.md and references/design-principles.md.

---

Frontmatter

---
name: my-skill                    # required — lowercase+hyphens, matches folder
description: >                    # required — the dispatch trigger; include what AND when
  What the skill does. Use when [trigger phrases and contexts]. Triggers on
  phrases like "X", "Y", "Z".
license: MIT
metadata:
  author: Your Name
  version: "1.0"
  max_tool_turns: 30              # optional — overrides the default budget (30)
  requires_config:                # optional — NESTED under metadata
    - url                         # keys under skill-config.<name>.* in config.yaml
requires_credentials:             # optional — TOP-LEVEL (sibling to metadata)
  - my_api_key                    # each becomes CRED_MY_API_KEY at runtime
---

Placement matters — the loader reads requires_config from metadata, requires_credentials from top-level. Getting it wrong silently breaks the skill.

Description writing decides whether your skill triggers at all. Include both what and when, list trigger phrases, cover edge cases. Be slightly pushy — models tend to undertrigger. Put all "when to use" info here, never in the body (the body is invisible to the dispatcher).

Credentials vs. Config

• requires_credentials — per-user secrets (API keys, tokens). Stored in session/.credentials/<user_id>.json (sandboxed, outside the per-user session dir) via credentials.py. Injected at runtime as CRED_<NORMALIZED> where <NORMALIZED> is the key uppercased with non-alphanumerics → _. Example: api-key → CRED_API_KEY. Skill scripts read them from env — they must never cat the credential store.
• metadata.requires_config — deployment-level settings in config.yaml under skill-config.<name>.*. Skills with missing required config are not loaded. The runtime injects the full per-skill config as JSON in PAWLIA_SKILL_CONFIG. Scripts must read config from that env var instead of requiring the LLM to pass URLs, timeouts, hosts, or model names as CLI args. Compiled workflow placeholders such as {url} or {timeout} are also filled automatically from skill-config.<name> when present.

Runtime Environment

Scripts receive:

| Env var | Value | |---------|-------| | PAWLIA_SESSION_DIR | Absolute path to the session root | | PAWLIA_USER_ID | Current user ID | | PAWLIA_SKILL_CONFIG | JSON object from skill-config.<skill-name> | | CRED_<KEY> | Each credential declared in requires_credentials |

Placeholders in the SKILL.md body — substituted by the runner before the sub-agent sees them: <scripts_dir>, <user_id>, <session_dir>. Always reference scripts as <scripts_dir>/<name>, never relative paths.

Python scripts should use:

skill_config = json.loads(os.environ.get("PAWLIA_SKILL_CONFIG", "{}"))
url = skill_config.get("url")

Do not teach skills to pass config values around as ordinary model-generated arguments. The model should provide user intent (query, limit, project), while the system supplies deployment config.

Automation scripts (scheduled jobs)

When the task is a scheduled automation (built via the automation skill — thunderstorm alert, train-delay watch, morning digest), build a standalone script, not an LLM instruction. The scheduler runs it deterministically and delivers whatever it prints; printing nothing → the user hears nothing.

Use the harness pawlia.automation_harness (always importable inside a job):

#!/usr/bin/env python
from pawlia.automation_harness import get_params, emit, silent, llm_call, log

params = get_params()                 # the job's --params dict
# 1. Deterministic gate: decide whether there is anything to report.
if nothing_to_report:
    silent()                          # print nothing → no notification
else:
    # 2. Optional: curate/phrase with the LLM, only when needed.
    emit(llm_call("Fasse zusammen: ...") if needs_llm else "kurze Meldung")

Skeleton rules:

• Gate first, deterministically. The decision to notify is plain code, never an LLM call.
• emit() only when there is something to say. Empty/whitespace = silent.
• llm_call() sparingly — a monitor often needs it never; a digest needs it once.
• Fail loud: let exceptions propagate (non-zero exit); the scheduler surfaces failures.
• Write the script to workspace/skills/scripts/<name>.py — the primary path the scheduler resolves job scripts from.
• Test before registering: run it with a representative AUTOMATION_PARAMS and confirm the silent case prints nothing and the alert case prints exactly the message.

Filesystem rules — where a skill may write

Hard rule: a skill may only create or modify files under two roots.

| Write to | When | |----------|------| | $PAWLIA_SESSION_DIR/$PAWLIA_USER_ID/... (the workspace, Downloads/, etc.) | Files the user keeps — documents, results worth re-reading later. Reachable via the files skill. | | /tmp/... | Throwaway, generated artefacts — a rendered chart, a rain-radar PNG, an intermediate download. The default for anything ephemeral. Prefer a unique name (/tmp/<skill>_<something>.png). |

Everything else is forbidden and blocked: the session/ root (e.g. $PAWLIA_SESSION_DIR/radar — a common mistake), /app, $HOME, the skill's own bundled directory, or any other absolute path. At runtime the bash tool runs commands inside a sandbox with a read-only root, so such writes fail with a permission/read-only error. creator.py test enforces the same rule and fails the harness if the skill writes outside these roots — so a violation is caught at the latest during testing.

Tighter rule for skill-creator specifically. Skill-creator writes code, not user documents, so it tightens the above to a single subtree:

| Write to | When | |----------|------| | workspace/skills/<name>/ | New or changed skills (SKILL.md, scripts/, etc.) | | workspace/skills/scripts/<name>.py | Automation scripts (scheduled jobs) — the only place automation add-job --script resolves from |

No direct writes to the workspace root (e.g. workspace/foo.md for ad-hoc notes), no /tmp artefacts from skill-creator, no writes outside the workspace/skills/ subtree. If a build needs an intermediate artefact, do it in a sandboxed scratch dir inside the skill's own scripts/ and clean up.

Delivering a file to the user (image, PDF, GIF): write it to /tmp (or the workspace if it should be kept) and return its path in the JSON payload. Do not embed the bytes as a base64 data: URI in the response text — chat surfaces like Matrix render that as raw text, not an image. The dispatcher attaches the file via the attach_file tool, which accepts workspace and /tmp paths.

---

Credential Management

python <scripts_dir>/credentials.py set --key "<name>" --value "<val>"
python <scripts_dir>/credentials.py check --keys "a,b,c"
python <scripts_dir>/credentials.py list
python <scripts_dir>/credentials.py delete --key "<name>"

The store is located outside the bash-sandboxed per-user dir, so ordinary skill scripts cannot reach it. The CLI is the only legitimate write path; reads happen implicitly at runtime via CRED_* env vars.

After set, the response confirms {"success": true, "key": "<name>"} — no value is echoed back. Verify success by checking the returned key matches what you intended.

---

Creating a Skill

1. Understand intent. Ask for concrete examples: what should trigger it, what's the input/output, does it need credentials or config? One question at a time.

2. Plan resources. For each example, ask "what would be rewritten or re-discovered every time?" — that becomes scripts/, references/, or assets/.

3. Scaffold.

   python <scripts_dir>/creator.py init \
     --name "<name>" --description "<desc>" \
     [--resources scripts,references,assets] \
     [--credentials "k1,k2"] [--config "url,timeout"] \
     [--script python|node|bash]
   

4. Implement. Write scripts first, test each one by running it directly with the right env vars, then write the SKILL.md body that guides the sub-agent. SKILL.md is imperative ("Run the script", "Parse the output"), shows the exact output shape, lists error-recovery steps in a table, and references any references/ files with a note on when to read them.

   Scripts must: parse user-provided args via argparse (or equivalent), read deployment config from PAWLIA_SKILL_CONFIG, read credentials from CRED_*, output {"success": bool, ...} as JSON, exit 0 on success and non-zero on failure.

   Data vs. presentation — hard rule: Scripts output raw structured data (facts, numbers, lists, timestamps) in the JSON payload. They do NOT pre-format the final answer as a user-facing string. The LLM sub-agent is responsible for turning the data into a response: choosing what to highlight, applying Pawlia's tone, trimming noise, and structuring the text. A script that returns a pre-built wall of text locks out the LLM and makes the skill impossible to adjust conversationally.

   Exception: skills whose output is explicitly required to be verbatim (e.g. a pre-formatted report) MUST say so in the SKILL.md with a clear "Return verbatim" rule AND provide a ## Example output that shows the exact expected format including any links or special elements. Without the example, the sub-agent will helpfully reformat — and destroy links and structure.

   ## Example output — MANDATORY in every SKILL.md body. It must:

   • Show one realistic sample of what the final user-facing text looks like
   • Explicitly mark elements that must not be changed: links, special formatting, exact phrases — annotate them with a ← keep comment or bold
   • Be 5–20 lines; representative but not exhaustive
   • Be updated first whenever the user requests a change to output format — agree on the new example, then change the script/instructions to match it

5. Validate. creator.py validate --name "<name>" — fix all issues, review warnings.

6. Harness (recommended). Add harness.sh at the skill root (also .py or .mjs) that runs 1–3 read-only probes and prints one JSON line {"success": true, "checks": [...]}. Write-capable skills do a write-then-delete roundtrip or gate writes behind --write. Harness leaves no side effects.

   Run via creator.py test --name "<name>" — loads real credentials and env, prints full stdout/stderr (no truncation). See references/patterns.md § Harness for the skeleton.

7. Compile. creator.py compile --name "<name>" — LLM-compiles SKILL.md into workflow.yaml. Skipped if version matches; pass --force to override. The skill runs without it (fallback mode), but compiled is better.

8. Package (optional). creator.py package --name "<name>" produces a .skill zip.

9. Iterate. Use it on real tasks, notice struggles, update SKILL.md or scripts, bump metadata.version, re-compile.

---

Installing a Skill from a Zip

When the user sends a .skill zip or any zip containing a skill:

1. Extract to /tmp/ — never extract directly into any skill directory.
2. Read the SKILL.md from the extracted zip to determine name.
3. Create the target directory in the user's workspace:

   python <scripts_dir>/creator.py init --name "<name>" --resources scripts
   

   This creates the skill under workspace/skills/<name>/.
4. Copy files from /tmp/<extracted>/ into workspace/skills/<name>/, overwriting the scaffold files from init with the real ones from the zip.
5. Adapt the SKILL.md for PawLia (ensure <scripts_dir> placeholders, ## Example output section, PawLia-compatible frontmatter).
6. Remove the /tmp/ extraction when done.
7. Validate: creator.py validate --name "<name>".

If creator.py init fails with a "refusing to overwrite" error, pass --force (the existing directory is from a previous failed install, not user work).

---

Fixing / Auditing an Existing Skill

Three phases with hard stop-gates. Do not blur them.

Phase 0 — Early exit (1 tool call)

Before diagnosing, do a single targeted read or grep for the requested change. If the target file already contains the requested state, report "already correct — no changes needed" and stop immediately. Do not continue to Phase 1.

Phase 1 — Diagnose (≤5 tool calls)

Read the skill files once. Run the harness or reproduce the failing command. Capture the full error (status code + response body).

Skill scripts often wrap upstream errors into generic "HTTP 500 - server error" strings. If the output is too generic, the first fix is to the script's error branch — make it print the real status + body — before any further investigation.

Stop-gate: as soon as you have a concrete, actionable root cause (specific missing field, wrong endpoint, validation message), stop diagnosing. Do not fuzz parameter names or endpoint variants once you have a working signal.

Phase 2 — Implement (≤5 tool calls)

Edit the script. Update SKILL.md when the external contract changed (endpoint path, payload shape, auth flow) or when the user requested a change to the output format — in that case update ## Example output first to reflect the desired result, then adjust scripts and instructions to match it.

Rule: in Phase 2, no new probes. Every tool call must be write_file, edit_file, or a single targeted re-read of a file you are editing. If you feel the urge to probe again, you ended Phase 1 too early — go back and capture what you missed, then resume Phase 2 fresh.

If an external reference skill exists (e.g. fittrackee vs. sparkyfitness), read it for payload / auth patterns. That's allowed in Phase 2 — it's referencing, not probing.

Phase 3 — Verify (≤3 tool calls)

Run the harness (test) — or reproduce the original failing command. If the skill has discrete, scriptable commands (a "simple" skill — lookups, CRUD, status checks), you must also re-compile its workflow and re-test: run compile --name "<name>", then test --name "<name>". A workflow-backed skill is only fixed once both the script and the compiled workflow.yaml are green. Invoke every script in the SKILL.md as <scripts_dir>/<script> so the compiler emits a runnable {scripts_dir}/... command — never a hand-written or invented path placeholder.

What "green" means — hard rule. A passing test is NOT "the command exited 0". For any command you added or changed, capture its --json output and confirm both:

1. It contains a top-level "success": true.
2. Its payload fields carry real data for the command's purpose — not all null/[]/{}.

An exit code 0 with an all-null envelope (every result field null) or with no success field is RED, period. Do not declare "alle Tests laufen einwandfrei" after eyeballing null output — that ships a broken command. Re-read the command's wiring (is its result actually written into the output envelope? is success set?) and loop back to Phase 2.

Commit atomically when green. Workspace sync (syncthing) handles cross-host propagation, so no local git commit is required from the sub-agent. Just make sure your own work is in a clean state the moment Phase 3 is green — half-finished edits may otherwise get picked up. Leave the workspace either fully green or rolled back — never broken and uncommitted across a pause.

Green → done, report a short summary to the user. Red → one loop back to Phase 2, same budget. Never go back to Phase 1 from here.

Failure exit

After 2–3 failed fix attempts → stop and report. Include: full error from Phase 1, what you changed in Phase 2, what still fails. Do not keep looping — it burns context without progress.

---

Commands

| Command | Script | What it does | |---------|--------|-------------| | init | creator.py | Scaffold a new skill | | validate | creator.py | Check SKILL.md for errors | | list | creator.py | Show all skills (workspace + bundled) | | test | creator.py | Run the skill's harness with real credentials/env | | compile | creator.py | LLM-compile SKILL.md → workflow.yaml | | package | creator.py | Create .skill zip | | implement | creator.py | Generate scripts via the in-process coding LLM | | fix | creator.py | Debug and fix a broken script via the in-process coding LLM | | set / list / delete / check | credentials.py | Manage credentials |

---

Implement (in-process coding LLM)

Use when a skill has been scaffolded (init) and the SKILL.md describes what the scripts should do, but the actual code still needs to be written — or when existing scripts need a substantial rewrite.

python <scripts_dir>/creator.py implement --name "<name>" --task "<what to implement>"

If --task is omitted, the LLM implements all scripts described in SKILL.md.

Coding runs in-process through the coder agent from agents.coder in config.yaml (falls back to agents.default and then to the first defined model). Set agents.coder: <model-key> to choose the model.

After implement, run validate and compile separately.

Fix (in-process coding LLM)

Use when a skill's script fails with a specific error. The LLM receives the failing command, error output, and the full skill context, then edits the script in-place.

python <scripts_dir>/creator.py fix --name "<name>" --error "<error message>" --command "<failing command>"

After fix, re-compile the workflow (compile) when the skill is workflow-suitable, then run the harness (test) to verify the fix worked.