elbwalker/walkeros-create-transformer

Create a New Transformer

Prerequisites

Before starting, read these skills:

• understanding-flow - How transformers fit in architecture
• understanding-transformers - Transformer interface
• understanding-events - Event structure
• testing-strategy - How to test
• writing-documentation - Documentation standards (for Phase 7)

Transformer Categories

| Category | Purpose | Example | | ------------ | ----------------------------- | ---------------------------- | | Validate | Check event structure/content | JSON Schema, required fields | | Enrich | Add data to events | Server timestamps, geo data | | Redact | Remove/mask sensitive data | Strip PII, anonymize IPs |

Process Overview

1. Research     → Understand use case (validate/enrich/redact)
2. Examples     → Create event before/after examples FIRST
3. Scaffold     → Copy template, configure package.json
4. Convention   → Add walkerOS.json metadata and buildDev
5. Implement    → Build transformer with TDD
6. Test         → Verify against example transformations
7. Document     → Write README

Supporting Files

This skill includes reference files you can copy:

• examples/ - Example events and configs to adapt for your transformer
  • events.ts - Before/after event examples
  • config.ts - Configuration examples
• templates/validation/ - Complete redact transformer template
  • index.ts - Implementation with context pattern
  • types.ts - Type definitions
  • schemas.ts - Zod validation schemas
  • index.test.ts - Test suite with helpers

---

Phase 1: Research

Goal: Understand what the transformer needs to do.

1.1 Define Use Case

• [ ] Category: Validate, Enrich, or Redact?
• [ ] Input: What events will this process?
• [ ] Output: What should change? What should be blocked?
• [ ] Configuration: What settings does user need?

1.2 Check Existing Patterns

# Reference implementation
ls packages/transformers/fingerprint/

# Transformer types
cat packages/core/src/types/transformer.ts

Gate: Research Complete

• [ ] Category identified (validate/enrich/redact)
• [ ] Input/output transformation defined
• [ ] Configuration options listed

---

Phase 2: Create Examples (BEFORE Implementation)

Goal: Define event transformations in dev entry FIRST.

2.1 Scaffold Directory Structure

mkdir -p packages/transformers/[name]/src/{examples,schemas,types}

2.2 Create Event Examples

Adapt examples/events.ts for your transformer's use case. Each example file should include:

• Events that should pass through (modified)
• Expected output after processing
• Events that should be blocked (for validators)

2.3 Create Config Examples

Adapt examples/config.ts for your transformer's settings.

2.4 Step Examples

Add step examples with { in, out } pairs for end-to-end step testing:

// examples/step.ts
export const step = {
  'order-passes': {
    in: { name: 'order complete', data: { id: 'ORD-123' } },
    out: { name: 'order complete', data: { id: 'ORD-123' } },
  },
  'debug-filtered': {
    in: { name: 'debug test', data: { message: 'noise' } },
    out: false, // Transformer rejects this event
  },
};

For transformers, both in and out are walkerOS events, except out: false which indicates the transformer filters (rejects) the event. Set title + description for public examples; mark test-only fixtures with public: false. See using-step-examples for the Three Type Zones.

2.5 Export via dev.ts

export * as schemas from './schemas';
export * as examples from './examples';

Gate: Examples Valid

• [ ] All example files compile (npm run build)
• [ ] Can trace: input event → expected output for each example

---

Phase 3: Scaffold

Template transformer: packages/transformers/fingerprint/

cp -r packages/transformers/fingerprint packages/transformers/[name]
cd packages/transformers/[name]

# Update package.json: name, description, repository.directory

Directory structure:

packages/transformers/[name]/
├── src/
│   ├── index.ts           # Main export
│   ├── transformer.ts       # Transformer implementation
│   ├── index.test.ts      # Tests against examples
│   ├── dev.ts             # Exports schemas and examples
│   ├── examples/
│   │   ├── index.ts       # Re-exports
│   │   ├── events.ts      # Before/after event examples
│   │   └── config.ts      # Configuration examples
│   ├── schemas/
│   │   └── index.ts       # Zod schemas for settings
│   └── types/
│       └── index.ts       # Settings, Types interfaces
├── package.json
├── tsconfig.json
├── tsup.config.ts
├── jest.config.mjs
└── README.md

---

Phase 4: walkerOS.json Convention

Every walkerOS package ships a walkerOS.json file for CDN-based schema discovery.

Add walkerOS field to package.json

{
  "walkerOS": {
    "type": "transformer"
  },
  "keywords": ["walkerOS", "walkerOS-transformer", ...]
}

Use buildDev() in tsup.config.ts

Replace buildModules({ entry: ['src/dev.ts'] }) with buildDev():

import { buildDev } from '@walkeros/config/tsup';
// In defineConfig array:
buildDev(),

This auto-generates dist/walkerOS.json from your Zod schemas at build time.

Hints (Optional)

If your transformer has capabilities, behaviors, or troubleshooting patterns not obvious from schemas alone, add hints. See walkeros-writing-documentation skill for full guidelines.

Create src/hints.ts:

import type { Hint } from '@walkeros/core';

export const hints: Hint.Hints = {
  'validation-behavior': {
    text: 'Describes how validation works. See settings schema for options.',
    code: [{ lang: 'json', code: '{ "settings": { ... } }' }],
  },
};

Export from src/dev.ts:

export * as schemas from './schemas';
export * as examples from './examples';
export { hints } from './hints';

Guidelines:

• Expand awareness — describe capabilities ("supports X, Y, Z"), don't prescribe one path
• Reference schemas and examples, don't duplicate them
• Verify every claim against actual implementation before publishing
• Key naming: kebab-case, group with prefixes (validation-*, enrichment-*, troubleshoot-*)
• Most transformers don't need hints — schemas and examples cover the common case

Gate: Convention Met

• [ ] walkerOS field in package.json with type: "transformer"
• [ ] buildDev() in tsup.config.ts
• [ ] Build generates dist/walkerOS.json
• [ ] Keywords include walkerOS and walkerOS-transformer

Runtime-only npm dependencies

If your package wraps a third-party npm dep that cannot be ESM-bundled (uses __dirname, ships a .node binary, etc.), declare it under walkerOS.bundle.external in your package.json. See walkeros-using-cli → Bundle externals for the complete contract.

---

Phase 5: Implement

Now write code to transform examples as expected.

5.1 Define Types

See templates/validation/types.ts for the pattern. Define Settings and Types interfaces.

5.2 Implement Transformer (Context Pattern)

Transformers use the context pattern - they receive a single context object containing config, env, logger, id, and collector.

See templates/validation/index.ts for a complete implementation example.

Key patterns:

1. Context destructuring: Extract config, logger, id from init context
2. Schema validation: Use Zod schemas to validate settings and provide defaults (see templates/validation/schemas.ts)
3. Push receives pushContext: The push function gets event + push context
4. Return values: { event } (continue), void (passthrough), false (cancel)

5.3 Export

src/index.ts:

export { transformerRedact } from './transformer';
export type { Settings, Types } from './types';

Gate: Implementation Compiles

• [ ] npm run build passes
• [ ] npm run verify:touched -- <transformer-name> passes (L1: typecheck + lint + test)

---

Phase 6: Test Against Examples

Verify implementation produces expected outputs.

See templates/validation/index.test.ts for a complete test suite showing:

1. createTransformerContext() helper - Standardizes init context creation
2. createPushContext() helper - Standardizes push context creation
3. Examples for test data - Don't hardcode test values
4. Return value testing - Verify event, void, or false returns

Gate: Tests Pass

• [ ] npm run verify:touched -- <transformer-name> passes (L1)
• [ ] Tests verify against example outputs

---

Phase 7: Document

Follow the writing-documentation skill for:

• README structure and templates
• Quality checklist before publishing

Key requirements for transformer documentation:

• [ ] Use case description (validate/enrich/redact)
• [ ] Configuration options table
• [ ] Working code example with imports
• [ ] Installation instructions
• [ ] Link to website docs

---

Transformer-Specific Validation

Beyond understanding-development requirements (build, test, lint, no any):

• [ ] dev.ts exports schemas and examples
• [ ] Examples include before/after event pairs
• [ ] Return values handle all cases (event, void, false)
• [ ] Tests use examples for assertions (not hardcoded values)
• [ ] walkerOS.json generated at build time
• [ ] walkerOS field in package.json

---

Reference Files

| What | Where | | -------------- | ---------------------------------------- | | Template | packages/transformers/fingerprint/ | | Types | packages/core/src/types/transformer.ts | | Chaining logic | packages/collector/src/transformer.ts |

Related Skills

• walkeros-understanding-transformers - Transformer interface and chaining
• walkeros-testing-strategy - Testing patterns and env mocking
• walkeros-writing-documentation - Documentation standards