overthinkos/migrate

ov migrate -- Schema migration commands

One-shot, idempotent migrators for every hard-cutover schema change the project has shipped. All ov migrate <name> commands follow the same contract (see /ov-dev:cutover-policy): running them twice is a no-op, running them once fully transforms the legacy surface into the new one. Legacy forms raise hard load-time errors at runtime — LoadUnified / LoadConfig errors point at the relevant ov migrate sub-verb.

Sub-verbs

| Sub-verb | Purpose | Skills covering the produced schema | |---|---|---| | ov migrate unified | Legacy image.yml + build.yml + flat-form layer.yml → overthink.yml with kind-keyed wrappers + includes: | /ov-build:layer, /ov-build:image | | ov migrate vm-spec | Legacy image.bootc: true + image.vm: {...} + image.libvirt: [...] + layer-level libvirt: → vms.yml kind: vm entities | /ov-vms:vms, /ov-dev:vm-spec | | ov migrate merge-vms | Separate vms.yml → deploy.yml top-level vm: key; rename vms: → vm:, arch-cloud-base → arch; bump schema v1 → v2 | /ov-vms:vms | | ov migrate deploy-schema-v-3 | Schema v2 → v3: rename vm:<name> deploy keys → <name>-vm, normalize target: container → pod, target: kubernetes → k8s, rename vm_source: → vm:, bump version: 2 → 3. Idempotent (second run = no-op). | /ov-core:deploy, /ov-advanced:vm, /ov-dev:disposable | | ov migrate shell-schema | Convert legacy cmd: shell-rc heredoc tasks (matching the # overthink:begin direnv-hook / # overthink:begin ssh-auth-sock fence patterns) into the structured shell: schema. Idempotent. Distinguishes install-style heredocs (cat >) from cleanup-style strips (sed -i) — only rewrites the former. 2026-05 cutover. | /ov-build:layer, /ov-coder:direnv | | ov migrate ov-cachyos | Rename the operator-specific CachyOS deployment to its 2026-05 canonical form ov-cachyos. Collapses the qc → cachyos-dx → ov-cachyos chain into a single hop — handles BOTH legacy keys (qc, cachyos-dx) AND moves the matching kind:local template name. Walks overthink.yml and ~/.config/ov/deploy.yml. Idempotent; line-oriented edits preserve comments. Residual deploy.qc, deploy.cachyos-dx, local.cachyos-dx raise hard load-time errors all pointing at this command. Demonstrates the cross-kind name reuse policy (CLAUDE.md): the kind:local template and the kind:deploy entry that applies it share one name. | /ov-build:local-spec, /ov-core:deploy | | ov migrate kind-files | Per-kind file split + kind: deployment → kind: deploy rename in one combined idempotent hop. (a) Extracts inline image: and vm: maps from overthink.yml into sibling image.yml / vm.yml. (b) Creates empty pod.yml / k8s.yml stubs if absent. (c) Appends each new file to overthink.yml's includes: list (preserving any existing entries). (d) Renames the root-key deployment: → deploy: in deploy.yml. (e) Walks every reachable YAML doc and renames kind: deployment → kind: deploy in place. Re-runs are no-ops. Implementation: ov/migrate_kind_files.go. Pairs with hard load-time errors in unified.go for residual kind: deployment docs and root deployment: keys, both pointing at this command. | /ov-build:image, /ov-build:local-spec, /ov-core:deploy, /ov-build:validate | | ov migrate local-deploy | Migrate the per-host file ~/.config/ov/deploy.yml from the pre-2026-04 legacy schema (top-level images: map + per-entry bind_mounts: field with path:/encrypted: + workspace: scalar) to schema v4 (deploy: map + per-entry volumes: with type: encrypted / type: bind). The workspace: scalar is promoted to a volumes: entry of type: bind with host: <legacy-value> + path: /workspace. target: pod is added to every entry (legacy schema only supported container deploys). Idempotent: re-running on a v4 file is a no-op. Writes a <file>.bak.<unix-ts> rollback before rewriting. Implementation: ov/migrate_local_deploy.go. Pairs with a hard load-time error in LoadDeployConfig (ov/deploy.go:hasLegacyImagesKey) that fires on any residual images: root key, pointing at this command. | /ov-core:deploy | | ov migrate quadlets | Walk ~/.config/containers/systemd/ov-*.container and regenerate any quadlet whose deploy declares type: encrypted volumes but whose on-disk unit lacks the ExecStartPre=ov config mount <image> auto-mount hook (added 2026-04-16). Pre-cutover quadlets silently boot containers against empty plain/ FUSE mountpoints whenever gocryptfs is unmounted — the actual root cause of the 2026-04-18 immich incident. Detection is INI-tolerant (matches /usr/bin/ov / bare ov / ~/.local/bin/ov); regeneration shells out to ov config <image> (kdbx prompt may fire on first regen, then cache-warms for siblings). Idempotent. Implementation: ov/migrate_quadlets.go. Pairs with the verifyBindMounts cipher-populated-plain-empty discrimination in /ov-advanced:enc. | /ov-core:deploy, /ov-advanced:enc, /ov-core:start |

ov migrate local-deploy

Invoked as ov migrate local-deploy. Converts the per-host deploy file ~/.config/ov/deploy.yml from the pre-2026-04 legacy schema to schema v4.

Why this exists

The 2026-04 unified-config cutover renamed the top-level images: map to deploy: and replaced the per-entry bind_mounts: field with a structured volumes: list that distinguishes type: encrypted / type: bind / type: volume. yaml.Unmarshal silently drops unknown root keys, so a pre-cutover file with images: parses to an empty DeployConfig.Deploy map — and downstream commands behave as if nothing was deployed. The most dangerous symptom: encrypted-volume entries declared under the legacy bind_mounts: [{encrypted: true}] shape are invisible to loadEncryptedVolumes, so the encryption guarantee silently disappears (the immich-recovery session that motivated this fix found a 2-week-5-day window of plaintext data accumulation against an unmounted gocryptfs vault). This migration closes that gap.

What it converts

| Legacy field | → | Modern field | |---|---|---| | images: (root) | → | deploy: (root) | | (none) | → | version: 4 (added if absent) | | (none) | → | target: pod (per entry) | | images.<name>.bind_mounts: [{name, path, encrypted: true}] | → | deploy.<name>.volumes: [{name, type: encrypted}] (path dropped — encrypted volumes derive their host path from encrypted_storage_path/ov-<image>-<name>) | | images.<name>.bind_mounts: [{name, path}] (plain) | → | deploy.<name>.volumes: [{name, type: bind, path}] | | images.<name>.workspace: <host-path> (scalar) | → | deploy.<name>.volumes: [{name: workspace, type: bind, host: <host-path>, path: /workspace}] | | images.<name>.tunnel, dns, ports, env_file, security, network | → | passed through verbatim under deploy.<name>.* |

Idempotency

Running on a v4 file (top-level deploy: and no legacy images:) is a no-op — exit 0, message nothing to migrate (already on schema v4), no backup written. Detection uses hasLegacyImagesKey(data) on the raw YAML body (a yaml.v3 Node walk on root-level mapping keys), so it correctly ignores nested images: keys that legitimately appear inside test fixtures or comment text under modern-schema entries.

Backup

Before rewriting, <file>.bak.<unix-timestamp> is written with the original content (mode 0600). Match the plaintext-secret migration pattern in ov/config_secret_migration.go. There is no automatic cleanup of old backups — the operator deletes them when satisfied with the migration.

Flags

• --dry-run — print the proposed transformation summary, leave the filesystem untouched, no backup written.
• --path <file> — override the default ~/.config/ov/deploy.yml (used by tests and for migrating saved snapshots from a different machine).

Typical flow

ov migrate local-deploy --dry-run                          # preview
ov migrate local-deploy                                    # apply
ov deploy show <name>                                      # confirm load works
ls ~/.config/ov/deploy.yml.bak.*                           # rollback file present

Hard load-time error

When a legacy file is present and the user runs ANY ov command that reads ~/.config/ov/deploy.yml (ov status, ov deploy show, ov config status, ov start, …), LoadDeployConfig returns:

deploy.yml at <path>: legacy top-level `images:` field detected — run `ov migrate local-deploy` to convert; the field was renamed to `deploy:` in the 2026-04 unified-config cutover (encryption guarantees disappear silently otherwise)

ov status surfaces this as a non-fatal warning (graceful degradation falls back to image-label-driven display); the strictly-deploy.yml-driven verbs (ov deploy show, ov config status) hard-fail.

ov migrate quadlets

Invoked as ov migrate quadlets. Walks the per-host quadlet directory (~/.config/containers/systemd/ov-*.container) and regenerates any unit whose deploy declares type: encrypted volumes but whose on-disk quadlet lacks the ExecStartPre=ov config mount <image> auto-mount hook.

Why this exists

The auto-mount hook was added 2026-04-16. Pre-cutover quadlets that include encrypted-volume bindings (Volume=<cipher-dir>/plain:<container-path>) silently boot containers against empty plain/ FUSE mountpoints whenever gocryptfs has been unmounted (host reboot, manual fusermount3 -u, scope-unit crash). The container's services then write plaintext data into plain/ on top of the populated cipher tree — the encryption guarantee silently disappears. This is the actual root cause of the 2026-04-18 immich incident (2 weeks 5 days of plaintext data accumulated on top of the original encrypted vault before anyone noticed).

The fix has TWO components:

1. ov migrate quadlets (this command) regenerates the on-disk quadlet so future restarts go through the auto-mount hook.
2. verifyBindMounts cipher-populated-plain-empty discrimination (ov/enc.go, see /ov-advanced:enc "Pre-start safety check") catches the dangerous state in the direct-mode CLI path before the container is started.

Detection

For each entry in ~/.config/ov/deploy.yml deploy: map:

• Skip if target is anything other than empty / pod / container (only container-class deploys have quadlets).
• Skip if no volume entry has type: encrypted (the hook is encryption-specific).
• Skip if the on-disk quadlet at ~/.config/containers/systemd/ov-<name>.container does not exist (the user hasn't run ov config <name> yet for this deploy — nothing to migrate, stay quiet).
• For each remaining entry: scan the quadlet body for an ExecStartPre=…ov config mount <name> line. Tolerant to ov-binary path variations (/usr/bin/ov, ~/.local/bin/ov, bare ov) and trailing flags. If the hook is missing, the entry is stale.

Regeneration

Stale quadlets are regenerated by re-invoking the running ov binary as ov config <name> (self-exec via os.Args[0], the same pattern documented in /ov-dev:go "Self-exec coordination"). This re-runs the entire ov config codepath — quadlet write + secret provisioning + encrypted-volume init + data seeding + daemon-reload — so every concomitant state stays in sync. The kdbx prompt will fire on the first regeneration; subsequent ones in the same migration cache-hit via the kernel-keyring cache (see /ov-build:secrets "Password Caching").

Flags

• --dry-run — list stale quadlets and their regeneration commands without modifying anything.

Idempotency

A quadlet that already carries the hook is silently skipped. Running ov migrate quadlets twice produces the same output as running it once on a fully-migrated tree:

ov migrate quadlets: nothing to migrate (all encrypted-volume quadlets carry ExecStartPre=ov config mount …)

Tests

ov/migrate_quadlets_test.go:

• TestQuadletHasMountHook — 8 sub-cases covering hook presence/absence/prefix-collision/path-variation.
• TestDetectStaleEncryptedQuadlets — full scratch-deploy.yml + scratch-quadlet-dir end-to-end: encrypted-needing-migration / encrypted-already-migrated / non-encrypted-irrelevant. Asserts only the stale entry is returned.
• TestDetectStaleEncryptedQuadlets_NoQuadletOnDisk — encrypted deploy with no quadlet file → empty result, no spurious "missing" warnings.
• TestCipherPopulatedPlainEmpty (in the same file) — discrimination helper used by verifyBindMounts; 5 sub-cases.

ov migrate kind-files

Invoked as ov migrate kind-files. Combined idempotent migration that performs the 2026-05-XX per-kind file split AND the kind: deployment → kind: deploy schema-key rename in a single atomic hop.

What it does

1. Extract inline kind maps from overthink.yml. If overthink.yml carries an inline image: map, write the entries to a sibling image.yml; ditto for an inline vm: map → vm.yml. Existing per-kind files are left untouched.
2. Create stubs for missing per-kind files. Empty pod.yml and k8s.yml files are created if absent (the recommended layout has all six per-kind files as siblings of overthink.yml regardless of whether they have content yet).
3. Re-wire includes:. Each newly created file is appended to overthink.yml's includes: list, preserving any existing entries and any existing comments.
4. Rename root key deployment: → deploy: in deploy.yml. Line-oriented edit, comments preserved.
5. Rename kind: deployment → kind: deploy in every reachable YAML doc. Walks overthink.yml plus every file in its includes: and rewrites the kind discriminator.

Re-runs are no-ops: every step is idempotent (extractions are gated on inline-map presence, stub creation is gated on file absence, includes: entries are dedup'd, root-key and kind renames are gated on the legacy form being present).

Implementation

Code lives in ov/migrate_kind_files.go. Pairs with hard load-time errors in ov/unified.go (and the validator) that fire on any residual kind: deployment doc OR any root-key deployment: map — each error points the operator at ov migrate kind-files.

Why combined

The file-split and the kind rename land together because the filename and the kind name now match by convention (kind: deploy lives in deploy.yml, kind: image in image.yml, etc.) — splitting into per-kind files without renaming the discriminator would leave kind: deployment docs in deploy.yml, contradicting the convention immediately. R3 (no duplication) demands one migration command for the one cutover.

ov migrate unified

Invoked as ov migrate unified. One-shot converter from legacy scattered config (image.yml + build.yml + per-layer flat layer.yml) into the canonical unified overthink.yml format.

The new format is what every other ov command reads (LoadUnified in ov/unified.go). See /ov-build:layer (authoring reference) and /ov-build:image (umbrella) for the format documentation itself.

Quick Reference

| Action | Command | Description | |--------|---------|-------------| | Convert project | ov migrate unified | Emit overthink.yml (via includes:) next to existing legacy files. Non-destructive by default. | | Convert + rewrite layers | ov migrate unified --rewrite-layers | Also rewrite every layers/<name>/layer.yml into kind-keyed form (layer: {...}). | | Monolithic output | ov migrate unified --monolithic | Emit a single flat overthink.yml instead of using includes: to reference existing files. | | Preview | ov migrate unified --dry-run | Print what would be written; touch nothing. Combines with --rewrite-layers and --monolithic. |

What it converts

1. Project structure

image.yml + build.yml at the repo root → overthink.yml with includes: referencing them (or a monolithic flattened body with --monolithic). Each entry becomes kind-keyed: build:, image:, layer: — matching the types that LoadUnified expects.

2. Flat-form layer.yml → kind-keyed layer: wrapper

Legacy flat form:

name: redis
depends: [base]
packages: [redis]

New form (required — the flat form is rejected by parseLayerYAML after migration with a hard error pointing to --rewrite-layers):

layer:
  name: redis
  requires: [base]
  packages: [redis]

Run with --rewrite-layers to convert every layers/*/layer.yml in the project in one pass.

3. Raw-INI service: → structured list

Legacy supervisord-style scalar:

service: |
  [program:redis]
  command=/usr/bin/redis-server
  autostart=true
  startretries=3

Becomes the structured form with all fidelity preserved (including the 8 supervisord directives added to ServiceEntry in 2026-04 — kind, events, auto_start, start_retries, start_secs, stop_signal, exit_codes, priority):

service:
  - name: redis
    exec: "/usr/bin/redis-server"
    auto_start: true
    start_retries: 3
    restart: always

[eventlistener:...] blocks are recognized and emitted with kind: eventlistener + events: + the original command: mapped to exec:. The rewrite is driven by rewriteServiceKeys in ov/migrate_unified.go:397.

4. services: (plural) → service: (singular)

A pure field rename. The unified schema uses the singular service: key. External layer repos authored against the plural form are transparently fixed.

5. system_services: [foo, bar] → structured entries

# Before
system_services:
  - sshd
  - cockpit

# After
service:
  - name: sshd
    use_packaged: sshd.service
  - name: cockpit
    use_packaged: cockpit.service

use_packaged: tells the generator to reuse the distro-shipped systemd unit rather than render a new one (see /ov-build:layer "Service Declaration" and /ov-dev:capabilities for how this flows through the init-system schema in build.yml).

Idempotency

Running ov migrate unified --rewrite-layers twice produces byte-identical output (enforced by TestMigrateUnified_IncludesSplit and TestMigrateUnified_Monolithic in ov/migrate_unified_test.go). CI can safely re-run the migrator as a guardrail without churn.

Auto-invocation on remote-cache downloads

ov/refs.go:164 invokes MigrateUnified(... RewriteLayers: true) automatically whenever a remote @host/org/repo:version include lands in the repo cache (~/.cache/ov/repos/). This means external projects that still ship the legacy layout pull-through cleanly — no manual step required before the local build can consume the remote layers. See /ov-build:pull for the remote-ref lifecycle.

Downside: the cache directory silently gets rewritten on first fetch. If you want to inspect a remote project's on-disk layout untouched, clone it directly instead of relying on the cache.

Typical flow

# Inspect what will change
ov migrate unified --rewrite-layers --dry-run

# Apply the rewrite
ov migrate unified --rewrite-layers

# Confirm the build still works
ov image validate
ov image build <image>

After a successful migration, the project's legacy image.yml / build.yml stay in place (referenced via includes: in the new overthink.yml). To fully collapse into a single file, re-run with --monolithic and then remove the originals.

---

ov migrate vm-spec

Invoked as ov migrate vm-spec. One-shot converter from the legacy per-image VM fields (image.bootc: true + image.vm: {...} + image.libvirt: [...] + layer-level libvirt: snippets) into the post-cutover kind: vm entity format in vms.yml. See /ov-vms:vms for the produced schema and /ov-dev:vm-spec for the Go types.

Quick Reference

| Action | Command | Description | |--------|---------|-------------| | Convert project | ov migrate vm-spec | Harvest legacy VM fields into vms.yml. Idempotent; preserves pre-existing vms: keys. | | Preview | ov migrate vm-spec --dry-run | Print what would be written to vms.yml; touch nothing. | | Target file override | ov migrate vm-spec --output vms-legacy.yml | Write to a different file instead of vms.yml. |

What it converts

Per-image fields (kind:image → kind:vm)

| Legacy location | → | New location (vms.yml) | |---|---|---| | images.<name>.bootc: true | → | vms.<name>.source.kind: bootc + source.image: <name> | | images.<name>.vm.disk_size | → | vms.<name>.disk_size | | images.<name>.vm.ram, .cpus | → | vms.<name>.ram, .cpus | | images.<name>.vm.rootfs, .root_size, .kernel_args | → | vms.<name>.source.rootfs, .root_size, .kernel_args | | images.<name>.vm.ssh_port | → | vms.<name>.ssh.port | | images.<name>.vm.firmware | → | vms.<name>.firmware | | images.<name>.vm.network (string) | → | vms.<name>.network.mode | | images.<name>.vm.transport | → | vms.<name>.source.transport |

Layer-level libvirt snippets

Layer-level libvirt: ["<xml>", ...] on the contributing layer is preserved (still supported post-cutover — that's where /ov-foundation:qemu-guest-agent contributes its virtio-serial channel, for example). Image-level libvirt: [...] on the kind: image entry is deleted — it had no home in the new VM model. The migrator does NOT move image-level libvirt into the VM entity automatically; it emits a warning listing each deleted snippet with a suggestion to re-home it on the produced vms.<name>.libvirt.snippets: if still needed.

Naming convention

Bootc VMs pair 1:1 with their container image. The migrator names produced entries <image-name>-bootc or <image-name> depending on whether the image name already ends in -bootc:

• image: aurora (bootc: true) → vms: aurora-bootc
• image: selkies-desktop-bootc (bootc: true) → vms: selkies-desktop-bootc-bootc (doubled suffix distinguishes VM entity from container image)

Preservation semantics

Never clobbers pre-existing vms: keys. If vms.aurora-bootc: already exists in vms.yml, the migrator skips emission for that key and prints a notice. Lets authors hand-customize entries (adding libvirt.devices.*, cloud_init.packages, etc.) without losing work on re-run.

Idempotent. Running twice produces byte-identical vms.yml (enforced by test fixtures in ov/migrate_vm_spec_test.go).

Auto-invocation

Like ov migrate unified, vm-spec is auto-invoked during remote-cache downloads when ov/refs.go detects legacy VM fields in a fetched @github.com/org/repo:version include. External repos on the old schema pull through cleanly without a manual migration step.

Typical flow

# Inspect what will change
ov migrate vm-spec --dry-run

# Apply the migration
ov migrate vm-spec

# Include the new file from overthink.yml
# (edit overthink.yml to add `vms.yml` under `includes:`)

# Confirm VM builds still work
ov vm build <name>
ov vm create <name>

Post-migration load errors

Once the legacy fields are gone from the schema, old projects loading under the new ov binary get hard load errors pointing at this migration:

Error: image entry "foo" declares legacy field "bootc: true".
Run: ov migrate vm-spec

Remediation hint points at the command directly — no docs reading required.

---

See Also

• /ov-build:layer — authoring reference for the layer: kind-keyed schema and the full service: / 22-field ServiceEntry
• /ov-build:image — umbrella skill for image: entries and ov image build/validate/inspect
• /ov-build:build — build.yml vocabulary (distros, builders, init-systems) that overthink.yml references
• /ov-build:pull — remote @... refs and the auto-migration hook
• /ov-advanced:vm — ov vm command family; reads the vms.yml produced by migrate vm-spec
• /ov-vms:vms — authoring reference for the kind: vm entities produced by migrate vm-spec
• /ov-dev:install-plan — the IR that the loader feeds into the build/deploy pipelines
• /ov-dev:capabilities — OCI label contract (LabelServices) that consumes the migrated service list
• /ov-dev:vm-spec — Go types produced by migrate vm-spec
• /ov-dev:cutover-policy — policy governing why hard-cutover + idempotent migrator is the required shape
• /ov-dev:go — loader internals (LoadUnified, parseLayerYAML, MigrateUnified, MigrateVmSpec)