overthinkos/kubernetes

Kubernetes — Deploying OpenCharly Images to K8s Clusters

Overview

OpenCharly can deploy built images to a Kubernetes cluster by emitting a Kustomize base/ + overlays/ tree. The deployment schema stays target-agnostic — authors describe what the workload needs (kind, replicas, resources, exposure, storage, probes); a per-cluster cluster profile file supplies the K8s-specific knobs (storage class, ingress class, cert issuer, secret backend).

Every box runtime contract is baked into OCI labels at build time, so a K8s deploy is possible without access to charly.yml — the charly deploy from-box verb reads capabilities from the pushed image alone.

Quick reference

| Action | Command | Description | |---|---|---| | Add K8s deploy | charly deploy add <name> <ref> --target kubernetes | Read BoxConfig + deployment + cluster profile; emit .opencharly/k8s/<name>/ Kustomize tree | | Source-less deploy | charly deploy from-box <registry/name:tag> [name] --cluster <name> | Deploy from OCI labels only — no charly.yml needed (see Part F.10) | | Sync to cluster | charly deploy sync <name> | kubectl apply -k .opencharly/k8s/<name>/overlays/default | | Show generated manifests | charly deploy show <name> | kubectl kustomize … — see what would apply | | Delete K8s deploy | charly deploy del <name> | Remove overlay dir; base stays if other instances reference it |

The three-layer model

| Concern | Schema slot | OCI label home | |---|---|---| | Build — what goes INTO the image | box.build: (or legacy BoxConfig) | no (consumed at build) | | Capabilities — box runtime contract | box.capabilities: (or layer rollups) | yes — every field under ai.opencharly.* | | Deployment — how to run the image | charly.yml:deployments.<name> + ~/.config/charly/charly.yml overlay | no |

The completeness invariant: every exported field on BoxMetadata/Capabilities has a CapabilityLabelMap entry. A compile-time test enforces this — a new capability field without a label mapping fails the build. See charly/capabilities.go.

Deployment schema — target-agnostic fields

deployments:
  box:
    openclaw:
      target: kubernetes
      kind: service                   # service | daemon | batch | scheduled | oneshot
      replicas: 3
      restart: always                 # always | on-failure | never (honored on Pod/Job/CronJob)
      resources:
        cpu_request: "500m"
        memory_request: 512Mi
      security:
        memory_max: 2Gi               # → resources.limits.memory
        cpus: "1.5"                   # → resources.limits.cpu
      expose:
        host: openclaw.example.com
        path: /
        tls: true                     # → cert-manager annotation from cluster profile
      storage:
        - { name: data, size: 20Gi, class_hint: fast, access: single-writer }
      probes:
        liveness:  { http: { path: /healthz, port: 8080 } }
        readiness: { http: { path: /ready,   port: 8080 } }
      kubernetes:
        cluster: production           # pointer to ~/.config/charly/clusters/production.yaml
        namespace: apps               # optional override of cluster profile default
        patches: []                   # escape hatch: strategic / JSON6902 patches
        raw: []                       # escape hatch: paths to raw manifests included verbatim

Workload kind heuristic (inside charly/k8s_generate.go):

kind: service   + storage: []       → Deployment
kind: service   + storage: [...]    → StatefulSet  (auto volumeClaimTemplates)
kind: daemon                        → DaemonSet
kind: batch                         → Job
kind: scheduled (+schedule:)        → CronJob
kind: oneshot                       → Pod

Explicit override: kubernetes.workload: Deployment (rare — prefer kind:).

Cluster profile

One file per cluster. Lives at ~/.config/charly/clusters/<name>.yaml (per-user) or clusters/<name>.yaml (in-repo, discoverable via discover.clusters:). The only place cluster-specific K8s knobs live.

# ~/.config/charly/clusters/production.yaml
version: 1
kind: cluster-profile
name: production
kubeconfig_context: gke_prod_us-east1
admission_policy: restricted         # restricted | baseline | privileged
default_namespace: apps

storage:
  class_default: fast-ssd-retain
  class_fast: fast-ssd
  class_cheap: hdd-delete
  class_encrypted: fast-ssd-luks
  access_mode_default: ReadWriteOnce

ingress:
  enabled: true
  class: nginx
  cert_issuer: letsencrypt-prod      # cert-manager ClusterIssuer
  path_type_default: Prefix

secrets:
  backend: external-secrets          # external-secrets | sealed-secrets | raw
  store: vault-prod
  prefix: prod/

box:
  pull_policy: IfNotPresent
  pull_secrets: [regcred-prod]

pod_defaults:
  priority_class: standard
  tolerations: []
  node_selector: {}

defaults:
  labels: { managed-by: opencharly }

New cluster = write a new profile; zero deployment-spec changes.

Generator output

.opencharly/k8s/<deployment-name>/
├── base/
│   ├── kustomization.yaml           # commonLabels, resources: […]
│   ├── deployment.yaml              # or statefulset.yaml / daemonset.yaml / job.yaml / cronjob.yaml / pod.yaml
│   ├── service.yaml
│   ├── pvc-<name>.yaml              # per storage entry (Deployment/DaemonSet); StatefulSet uses volumeClaimTemplates
│   ├── ingress.yaml                 # when expose.host set AND cluster profile ingress.enabled
│   └── raw/                         # copied from kubernetes.raw:
└── overlays/
    └── <instance>/                  # "default" for bare name; "prod" for image/prod
        └── kustomization.yaml       # namespace override + patches from kubernetes.patches:

Apply: kubectl apply -k .opencharly/k8s/<name>/overlays/<instance> (or charly deploy sync <name>).

Source-less deploy — charly deploy from-box

Proves the self-contained image invariant: a deploy pipeline with no access to charly.yml can still produce a correct Kustomize tree.

# On a machine that doesn't have the source repo:
charly deploy from-box quay.io/myorg/openclaw:v2 openclaw \
    --target kubernetes --cluster production --namespace apps
# Reads: OCI labels (capabilities) + ~/.config/charly/clusters/production.yaml
#        + ~/.config/charly/charly.yml (if present, for per-machine overrides)
# Emits: .opencharly/k8s/openclaw/base/ + overlays/default/
charly deploy sync openclaw                   # kubectl apply -k ...

Relevant code

• charly/k8s_config.go — K8sDeployConfig + ClusterProfile + LoadClusterProfile
• charly/k8s_target.go — K8sDeployTarget (fourth DeployTarget alongside OCI / container / host)
• charly/k8s_generate.go — GenerateK8sKustomize + workload-kind heuristic + Ingress/PVC emission
• charly/k8s_deploy_from_image.go — DeployFromImage
• charly/capabilities.go — Capabilities (alias of BoxMetadata) + CapabilityLabelMap + completeness check

Related skills

• /charly-core:deploy — unified charly deploy add/del verb; K8s is one of three targets
• /charly-internals:capabilities — OCI label contract the K8s generator reads from
• /charly-internals:install-plan — shared IR across build + deploy targets