chrishuffman5/kubectl

kubectl Expert

This skill covers kubectl, the Kubernetes command-line tool. It has deep coverage of:

• Configuration (kubeconfig, contexts, namespaces, multiple clusters, auth)
• Output formats (default, wide, yaml, json, jsonpath, custom-columns, go-template)
• All major verbs (get, describe, create, apply, delete, edit, patch, exec, logs, port-forward, cp, top, explain, diff, rollout, scale, autoscale, drain, cordon, taint)
• Workload resources (pods, deployments, replicasets, statefulsets, daemonsets, jobs, cronjobs)
• Configuration and storage (configmaps, secrets, PVCs, PVs, storage classes)
• Networking (services, ingress, network policies, endpoints, DNS debugging)
• RBAC (roles, rolebindings, clusterroles, clusterrolebindings, service accounts, auth can-i)
• Node management (labels, taints, cordon, drain, top)
• Debugging (pod states, CrashLoopBackOff, ImagePullBackOff, OOMKilled, ephemeral containers, events)
• Scripting patterns (dry-run, diff, wait, jq integration, batch operations, kustomize)

How to Approach Tasks

When you receive a request:

1. Classify the request type:

   • Configuration/context -- Load references/core.md
   • Resource management -- Load references/commands.md
   • Debugging/troubleshooting -- Load references/debugging.md
   • Scripting/automation -- Load references/patterns.md

2. Identify scope -- Determine namespace. Remind user to check kubectl config current-context and default namespace.

3. Prefer declarative -- Use kubectl apply -f over imperative commands for production workloads. Use imperative for quick debugging.

4. Use dry-run for safety -- Preview changes with --dry-run=client -o yaml before applying.

5. Provide complete commands -- Include namespace flags (-n) when relevant.

Core Expertise

Configuration

kubectl config get-contexts                            # list contexts
kubectl config current-context                         # show current
kubectl config use-context my-cluster                  # switch context
kubectl config set-context --current --namespace=myapp # set default ns

# Multiple clusters
KUBECONFIG=~/.kube/config:~/.kube/other kubectl config view --merge --flatten > ~/.kube/merged

Output Formats

kubectl get pods -o wide                               # extra columns
kubectl get pods -o yaml                               # full YAML
kubectl get pods -o json                               # full JSON
kubectl get pods -o name                               # resource names only

# JSONPath
kubectl get pods -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.status.phase}{"\n"}{end}'
kubectl get pod my-pod -o jsonpath='{.spec.nodeName}'
kubectl get secret my-secret -o jsonpath='{.data.password}' | base64 --decode

# Custom columns
kubectl get pods -o custom-columns='NAME:.metadata.name,STATUS:.status.phase,NODE:.spec.nodeName'

# Go template
kubectl get pods -o go-template='{{range .items}}{{.metadata.name}}{{"\n"}}{{end}}'

Common Operations

# Pods
kubectl run nginx --image=nginx:1.25 --port=80
kubectl run -it --rm debug --image=busybox --restart=Never -- /bin/sh
kubectl get pods -A -o wide
kubectl describe pod my-pod
kubectl logs my-pod -f --tail=100
kubectl logs my-pod --previous
kubectl exec -it my-pod -- /bin/bash
kubectl delete pod my-pod

# Deployments
kubectl create deployment nginx --image=nginx:1.25 --replicas=3
kubectl set image deployment/my-app my-app=myapp:v2
kubectl rollout status deployment/my-app
kubectl rollout undo deployment/my-app
kubectl rollout restart deployment/my-app
kubectl scale deployment my-app --replicas=5

# Services
kubectl expose deployment my-app --port=80 --target-port=8080 --type=ClusterIP
kubectl port-forward service/my-app 8080:80

Debugging

kubectl describe pod my-pod                            # events + state
kubectl logs my-pod --previous                         # crashed container
kubectl get pod my-pod -o jsonpath='{.status.containerStatuses[0].lastState}'
kubectl debug my-pod -it --image=busybox --target=app  # ephemeral container
kubectl get events --sort-by=.lastTimestamp             # recent events
kubectl get events --field-selector=type=Warning       # warnings only

Common Pitfalls

1. Applying without checking diff Always run kubectl diff -f manifest.yaml before kubectl apply in production.

2. Not using --previous for crashed container logs When a container CrashLoopBackOff, kubectl logs shows current (empty) output. Use --previous.

3. Forgetting -n namespace flag Without -n, commands target the default namespace. Set a default with set-context --current --namespace=X.

4. Using kubectl delete instead of declarative removal Use kubectl delete -f manifest.yaml to match the apply workflow.

5. Draining nodes without --ignore-daemonsets DaemonSet pods cannot be evicted. Always pass --ignore-daemonsets to kubectl drain.

6. Exit code 137 misidentified Exit code 137 = OOMKilled (128 + SIGKILL 9). Increase memory limits, do not look for application bugs.

7. Not checking endpoints after service creation If kubectl get endpoints my-svc shows no endpoints, the service selector does not match any pod labels.

8. Ignoring resource requests/limits Pods without requests are best-effort and will be evicted first under memory pressure. Always set requests.

9. Using kubectl create in production create fails if the resource exists. Use apply for idempotent operations.

10. Not using --field-selector for events kubectl get events returns all events. Filter with --field-selector=involvedObject.name=my-pod.

Reference Files

• references/core.md -- kubeconfig, contexts, namespaces, output formats (jsonpath, custom-columns, go-template), core verbs. Read for configuration and output questions.
• references/commands.md -- Complete command reference: workloads, config/storage, networking, RBAC, node management. Read for specific resource commands.
• references/debugging.md -- Pod debugging, ImagePullBackOff/CrashLoopBackOff/OOMKilled diagnosis, network debugging, events, explain. Read when troubleshooting.
• references/patterns.md -- Scripting: dry-run, diff, wait, jq integration, batch operations, kustomize, multi-container patterns. Read for automation scripts.

Scripts

• scripts/01-cluster-health.sh -- Cluster overview: nodes, pod summary, resource usage, events
• scripts/02-app-debug.sh -- Application debugging: deployment, pods, logs, connectivity
• scripts/03-namespace-report.sh -- Per-namespace resource report with issue detection