MARGA by EknathaLabs

Kubernetes administration and operations cheat sheet

etcd backup, upgrades, node maintenance, RBAC, quotas, policies and a cluster debugging checklist.

etcd snapshot

ETCDCTL_API=3 etcdctl snapshot save /backup/etcd.db \
  --endpoints=https://127.0.0.1:2379 \
  --cacert=/etc/kubernetes/pki/etcd/ca.crt \
  --cert=/etc/kubernetes/pki/etcd/server.crt \
  --key=/etc/kubernetes/pki/etcd/server.key
etcdutl snapshot status /backup/etcd.db -w table

Upgrade with kubeadm

kubeadm upgrade plan
kubeadm upgrade apply v1.NN.X
kubeadm upgrade node
kubeadm certs check-expiration

Node maintenance

kubectl cordon node-1
kubectl drain node-1 --ignore-daemonsets --delete-emptydir-data
kubectl uncordon node-1
kubectl taint nodes gpu-1 gpu=true:NoSchedule

RBAC

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: read-pods
  namespace: shop
subjects:
  - kind: Group
    name: shop-developers
    apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: pod-reader
  apiGroup: rbac.authorization.k8s.io

Check permissions

kubectl auth can-i delete pods -n shop --as=jane
kubectl auth can-i --list -n shop --as=system:serviceaccount:shop:web

Quotas and autoscaling

apiVersion: v1
kind: ResourceQuota
metadata:
  name: team-a
  namespace: team-a
spec:
  hard:
    requests.cpu: "20"
    requests.memory: 64Gi
    pods: "100"

Default deny network policy

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny
  namespace: shop
spec:
  podSelector: {}
  policyTypes: ["Ingress", "Egress"]

Debugging order

kubectl describe pod NAME
kubectl get nodes
kubectl describe node NODE
journalctl -u kubelet -n 100
kubectl get --raw=/readyz?verbose

Official references

Read the vendor and project documentation for the full, current detail.