1. kubectl get 2. kubectl describe 3. kubectl logs 4. kubectl exec 5. kubectl get events
This solves a large percentage of troubleshooting questions.
Think:
Scheduler Resources Taints PVC Affinity NodeSelector
Commands:
kubectl describe pod <pod> kubectl get nodes kubectl get events --sort-by='.lastTimestamp'
Look for:
Think:
Application problem Container starts then crashes
Commands:
kubectl describe pod <pod> kubectl logs <pod> kubectl logs <pod> --previous
Look for:
Think:
Image Name Tag Registry imagePullSecrets
Commands:
kubectl describe pod <pod>
Look in Events:
Common Causes:
Think:
Endpoints Selectors Pod Labels
Commands:
kubectl get endpoints <svc> kubectl describe svc <svc> kubectl get pods --show-labels
Most Common Cause:
Selector does not match Pod labels
Think:
PV exists? StorageClass? Capacity? AccessModes?
Commands:
kubectl describe pvc <pvc> kubectl get pv
Common Causes:
Error Example:
node(s) had taint {gpu=true:NoSchedule}
Think:
Taint → Toleration
Commands:
kubectl describe node <node> kubectl get pod <pod> -o yaml
Toleration Example:
tolerations: - key: "gpu" operator: "Equal" value: "true" effect: "NoSchedule"
Commands:
kubectl describe node <node> kubectl get events
Check:
kubectl describe pod <pod> kubectl logs <pod> kubectl logs <pod> --previous kubectl describe deploy <deploy> kubectl get rs kubectl describe pvc <pvc> kubectl get pv kubectl get endpoints <svc> kubectl describe svc <svc> kubectl describe node <node> kubectl get all -n <namespace> kubectl get pods --show-labels kubectl run busybox \ --image=busybox \ --restart=Never -- sleep 3600
Pending → Scheduler CrashLoopBackOff → Logs ImagePullBackOff → Image / Registry Service Failing → Endpoints PVC Pending → PV / StorageClass Node NotReady → Node Conditions FailedScheduling → Resources / Taints