Kubernetes pod crash and readiness investigation cheat sheet
Trace a failing pod through context, scheduling, container state, current logs, previous logs and namespace events.
kubectl against a supported Kubernetes cluster. Examples target namespace production, pod web-abc123 and container web.
Updated · Kubernetes · 6 min read
| Command | What it does |
|---|---|
kubectl config current-context | Confirm the selected cluster context |
kubectl get pods -n production -o wide | Inspect pod readiness, restarts and placement |
kubectl describe pod web-abc123 -n production | Inspect container states, probes and events |
kubectl logs web-abc123 -n production -c web --since=30m --tail=100 --timestamps=true | Read the current container logs |
kubectl logs web-abc123 -n production -c web --previous --tail=100 --timestamps=true | Read the previous terminated instance |
kubectl get events -n production --sort-by=.metadata.creationTimestamp | Correlate recent namespace events |
Walkthrough
Replace namespace, pod and container names. Confirm cluster context and RBAC read access. Pod replacement changes the name and UID; use the pod that exists in the incident window.
Step 1: Confirm the selected cluster context
kubectl config current-context
Record the context and compare it with the incident environment. A context name is locally configured and can be misleading; verify the intended cluster through your approved inventory before interpreting workload results.
Step 2: Inspect pod readiness, restarts and placement
kubectl get pods -n production -o wide
Find the affected pod and note Ready, Status, Restarts, age and node. Running is a lifecycle phase, not a guarantee that every container is ready. Pending requires scheduling investigation; a crash loop requires container-state and log inspection.
Step 3: Inspect container states, probes and events
kubectl describe pod web-abc123 -n production
Read current State, Last State, exit reason, readiness/liveness probes and events. OOMKilled, failed probes and image-pull failures lead to different investigations. A failing readiness probe removes readiness; a failing liveness probe can trigger a restart.
Step 4: Read the current container logs
kubectl logs web-abc123 -n production -c web --since=30m --tail=100 --timestamps=true
Use the explicit container name in multi-container pods. Correlate timestamps with restarts and probes. Empty current logs may simply mean that the newest container has barely started; examine the previous container instance if it exists.
Step 5: Read the previous terminated instance
kubectl logs web-abc123 -n production -c web --previous --tail=100 --timestamps=true
Look for the first fatal message before termination. Previous logs cover the prior container instance in this pod, not every historical restart or a deleted pod. Unavailable logs require your centralized logging system or node-level investigation.
Step 6: Correlate recent namespace events
kubectl get events -n production --sort-by=.metadata.creationTimestamp
Match involved object names, messages and times to the pod under investigation. Events are retained for a limited period and can aggregate repeated occurrences. A broad namespace list may include unrelated failures, so do not infer causality from proximity alone.
WORKED INCIDENT
Practical case: A deployment has Running pods but no ready replicas
Fictional teaching scenario; observations are examples, not captured production output.
Situation
After a rollout, a web pod appears Running but shows 0/1 Ready and an increasing restart count.
Investigation sequence
1. Confirm the selected cluster context
kubectl config current-contextConfirm the cluster context before querying the production namespace.
2. Inspect pod readiness, restarts and placement
kubectl get pods -n production -o wideRecord the pod name, node and restart count twice to determine whether restarts are still increasing.
3. Inspect container states, probes and events
kubectl describe pod web-abc123 -n productionRead the termination reason and probe events. This example shows a nonzero application exit rather than OOMKilled.
4. Read the previous terminated instance
kubectl logs web-abc123 -n production -c web --previous --tail=100 --timestamps=trueRead previous logs. The fictional log reports a missing required configuration key at startup.
5. Correlate recent namespace events
kubectl get events -n production --sort-by=.metadata.creationTimestampCorrelate rollout timing and restart events without dumping Secret values into the incident ticket.
Example findings
Illustrative findings: Ready = 0/1; restart count increases; previous logs report a required configuration key missing; no OOMKilled reason.
Interpret the findings
The startup configuration is the leading hypothesis. A readiness failure alone would not explain repeated restarts; correlate application exits and liveness events before deciding on the trigger.
Verify the outcome
After the owner corrects the deployment configuration through the approved rollout, inspect the replacement pods. Confirm all expected containers are ready, restart counts stay stable, and the service responds through its normal route.
Escalation handoff
Provide namespace, pod UID/name, image, first fatal message and relevant event times to the workload owner. Record configuration key names without exposing Secret values.