100-Days-Of-DevOps-Challenge-KodeKloud

Troubleshoot Deployment Issues in Kubernetes

Last week, the Nautilus DevOps team deployed a redis app on Kubernetes cluster, which was working fine so far. This morning one of the team members was making some changes in this existing setup, but he made some mistakes and the app went down. We need to fix this as soon as possible. Please take a look.

• The deployment name is redis-deployment. The pods are not in running state right now, so please look into the issue and fix the same.

Note: The kubectl utility on jump_host has been configured to work with the kubernetes cluster.

Steps

1. Let’s check the pod in details

    kubectl describe pod redis-deployment-pod-name
   

    thor@jumphost ~$ kubectl describe pod redis-deployment-6fd9d5fcb-k9n65
    Name:             redis-deployment-6fd9d5fcb-k9n65
    Namespace:        default
    Priority:         0
    Service Account:  default
    Node:             kodekloud-control-plane/172.17.0.2
    Start Time:       Mon, 22 Sep 2025 16:42:08 +0000
    Labels:           app=redis
                    pod-template-hash=6fd9d5fcb
    Annotations:      <none>
    Status:           Pending
    IP:               
    IPs:              <none>
    Controlled By:    ReplicaSet/redis-deployment-6fd9d5fcb
    Containers:
    redis-container:
        Container ID:   
        Image:          redis:alpin
        Image ID:       
        Port:           6379/TCP
        Host Port:      0/TCP
        State:          Waiting
        Reason:       ContainerCreating
        Ready:          False
        Restart Count:  0
        Requests:
        cpu:        300m
        Environment:  <none>
        Mounts:
        /redis-master from config (rw)
        /redis-master-data from data (rw)
        /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-z6ndg (ro)
    Conditions:
    Type              Status
    Initialized       True 
    Ready             False 
    ContainersReady   False 
    PodScheduled      True 
    Volumes:
    data:
        Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
        Medium:     
        SizeLimit:  <unset>
    config:
        Type:      ConfigMap (a volume populated by a ConfigMap)
        Name:      redis-cofig
        Optional:  false
    kube-api-access-z6ndg:
        Type:                    Projected (a volume that contains injected data from multiple sources)
        TokenExpirationSeconds:  3607
        ConfigMapName:           kube-root-ca.crt
        ConfigMapOptional:       <nil>
        DownwardAPI:             true
    QoS Class:                   Burstable
    Node-Selectors:              <none>
    Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                                node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
    Events:
    Type     Reason       Age                 From               Message
    ----     ------       ----                ----               -------
    Normal   Scheduled    3m7s                default-scheduler  Successfully assigned default/redis-deployment-6fd9d5fcb-k9n65 to kodekloud-control-plane
    Warning  FailedMount  64s                 kubelet            Unable to attach or mount volumes: unmounted volumes=[config], unattached volumes=[], failed to process volumes=[]: timed out waiting for the condition
    Warning  FailedMount  59s (x9 over 3m7s)  kubelet            MountVolume.SetUp failed for volume "config" : configmap "redis-cofig" not found
   

   It seems the pod is looking for a config-map that’s not available or soemthing wrong

2. Let’s see the list of configmaps

    kubectl get cm
   

    thor@jumphost ~$ kubectl get cm
    NAME               DATA   AGE
    kube-root-ca.crt   1      27m
    redis-config       2      8m16s
   

3. If you notice carefully, there is mismatch between name. The actual config-map name is: redis-config but pod is lokking for redis-cofig

4. Let’s extract the deployment into a YAML file and fix it there

    kubectl get deployment.apps redis-deployment -o yaml > redis-deployment.yaml
   

5. Let’s fix the deployment file cofig to config in volumes

6. Let’s run the deployment

    kubectl apply -f redis-deployment.yaml
   

7. Check pod status

    kubectl get pods
   

   new error:

    thor@jumphost ~$ kubectl get pods
    NAME                                READY   STATUS              RESTARTS   AGE
    redis-deployment-5bcd4c7d64-rmcxj   0/1     ImagePullBackOff    0          4m9s
   

8. Let’s fix the image tag alpin to alpine

9. Recreate the deployment

    kubectl delete deployments.apps redis-deployment
    kubectl apply -f redis-deployment
   

10. Verify the results

    kubectl get pods

Good to Know?

Kubernetes Troubleshooting

• Pod States: Pending, Running, Succeeded, Failed, Unknown
• Container States: Waiting, Running, Terminated
• Events: Kubernetes events show what happened
• Logs: Container logs provide application-level info

Common Issues

• Image Pull Errors: Wrong image name or registry access
• Resource Constraints: Insufficient CPU/memory
• ConfigMap/Secret Missing: Referenced resources don’t exist
• Volume Mount Issues: Incorrect paths or permissions

Debugging Commands

• Describe: kubectl describe pod/deployment/service
• Logs: kubectl logs pod-name -c container-name
• Events: kubectl get events --sort-by=.metadata.creationTimestamp
• Exec: kubectl exec -it pod-name -- /bin/sh

Systematic Approach

• Check Pod Status: kubectl get pods
• Examine Events: kubectl describe pod
• Review Logs: kubectl logs
• Validate Config: Check YAML syntax and references
• Test Connectivity: Network and service discovery

This site is open source. Improve this page.