Description of problem: Seeing an issue with StatefulSet where the pod throws an error saying devicePath is empty. This is not impacting all StatefulSets but whenever there is redeployment. Version-Release number of selected component (if applicable): OCP v3.11.43 How reproducible: Issue is repeatable when re-deployments are made Steps to Reproduce: 1. Start with running 3.11.43 cluster 2. Create a StatefulSet with attached PVC connected to a dynamic provisioned volume 3. Delete StatefulSet Actual results: As the StatefulSet recreates: 1. I see that vmdk is attached to the VM. 2. I see lsblk command shows the disk attached on the VM. 3. Pod shows the error as shown below. -------------------------------------------------- Type Reason Age From Message ---- ------ ---- ---- ------- Warning FailedMount 22m (x51 over 2h) kubelet, customerqa008 Unable to mount volumes for pod "query-master-0_app-qa(7d30a1db-ffa0-11e8-a650-005056baf2f2)": timeout expired waiting for volumes to attach or mount for pod "app-qa"/"search-master-0". list of unmounted volumes=[query-raw-data]. list of unattached volumes=[query-raw-data log-data splunk-config default-datasource] Warning FailedMount 11m (x70 over 2h) kubelet, suocpedcmpqa008 MountVolume.WaitForAttach failed for volume "pvc-cc116f5f-fee9-11e8-8682-005056baf2f2" : WaitForAttach failed for VMDK "[CUST_VPLEX0146_SG01_D700_local] kubevols/kubernetes-dynamic-pvc-cc116f5f-fee9-11e8-8682-005056baf2f2.vmdk": devicePath is empty. ---------------------------------------------------------- Expected results: Pod recreates correctly with PVC mounted. Additional info:
Moving to storage.
It is passed on Server https://ip-172-18-13-58.ec2.internal:8443 openshift v3.11.75 kubernetes v1.11.0+d4cacc0 1.Create a StatefulSet app apiVersion: v1 kind: Service metadata: name: nginx labels: app: nginx spec: ports: - port: 80 name: web clusterIP: None selector: app: nginx --- apiVersion: apps/v1 kind: StatefulSet metadata: name: web spec: selector: matchLabels: app: nginx # has to match .spec.template.metadata.labels serviceName: "nginx" replicas: 3 # by default is 1 template: metadata: labels: app: nginx # has to match .spec.selector.matchLabels spec: terminationGracePeriodSeconds: 10 containers: - name: nginx image: aosqe/hello-openshift ports: - containerPort: 80 name: web volumeMounts: - name: www mountPath: /usr/share/nginx/html volumeClaimTemplates: - metadata: name: www spec: accessModes: [ "ReadWriteOnce" ] resources: requests: storage: 1Gi 2.After all pods are running, delete the StatefulSet 3.Recreate the StatefulSet 4.Pods are running, no error met
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:0326