Description of problem: Could not dynamic create volume for ebs pvc. We could dynamic create ebs volume on this server at the first, but failed to dynamic create volumes later. Version-Release number of selected component (if applicable): version 4.1.7 How reproducible: Not always. Steps to Reproduce: 1.Create a pvc oc describe pvc ebsc Name: ebsc Namespace: chaoyang StorageClass: gp2 Status: Pending Volume: Labels: <none> Annotations: volume.beta.kubernetes.io/storage-class: gp2 Finalizers: [kubernetes.io/pvc-protection] Capacity: Access Modes: VolumeMode: Filesystem Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal WaitForFirstConsumer 76s (x903 over 3h41m) persistentvolume-controller waiting for first consumer to be created before binding Mounted By: mypod 2.Create pod oc describe pods mypod Name: mypod Namespace: chaoyang Priority: 0 PriorityClassName: <none> Node: ip-10-0-69-82.us-east-2.compute.internal/10.0.69.82 Start Time: Mon, 22 Jul 2019 17:49:51 +0800 Labels: name=frontendhttp Annotations: openshift.io/scc: anyuid Status: Pending IP: Containers: myfrontend: Container ID: Image: jhou/hello-openshift Image ID: Port: 80/TCP Host Port: 0/TCP State: Waiting Reason: ContainerCreating Ready: False Restart Count: 0 Environment: <none> Mounts: /tmp from aws (rw) /var/run/secrets/kubernetes.io/serviceaccount from default-token-8v6wl (ro) Conditions: Type Status Initialized True Ready False ContainersReady False PodScheduled True Volumes: aws: Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace) ClaimName: ebs ReadOnly: false default-token-8v6wl: Type: Secret (a volume populated by a Secret) SecretName: default-token-8v6wl Optional: false QoS Class: BestEffort Node-Selectors: <none> Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s node.kubernetes.io/unreachable:NoExecute for 300s Events: Type Reason Age From Message ---- ------ ---- ---- ------- Warning FailedMount 4m18s (x436 over 16h) kubelet, ip-10-0-69-82.us-east-2.compute.internal Unable to mount volumes for pod "mypod_chaoyang(0c414fd2-ac66-11e9-a974-0224b1289da4)": timeout expired waiting for volumes to attach or mount for pod "chaoyang"/"mypod". list of unmounted volumes=[aws]. list of unattached volumes=[aws default-token-8v6wl] Actual results: Pod is in status of 'ContainerCreating' Expected results: Pod is running Master Log: openshift-kube-controller-manager/pods/kube-controller-manager-ip-10-0-74-79.us-east-2.compute.internal/kube-controller-manager-5/kube-controller-manager-5/logs/current.log:2019-07-22T07:46:53.182387077Z I0722 07:46:53.182369 1 event.go:221] Event(v1.ObjectReference{Kind:"PersistentVolumeClaim", Namespace:"chaoyang", Name:"ebsc", UID:"9b1194e0-ac4f-11e9-a916-0224b1289da4", APIVersion:"v1", ResourceVersion:"112329", FieldPath:""}): type: 'Normal' reason: 'WaitForFirstConsumer' waiting for first consumer to be created before binding Node Log (of failed PODs): PV Dump: PVC Dump: StorageClass Dump (if StorageClass used by PV/PVC): Additional info: