@Ilya can you retry same thing with creating a bind mount of "/mnt" somewhere else with "_netdev" option. something like: # mount -o _netdev /dev/rd0 /mnt # mount -o bind,_netdev /mnt /foo And then run shutdown/poweroff? It appears to me that creating a bind mount consistenly reproduces this problem.
Move this to systemd. lets see where this one goes.
The PR is merged.
Tested on below version: openshift v3.6.173.0.147 kubernetes v1.6.1+5115d708d7 # uname -a Linux xxx 3.10.0-957.1.3.el7.x86_64 #1 SMP Thu Nov 15 17:36:42 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux # cat /etc/redhat-release Red Hat Enterprise Linux Server release 7.6 (Maipo) # oc get sc NAME TYPE cephrbdprovisioner kubernetes.io/rbd # oc get pvc NAME STATUS VOLUME CAPACITY ACCESSMODES STORAGECLASS AGE cephrbdprovisioner Bound pvc-ed457136-1e0d-11e9-a26a-fa163e2b34ed 1Gi RWO cephrbdprovisioner 25m pvc Bound pvc-9f5054de-1e0f-11e9-a26a-fa163e2b34ed 1Gi RWO cephrbdprovisioner 13m pvc1 Bound pvc-abda841b-1e0f-11e9-a26a-fa163e2b34ed 1Gi RWO cephrbdprovisioner 13m # oc get pods NAME READY STATUS RESTARTS AGE cephpod 1/1 Running 0 12m cephpod1 1/1 Running 0 11m cephpod2 1/1 Running 0 11m And on the node # mount | grep rbd /dev/rbd0 on /var/lib/origin/openshift.local.volumes/plugins/kubernetes.io/rbd/rbd/rbd-image-kubernetes-dynamic-pvc-ed47d478-1e0d-11e9-a26a-fa163e2b34ed type ext4 (rw,relatime,seclabel,stripe=1024,data=ordered) /dev/rbd0 on /var/lib/origin/openshift.local.volumes/pods/e667d4ba-1e0f-11e9-a26a-fa163e2b34ed/volumes/kubernetes.io~rbd/pvc-ed457136-1e0d-11e9-a26a-fa163e2b34ed type ext4 (rw,relatime,seclabel,stripe=1024,data=ordered) /dev/rbd1 on /var/lib/origin/openshift.local.volumes/plugins/kubernetes.io/rbd/rbd/rbd-image-kubernetes-dynamic-pvc-9f52594e-1e0f-11e9-a26a-fa163e2b34ed type ext4 (rw,relatime,seclabel,stripe=1024,data=ordered) /dev/rbd1 on /var/lib/origin/openshift.local.volumes/pods/f8c6e84b-1e0f-11e9-a26a-fa163e2b34ed/volumes/kubernetes.io~rbd/pvc-9f5054de-1e0f-11e9-a26a-fa163e2b34ed type ext4 (rw,relatime,seclabel,stripe=1024,data=ordered) /dev/rbd2 on /var/lib/origin/openshift.local.volumes/plugins/kubernetes.io/rbd/rbd/rbd-image-kubernetes-dynamic-pvc-abdd4923-1e0f-11e9-a26a-fa163e2b34ed type ext4 (rw,relatime,seclabel,stripe=1024,data=ordered) /dev/rbd2 on /var/lib/origin/openshift.local.volumes/pods/0a0d17d4-1e10-11e9-a26a-fa163e2b34ed/volumes/kubernetes.io~rbd/pvc-abda841b-1e0f-11e9-a26a-fa163e2b34ed type ext4 (rw,relatime,seclabel,stripe=1024,data=ordered) # reboot The server never come back, the mount above do not have "_netdev" option. Does we have the 3.6 PR for fixing this?