Description of problem: Since errata tracked in this BZ https://bugzilla.redhat.com/show_bug.cgi?id=1612783 Information like VolumeInUse on node are not update properly (FC volume appears twice) Kubelet tries to umount the same volume twice and failed because volume is already umount. Pods using FC are not rescheduled because FC volume is still mount from a kubelet point of view Version-Release number of selected component (if applicable): 3.9 with errata RHBA-2018:3748 https://access.redhat.com/errata/RHBA-2018:3748 https://bugzilla.redhat.com/show_bug.cgi?id=1612783 How reproducible: 3.9 with errata RHBA-2018:3748 PV/PVC with fiberchannel lun with multiple path Drain the node Actual results: FC volume apprears twice in VolumeInUse Kubelet tries to umount twice Expected results: Information on node about FC volume should be accurate
Checked on below version, # oc version oc v3.9.74 kubernetes v1.9.1+a0ce1bc657 features: Basic-Auth GSSAPI Kerberos SPNEGO Server https://xxx-qe-cluster-xxx:8443 openshift v3.9.74 kubernetes v1.9.1+a0ce1bc657 Scenarios tested: 1. Create pod with fc, then drain node. 2. Create pod with fc, then scale down. 3. Create pod with fc, scale down, then scale up. Then check the logs on the node, [root@node1 lxia]# journalctl -u atomic-openshift-node | grep -i 'failed to detach' [root@node1 lxia]# journalctl -u atomic-openshift-node | grep -i 'ConstructVolumeSpec' Mar 22 01:07:23 xxx-qe-cluster-node1-xxx atomic-openshift-node[18940]: I0322 01:07:23.828142 18940 fc.go:265] ConstructVolumeSpec: TargetWWNs: [50050768030539b6 50050768030539b7], Lun: 0, WWIDs: [] Mar 22 01:10:23 xxx-qe-cluster-node1-xxx atomic-openshift-node[18940]: I0322 01:10:23.841256 18940 fc.go:265] ConstructVolumeSpec: TargetWWNs: [50050768030539b6 50050768030539b7], Lun: 0, WWIDs: [] [root@node2 ~]# journalctl -u atomic-openshift-node | grep -i 'failed to detach' [root@node2 ~]# journalctl -u atomic-openshift-node | grep -i 'ConstructVolumeSpec' Mar 22 00:55:23 xxx-qe-cluster-node2-xxx atomic-openshift-node[5467]: I0322 00:55:23.013415 5467 fc.go:265] ConstructVolumeSpec: TargetWWNs: [50050768030539b6 50050768030539b7], Lun: 0, WWIDs: [] Mar 22 00:58:23 xxx-qe-cluster-node2-xxx atomic-openshift-node[5467]: I0322 00:58:23.020057 5467 fc.go:265] ConstructVolumeSpec: TargetWWNs: [50050768030539b6 50050768030539b7], Lun: 0, WWIDs: [] Mar 22 01:01:23 xxx-qe-cluster-node2-xxx atomic-openshift-node[5467]: I0322 01:01:23.034351 5467 fc.go:265] ConstructVolumeSpec: TargetWWNs: [50050768030539b6 50050768030539b7], Lun: 0, WWIDs: []
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:0619