Description of problem: - We have an instance named "feb3-selinux" which has a 5GB volume attached to it. ======================================= [root@feb3-selinux data1]# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT vda 253:0 0 20G 0 disk └─vda1 253:1 0 20G 0 part / vdb 253:16 0 4.8G 0 disk /data1 [root@feb3-selinux data1]# date Tue Feb 16 05:35:13 EST 2016 [root@feb3-selinux data1]# df -kh Filesystem Size Used Avail Use% Mounted on /dev/vda1 20G 859M 20G 5% / devtmpfs 1.9G 0 1.9G 0% /dev tmpfs 1.9G 0 1.9G 0% /dev/shm tmpfs 1.9G 17M 1.9G 1% /run tmpfs 1.9G 0 1.9G 0% /sys/fs/cgroup /dev/vdb 4.6G 19M 4.3G 1% /data1 ==================================== - Next, we try to detach the volume from the Horizon dashboard. The volume is detached from the instance, but the state on the horizon dashboard keeps as detaching even after very long time. - But, the volume is not attached to the instance any longer. [root@feb3-selinux data1]# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT vda 253:0 0 20G 0 disk └─vda1 253:1 0 20G 0 part / [root@feb3-selinux data1]# date Tue Feb 16 05:42:34 EST 2016 [root@feb3-selinux data1]# df -kh Filesystem Size Used Avail Use% Mounted on /dev/vda1 20G 859M 20G 5% / devtmpfs 1.9G 0 1.9G 0% /dev tmpfs 1.9G 0 1.9G 0% /dev/shm tmpfs 1.9G 17M 1.9G 1% /run tmpfs 1.9G 0 1.9G 0% /sys/fs/cgroup /dev/vdb 4.6G 19M 4.3G 1% /data1 [root@feb3-selinux data1]# cd /data1 [root@feb3-selinux data1]# [root@feb3-selinux data1]# ls -lrth ls: reading directory .: Input/output error total 0 - The volume is not shown in the multipath O/p on the compute node. - When the volume is in detaching state, i manually go and update the status of the volume as available. But in the attached to column , it still shows "attached to feb3-selinux", even when it is actually not. - After this, surprisingly when i do a hard reboot of the isntance, it actually attaches that volume back to the instance. Find the below O/p for your reference. ============================ [gaurav.MCKD-4041] ➤ ssh root.xx.xx X11 forwarding request failed on channel 0 Last login: Tue Feb 16 05:11:03 2016 from xx.xx.xx.xx [root@feb3-selinux ~]# [root@feb3-selinux ~]# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT vda 253:0 0 20G 0 disk └─vda1 253:1 0 20G 0 part / vdb 253:16 0 4.8G 0 disk [root@feb3-selinux ~]# [root@comp3 ~]# multipath -ll | grep 360002ac0000000000000006400008044 360002ac0000000000000006400008044 dm-13 3PARdata,VV Version-Release number of selected component (if applicable): - RHOS6(non-ha) - HP 3PAR backend for cinder(cinder.volume.drivers.san.hp.hp_3par_fc.HP3PARFCDriver) How reproducible: 100% Steps to Reproduce: 1. 2. 3. Actual results: unable to detach volume from instance. Expected results: volume should be detached from the instance. Additional info:
HP 3PAR driver fails to delete a vlun, most probably because volume doesn't exist on the storage anymore. We have to escalate this issue to HP. 2016-02-16 16:09:07.501 55809 ERROR cinder.volume.drivers.san.hp.hp_3par_common [req-7aff09d7-2a6f-4c4f-aa78-da3d43876568 0691b74100f64062a160f986269f2ed5 767f5371e9d94a38b1169b4870d73daf - - -] Not found (HTTP 404) 17 - host does not exist
It seems that others also experience very similar problems trying detach cinder volume backed by the HP 3PAR driver. Its not necessarily exactly the same problem, but the error in the log would look the same. https://bugs.launchpad.net/cinder/+bug/1546453
*** Bug 1309316 has been marked as a duplicate of this bug. ***
Created attachment 1129166 [details] 3par_volume_hostinfo
fix is in openstack-cinder-2014.2.4-6.el7ost.noarch automation passed https://rhos-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/RHOS/view/RHOS6/job/rhos-jenkins-rhos-6.0-puddle-rhel-7.2-multi-node-packstack-neutron-ml2-gre-rabbitmq-tempest-git-all/13/
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHBA-2016-0436.html