Bug 1157673

Summary: Sanlock doesn't unlock iSCSi devices upon host removal
Product: [oVirt] vdsm Reporter: Ori Gofen <ogofen>
Component: GeneralAssignee: Daniel Erez <derez>
Status: CLOSED DUPLICATE QA Contact: Aharon Canan <acanan>
Severity: medium Docs Contact:
Priority: unspecified    
Version: ---CC: acanan, ahino, amureini, bazulay, bugs, gklein, lpeer, lsurette, scohen, srevivo, tnisan, ykaul, ylavi
Target Milestone: ovirt-4.1.0-alphaFlags: amureini: ovirt-4.1?
rule-engine: planning_ack?
rule-engine: devel_ack?
rule-engine: testing_ack?
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-06-01 09:00:57 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Storage RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
logs none

Description Ori Gofen 2014-10-27 13:14:46 UTC
Created attachment 951003 [details]
logs

Description of problem:

The consequences of this problem could lead to multiple failures, among them security issues, though I haven't explored those consequences enough, I hope the following description will be enough to put this one on priority.

Lets say I have a host "camel-$name" with no active sessions

root@camel-$name ~ # iscsiadm -m session                                                                                                                  127 ↵
iscsiadm: No active sessions.
host does not "see" any multipath devices either

root@camel-$name ~ # multipath -ll  
root@camel-$name ~ #

now we add a host and created a iSCSi domain, after creation, we are removing this host.

vdsm logs out from host's sessions:

root@camel-$name ~ # iscsiadm -m session                                                                                                                  127 ↵
iscsiadm: No active sessions.

but multipath -ll still see an active device:

root@camel-$name ~ # multipath -ll                                                                                                                         21 ↵
360060160f4a030007beed85291dbe311 dm-11 
size=30G features='0' hwhandler='1 emc' wp=rw

root@camel-$name ~ # dmsetup info /dev/mapper/360060160f4a030007beed85291dbe311                                                                             1 ↵
Name:              360060160f4a030007beed85291dbe311
State:             ACTIVE
Read Ahead:        1024
Tables present:    LIVE
Open count:        6
Event number:      0
Major, minor:      253, 11
Number of targets: 1
UUID: mpath-360060160f4a030007beed85291dbe311

when exploring further into what causing this we see all lv's on Lun
root@camel-$name /dev # ls -lR | grep dm-name
lrwxrwxrwx. 1 root root 11 Oct 27 12:12 dm-name-360060160f4a030007beed85291dbe311 -> ../../dm-11
lrwxrwxrwx. 1 root root 11 Oct 26 11:36 dm-name-f26311e7--832d--4060--944e--2e7228f6e1f6-ids -> ../../dm-13
lrwxrwxrwx. 1 root root 11 Oct 26 11:30 dm-name-f26311e7--832d--4060--944e--2e7228f6e1f6-inbox -> ../../dm-16
lrwxrwxrwx. 1 root root 11 Oct 26 11:36 dm-name-f26311e7--832d--4060--944e--2e7228f6e1f6-leases -> ../../dm-14
lrwxrwxrwx. 1 root root 11 Oct 26 11:38 dm-name-f26311e7--832d--4060--944e--2e7228f6e1f6-master -> ../../dm-17
lrwxrwxrwx. 1 root root 11 Oct 26 11:35 dm-name-f26311e7--832d--4060--944e--2e7228f6e1f6-metadata -> ../../dm-12
lrwxrwxrwx. 1 root root 11 Oct 26 11:31 dm-name-f26311e7--832d--4060--944e--2e7228f6e1f6-outbox -> ../../dm-15

two of them belongs to Sanlock

root@camel-$name /dev # ls -lR | grep "dm-13|dm-14"
brw-rw----.  1 vdsm sanlock 253,  13 Oct 26 11:36 dm-13
brw-rw----.  1 vdsm sanlock 253,  14 Oct 26 11:36 dm-14

tried to kill sanlock, it failed, restarting service also failed.
and

root@camel-$name /dev # dmsetup remove /dev/mapper/360060160f4a030007beed85291dbe311 
device-mapper: remove ioctl on 360060160f4a030007beed85291dbe311 failed: Device or resource busy
Command failed

After host reboot, multipath -ll doesn't prompt any output

Version-Release number of selected component (if applicable):
vt7

How reproducible:
100%

Steps to Reproduce:
1.create iSCSi domain
2.remove host

Actual results:
sanlock doesn't release one target upon host's remove

Expected results:
after a host is removed from setup, he shouldn't have any iscsiadm -m sessions, and he shouldn't be exposed to any Lun he used to be on previous setups 

Additional info:

Comment 2 Eyal Edri 2015-02-25 08:39:45 UTC
3.5.1 is already full with bugs (over 80), and since none of these bugs were added as urgent for 3.5.1 release in the tracker bug, moving to 3.5.2

Comment 3 Eyal Edri 2015-04-28 11:21:58 UTC
moving to 3.5.4 due to capacity planning for 3.5.3.
if you believe this should remain in 3.5.3, please sync with pm/dev/qe and a full triple ack for it. also - ensure priority is set accordingly.

Comment 4 Allon Mureinik 2016-01-27 05:39:58 UTC
Related to bug 1163890?

Comment 5 Ala Hino 2016-01-27 09:25:31 UTC
Yes, two issues seems to be in the same area

Comment 7 Daniel Erez 2016-06-01 09:00:57 UTC
Similar issue as described in bug 1163890. Should be fixed as part of it. Closing as a duplicate.

*** This bug has been marked as a duplicate of bug 1163890 ***