Bug 1140850
| Summary: | Using "iSCSI Bond", host does not disconnect from iSCSI targets | |||
|---|---|---|---|---|
| Product: | Red Hat Enterprise Virtualization Manager | Reporter: | Amador Pahim <asegundo> | |
| Component: | ovirt-engine | Assignee: | Maor <mlipchuk> | |
| Status: | CLOSED CURRENTRELEASE | QA Contact: | Elad <ebenahar> | |
| Severity: | high | Docs Contact: | ||
| Priority: | high | |||
| Version: | 3.4.0 | CC: | amureini, asegundo, gklein, lpeer, rbalakri, Rhev-m-bugs, schlegel, scohen, tnisan, yeylon, ykaul, ylavi | |
| Target Milestone: | ovirt-3.6.0-rc | Keywords: | ZStream | |
| Target Release: | 3.6.0 | Flags: | amureini:
Triaged+
|
|
| Hardware: | All | |||
| OS: | All | |||
| Whiteboard: | ||||
| Fixed In Version: | Doc Type: | Bug Fix | ||
| Doc Text: | Story Points: | --- | ||
| Clone Of: | ||||
| : | 1191466 (view as bug list) | Environment: | ||
| Last Closed: | Type: | Bug | ||
| Regression: | --- | Mount Type: | --- | |
| Documentation: | --- | CRM: | ||
| Verified Versions: | Category: | --- | ||
| oVirt Team: | Storage | RHEL 7.3 requirements from Atomic Host: | ||
| Cloudforms Team: | --- | Target Upstream Version: | ||
| Embargoed: | ||||
| Bug Depends On: | 1178838, 1279485, 1298534 | |||
| Bug Blocks: | 1191466 | |||
Raising priority/severity since it blocks BZ#1112861, which is high/high. Hi Amador, I uploaded a patch which fixes the Host moving to maintenance, though I'm not sure I understood part of your comment "...OR when the target connection is changed." Currently oVirt does not support connection removal in an existing Storage Domain meaning The user can only connect a Storage Domain to new targets, but one can not disconnect from existing connections. (In reply to Maor from comment #4) > Hi Amador, I uploaded a patch which fixes the Host moving to maintenance, > though I'm not sure I understood part of your comment "...OR when the target > connection is changed." I was wondering what happens when the user changes the networks/targets to be used in Admin. Portal. Will we disconnect from removed paths to connect to new ones? I have no good answer for this. > > Currently oVirt does not support connection removal in an existing Storage > Domain meaning The user can only connect a Storage Domain to new targets, > but one can not disconnect from existing connections. Ok, then we are working as designed and no extra effort is needed here. Thank you! iSCSI sessions are not disconnected on disconnectStorageServer with and without bond configured: https://bugzilla.redhat.com/show_bug.cgi?id=1279485 Cannot attach static configured networks, which are required for iSCSI bond configuration in our environment, due to https://bugzilla.redhat.com/show_bug.cgi?id=1298534 Using iSCSI bond, host disconnects successfully from the iSCSI targets.
Configured iSCSI bond with 2 host networks (host mgmt network doesn't participate in the bond):
[root@puma13 nodes]# tree /var/lib/iscsi/nodes
/var/lib/iscsi/nodes
|-- iqn.2008-05.com.xtremio:xio00153500071-514f0c50023f6c00
| `-- 10.35.146.129,3260,1
| |-- enp4s0f1
| `-- enp5s0f1
|-- iqn.2008-05.com.xtremio:xio00153500071-514f0c50023f6c01
| `-- 10.35.146.161,3260,1
| |-- enp4s0f1
| `-- enp5s0f1
|-- iqn.2008-05.com.xtremio:xio00153500071-514f0c50023f6c04
| `-- 10.35.146.193,3260,1
| |-- enp4s0f1
| `-- enp5s0f1
`-- iqn.2008-05.com.xtremio:xio00153500071-514f0c50023f6c05
`-- 10.35.146.225,3260,1
|-- enp4s0f1
`-- enp5s0f1
8 directories, 8 files
**Host in maintenance with no iscsi sessions**
[root@puma13 ~]# iscsiadm -m node -R
iscsiadm: No session found.
**Host is up with bond configured**
[root@puma13 ~]# iscsiadm -m node -R
Rescanning session [sid: 21, target: iqn.2008-05.com.xtremio:xio00153500071-514f0c50023f6c00, portal: 10.35.146.129,3260]
Rescanning session [sid: 22, target: iqn.2008-05.com.xtremio:xio00153500071-514f0c50023f6c01, portal: 10.35.146.161,3260]
Rescanning session [sid: 23, target: iqn.2008-05.com.xtremio:xio00153500071-514f0c50023f6c04, portal: 10.35.146.193,3260]
Rescanning session [sid: 24, target: iqn.2008-05.com.xtremio:xio00153500071-514f0c50023f6c05, portal: 10.35.146.225,3260]
Rescanning session [sid: 25, target: iqn.2008-05.com.xtremio:xio00153500071-514f0c50023f6c00, portal: 10.35.146.129,3260]
Rescanning session [sid: 26, target: iqn.2008-05.com.xtremio:xio00153500071-514f0c50023f6c01, portal: 10.35.146.161,3260]
Rescanning session [sid: 27, target: iqn.2008-05.com.xtremio:xio00153500071-514f0c50023f6c04, portal: 10.35.146.193,3260]
Rescanning session [sid: 28, target: iqn.2008-05.com.xtremio:xio00153500071-514f0c50023f6c05, portal: 10.35.146.225,3260]
Put host in maintenance:
engine.log:
2016-02-11 15:17:43,116 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-12) [6c81b0a3] START, SetVdsStatusVDSCommand(HostName = puma13, SetVdsStatusVDSCommandParameters:{runAsync='true', hostId='97a6f779-2b2d-42d4-b0e5-2a734f3b11c5', status='PreparingForMaintenance', nonOperationalReason='NONE', stopSpmFailureLogged='true', maintenanceReason='null'}), log id: 379705ed
2016-02-11 15:17:48,443 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-12) [6a43bf05] START, DisconnectStorageServerVDSCommand(HostName = puma13, StorageServerConnectionManagementVDSParameters:{runAsync='true', hostId='97a6f779-2b2d-42d4-b0e5-2a734f3b11c5', storagePoolId='71ecc6b1-2c59-4616-81a8-703abbf7d97f', storageType='ISCSI', connectionList='[StorageServerConnections:{id='85598141-a22c-43e9-9b92-8b2dea136557', connection='10.35.146.129', iqn='iqn.2008-05.com.xtremio:xio00153500071-514f0c50023f6c00', vfsType='null', mountOptions='null', nfsVersion='null', nfsRetrans='null', nfsTimeo='null', iface='enp4s0f1', netIfaceName='enp4s0f1'}, StorageServerConnections:{id='b7515dc1-02b7-4413-bcce-e2daad87f91c', connection='10.35.146.161', iqn='iqn.2008-05.com.xtremio:xio00153500071-514f0c50023f6c01', vfsType='null', mountOptions='null', nfsVersion='null', nfsRetrans='null', nfsTimeo='null', iface='enp4s0f1', netIfaceName='enp4s0f1'}, StorageServerConnections:{id='3594b735-fcd2-4b9a-8346-c3fa2cc4b17e', connection='10.35.146.193', iqn='iqn.2008-05.com.xtremio:xio00153500071-514f0c50023f6c04', vfsType='null', mountOptions='null', nfsVersion='null', nfsRetrans='null', nfsTimeo='null', iface='enp4s0f1', netIfaceName='enp4s0f1'}, StorageServerConnections:{id='f85cd1d7-7ff2-4138-9d4a-19bc60b3cb81', connection='10.35.146.225', iqn='iqn.2008-05.com.xtremio:xio00153500071-514f0c50023f6c05', vfsType='null', mountOptions='null', nfsVersion='null', nfsRetrans='null', nfsTimeo='null', iface='enp4s0f1', netIfaceName='enp4s0f1'}, StorageServerConnections:{id='2e938bd4-4b0f-47ec-9b90-0fbc4708b370', connection='10.35.146.129', iqn='iqn.2008-05.com.xtremio:xio00153500071-514f0c50023f6c00', vfsType='null', mountOptions='null', nfsVersion='null', nfsRetrans='null', nfsTimeo='null', iface='enp5s0f1', netIfaceName='enp5s0f1'}, StorageServerConnections:{id='c32c9d61-b0de-4934-a82a-233e80e1a663', connection='10.35.146.161', iqn='iqn.2008-05.com.xtremio:xio00153500071-514f0c50023f6c01', vfsType='null', mountOptions='null', nfsVersion='null', nfsRetrans='null', nfsTimeo='null', iface='enp5s0f1', netIfaceName='enp5s0f1'}, StorageServerConnections:{id='11a4547c-aae1-491e-ba93-0c877a444ec5', connection='10.35.146.193', iqn='iqn.2008-05.com.xtremio:xio00153500071-514f0c50023f6c04', vfsType='null', mountOptions='null', nfsVersion='null', nfsRetrans='null', nfsTimeo='null', iface='enp5s0f1', netIfaceName='enp5s0f1'}, StorageServerConnections:{id='d9399fba-5eb7-495a-a418-fb382b63016a', connection='10.35.146.225', iqn='iqn.2008-05.com.xtremio:xio00153500071-514f0c50023f6c05', vfsType='null', mountOptions='null', nfsVersion='null', nfsRetrans='null', nfsTimeo='null', iface='enp5s0f1', netIfaceName='enp5s0f1'}]'}), log id: 1b2a3eb2
**Host in maintenance**
[root@puma13 ~]# iscsiadm -m node -R
iscsiadm: No session found.
Verified using:
rhevm-3.6.3.1-0.1.el6.noarch
vdsm-4.17.20-0.el7ev.noarch
iscsi-initiator-utils-iscsiuio-6.2.0.873-32.el7.x86_64
libiscsi-1.9.0-6.el7.x86_64
iscsi-initiator-utils-6.2.0.873-32.el7.x86_64
sanlock-3.2.4-2.el7_2.x86_64
RHEV 3.6.0 has been released, setting status to CLOSED CURRENTRELEASE RHEV 3.6.0 has been released, setting status to CLOSED CURRENTRELEASE RHEV 3.6.0 has been released, setting status to CLOSED CURRENTRELEASE |
Description of problem: When using "iSCSI Bond" feature, Host should disconnect from the target when changed to maintenance mode OR when the target connection is changed. Version-Release number of selected component (if applicable): rhevm-3.4.1-0.31.el6ev.noarch Actual results: Host changes to maintenance status, but all iSCSI targets are still connected. Expected results: Disconnect from iSCSI targets the same way we do when not using iSCSI Bond. Additional info: Also, changing iSCSI Bond configuration should disconnect from iSCSI targets properly. The issue is engine not sending iscsi iface along with DisconnectStorageServerVDSCommand. Vdsm seems to be prepared to disconnect from iscsi targets whenever the correct iface name arrives from engine. Since the iface name is not sent by engine, vdsm assumes it is "default" and fails silently. Some results: - No iSCSI Bond configuration (working as expected): # iscsiadm -m node -R Rescanning session [sid: 6, target: iqn.2008-09.com.example:server.target1, portal: 192.168.123.216,3260] ** Changing host to maintenance mode ** engine: 2014-09-11 17:22:06,079 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-34) [3fc506bb] START, DisconnectStorageServerVDSCommand(HostName = vdsm, HostId = 989a0fcc-9f37-42e5-a84e-9473da2fdd3d, storagePoolId = 00000002-0002-0002-0002-000000000162, storageType = ISCSI, connectionList = [{ id: 2a7e99ff-29ac-40b8-872d-1e75343b2414, connection: 192.168.123.216, iqn: iqn.2008-09.com.example:server.target1, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 3a07bbdc vdsm: Thread-309::INFO::2014-09-11 17:22:05,804::logUtils::48::dispatcher::(wrapper) Run and protect: disconnectStorageServer(domType=3, spUUID=u'00000002-0002-0002-0002-000000000162', conList=[{u'connection': u'192.168.123.216', u'iqn': u'iqn.2008-09.com.example:server.target1', u'user': u'', u'tpgt': u'1', u'password': '******', u'id': u'2a7e99ff-29ac-40b8-872d-1e75343b2414', u'port': u'3260'}], options=None) Thread-309::DEBUG::2014-09-11 17:22:05,815::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /usr/sbin/iscsiadm -m session -r 6 -u (cwd None) Thread-309::DEBUG::2014-09-11 17:22:06,350::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 Thread-309::DEBUG::2014-09-11 17:22:06,350::misc::743::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) Thread-309::DEBUG::2014-09-11 17:22:06,350::misc::746::Storage.SamplingMethod::(__call__) Got in to sampling method Thread-309::DEBUG::2014-09-11 17:22:06,351::misc::743::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) Thread-309::DEBUG::2014-09-11 17:22:06,351::misc::746::Storage.SamplingMethod::(__call__) Got in to sampling method Thread-309::DEBUG::2014-09-11 17:22:06,351::iscsi::412::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds Thread-309::DEBUG::2014-09-11 17:22:06,351::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /usr/sbin/iscsiadm -m session -R (cwd None) ** Host in maintenance mode ** # iscsiadm -m node -R iscsiadm: No session found. ----------------------- - With iSCSI Bond configuration (not working): # iscsiadm -m node -R Rescanning session [sid: 4, target: iqn.2008-09.com.example:server.target1, portal: 192.168.123.216,3260] Rescanning session [sid: 5, target: iqn.2008-09.com.example:server.target1, portal: 192.168.123.216,3260] ** Changing host to maintenance mode ** engine: 2014-09-11 17:17:58,971 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-8) [2d1b3863] START, DisconnectStorageServerVDSCommand(HostName = vdsm, HostId = 989a0fcc-9f37-42e5-a84e-9473da2fdd3d, storagePoolId = 00000002-0002-0002-0002-000000000162, storageType = ISCSI, connectionList = [{ id: 2a7e99ff-29ac-40b8-872d-1e75343b2414, connection: 192.168.123.216, iqn: iqn.2008-09.com.example:server.target1, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: dcb0692 vdsm: Thread-177::INFO::2014-09-11 17:17:58,673::logUtils::48::dispatcher::(wrapper) Run and protect: disconnectStorageServer(domType=3, spUUID=u'00000002-0002-0002-0002-000000000162', conList=[{u'connection': u'192.168.123.216', u'iqn': u'iqn.2008-09.com.example:server.target1', u'user': u'', u'tpgt': u'1', u'password': '******', u'id': u'2a7e99ff-29ac-40b8-872d-1e75343b2414', u'port': u'3260'}], options=None) Thread-177::DEBUG::2014-09-11 17:17:58,676::misc::743::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) Thread-177::DEBUG::2014-09-11 17:17:58,676::misc::746::Storage.SamplingMethod::(__call__) Got in to sampling method Thread-177::DEBUG::2014-09-11 17:17:58,677::misc::743::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) Thread-177::DEBUG::2014-09-11 17:17:58,677::misc::746::Storage.SamplingMethod::(__call__) Got in to sampling method Thread-177::DEBUG::2014-09-11 17:17:58,677::iscsi::412::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds Thread-177::DEBUG::2014-09-11 17:17:58,677::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /usr/sbin/iscsiadm -m session -R (cwd None) ** Host in maintenance mode ** # iscsiadm -m node -R Rescanning session [sid: 4, target: iqn.2008-09.com.example:server.target1, portal: 192.168.123.216,3260] Rescanning session [sid: 5, target: iqn.2008-09.com.example:server.target1, portal: 192.168.123.216,3260]