Bug 1094144 - [engine-backend] [iSCSI multipath] No indication that updating an iSCSI multipath bond doesn't trigger any operation from vdsm side
Summary: [engine-backend] [iSCSI multipath] No indication that updating an iSCSI multi...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-engine
Version: 3.4.0
Hardware: x86_64
OS: Unspecified
unspecified
high
Target Milestone: ---
: 3.5.0
Assignee: Maor
QA Contact: Elad
URL:
Whiteboard: storage
Depends On:
Blocks: 1126428 rhev3.5beta 1156165
TreeView+ depends on / blocked
 
Reported: 2014-05-05 07:10 UTC by Elad
Modified: 2016-02-10 19:52 UTC (History)
10 users (show)

Fixed In Version: ovirt-3.5.0_rc1
Doc Type: Bug Fix
Doc Text:
Clone Of:
: 1126428 (view as bug list)
Environment:
Last Closed:
oVirt Team: Storage
Target Upstream Version:
Embargoed:
amureini: Triaged+


Attachments (Terms of Use)
logs from engine and host (584.06 KB, application/x-gzip)
2014-05-05 07:10 UTC, Elad
no flags Details


Links
System ID Private Priority Status Summary Last Updated
oVirt gerrit 30988 0 master MERGED core: Add audit log when removing networks from iSCSI bond. Never
oVirt gerrit 31012 0 ovirt-engine-3.5 MERGED core: Add audit log when removing networks from iSCSI bond. Never

Description Elad 2014-05-05 07:10:26 UTC
Created attachment 892439 [details]
logs from engine and host

Description of problem:
When updating an iSCSI multipath bond, engine doesn't send any command to vdsm regarding the update. For example, when removing a network from an iscsi bond, engine doesn't order the hosts in the DC to disconnect one of the paths leading to the storage servers. vdsm still sees the targets via a network which was supposed to be removed from the iscsi multipath bond.

Version-Release number of selected component (if applicable):


How reproducible:
Always

Steps to Reproduce:
On a shared DC with 1 active iSCSI storage domain:
1. Create 2 new networks and attach them to the cluster with required check-box checked
2. Attach the networks to the cluster's hosts NICs 
3. Create a new iSCSI multipath bond (under DC tab -> pick the relevant DC -> iSCSI multipath sub-tab -> new) and add the new networks along with the targets to it.
4. remove one of the networks from the bond


Actual results:
Engine doesn't order the hosts in the DC to update their connections to the storage servers. In order for the changes to take effect, the hosts have to be disconnected from all of their paths to the storage servers by a manual intervention of maintenance and activate to the hosts or the storage domains.

Edit iSCSI bond as appears in engine.log:

2014-05-05 09:58:06,894 INFO  [org.ovirt.engine.core.bll.storage.EditIscsiBondCommand] (ajp-/127.0.0.1:8702-15) [3f8ec4e5] Running command: EditIscsiBondCommand internal: false. Entities affected :  ID: 00000002-0002-0002-0002-00000000030f Type: StoragePool
2014-05-05 09:58:06,910 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-15) [3f8ec4e5] Correlation ID: 3f8ec4e5, Call Stack: null, Custom Event ID: -1, Message: iSCSI bond 'iscsi' was successfully updated.

After the edit, when running 'iscsiadm -m session' on the host, the sessions between the host to the storage servers remain as they were before the update

Expected results:
When updating an existing iscsi multipath bond, the changes have to be sent to the hosts automatically, without the need to disconnect all the paths leading to the storage servers 

Additional info: logs from engine and host

Comment 1 Elad 2014-05-05 07:11:37 UTC
Version-Release number of selected component (if applicable):
AV7
rhevm-3.4.0-0.15.beta3.el6ev.noarch

Comment 2 Sergey Gotliv 2014-05-05 09:22:27 UTC
We discussed this limitation during development cycle, currently RHEV can't disconnect from the target. It will happen automatically after next VDSM restart.
Meantime there is no harm to the system.

Comment 3 Allon Mureinik 2014-05-11 22:13:28 UTC
(In reply to Sergey Gotliv from comment #2)
> We discussed this limitation during development cycle, currently RHEV can't
> disconnect from the target. It will happen automatically after next VDSM
> restart.
> Meantime there is no harm to the system.

Perhaps we should make this issue more visible?
Perhaps an audit log?

Comment 5 Maor 2014-08-03 12:40:14 UTC
Allon, how about the following audit log:

"The following Networks has been removed from the iSCSI bond ${IscsiBondName}: ${NetworkNames}. For the changes to take affect on the hosts, they should be moved to maintenance and get activated again."

Comment 6 Allon Mureinik 2014-08-03 12:57:10 UTC
Minor correction:

"The following Networks has been removed from the iSCSI bond ${IscsiBondName}: ${NetworkNames}. For these changes to take affect on the hosts, they should be moved to maintenance and then activated again."

Comment 7 Eyal Edri 2014-08-04 11:09:11 UTC
these bugs are candidates for z-stream, but not ready yet.
they were not included in 3.4.2 bug tracker [1] for critical bugs by gss,
and out of of scope for the 3.4.2 build.
moving to 3.4.3.


[1] https://bugzilla.redhat.com/show_bug.cgi?id=1123858

Comment 9 Elad 2014-08-10 15:03:14 UTC
When updating an iSCSI bond, the following message is shown to user in audit log:

2014-08-10 17:59:55,955 WARN  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-2) [79a5a61a] Correlation ID: 79a5a61a, Call Stack: null, Custom Event ID: -1, Message: The following Networks has been removed from the iSCSI bond 1: s1. for those changes to take affect, the hosts must be moved to maintenance and activated again

Verified with ovirt-3.5 RC1

Comment 10 Allon Mureinik 2015-02-16 19:12:31 UTC
RHEV-M 3.5.0 has been released, closing this bug.

Comment 11 Allon Mureinik 2015-02-16 19:12:33 UTC
RHEV-M 3.5.0 has been released, closing this bug.


Note You need to log in before you can comment on or make changes to this bug.