Re-targeting to 4.2.6 being next build blockers only and this not being considered a blocker for 4.2.5.
Change is already in
I've tried to separate iSCSI traffic over iSCSI bond and got all interfaces participating in iSCSI sessions. Instead of taking all iSCSI traffic over enp3s0f1, traffic was distributed over default interface enp3s0f0 and enp3s0f1 in this pattern: alma04 ~]# iscsiadm -m session -P1 Target: iqn.2008-05.com.xtremio:xio00153500071-514f0c50023f6c01 (non-flash) Current Portal: 10.35.146.161:3260,1 Persistent Portal: 10.35.146.161:3260,1 ********** Interface: ********** Iface Name: default Iface Transport: tcp Iface Initiatorname: iqn.1994-05.com.redhat:20179eabd3d Iface IPaddress: 10.35.92.4 Iface HWaddress: <empty> Iface Netdev: <empty> SID: 1 iSCSI Connection State: LOGGED IN iSCSI Session State: LOGGED_IN Internal iscsid Session State: NO CHANGE Target: iqn.2008-05.com.xtremio:xio00153500071-514f0c50023f6c00 (non-flash) Current Portal: 10.35.146.129:3260,1 Persistent Portal: 10.35.146.129:3260,1 ********** Interface: ********** Iface Name: default Iface Transport: tcp Iface Initiatorname: iqn.1994-05.com.redhat:20179eabd3d Iface IPaddress: 10.35.92.4 Iface HWaddress: <empty> Iface Netdev: <empty> SID: 2 iSCSI Connection State: LOGGED IN iSCSI Session State: LOGGED_IN Internal iscsid Session State: NO CHANGE Target: iqn.2008-05.com.xtremio:xio00153500071-514f0c50023f6c04 (non-flash) Current Portal: 10.35.146.193:3260,1 Persistent Portal: 10.35.146.193:3260,1 ********** Interface: ********** Iface Name: default Iface Transport: tcp Iface Initiatorname: iqn.1994-05.com.redhat:20179eabd3d Iface IPaddress: 10.35.92.4 Iface HWaddress: <empty> Iface Netdev: <empty> SID: 3 iSCSI Connection State: LOGGED IN iSCSI Session State: LOGGED_IN Internal iscsid Session State: NO CHANGE Target: iqn.2008-05.com.xtremio:xio00153500071-514f0c50023f6c05 (non-flash) Current Portal: 10.35.146.225:3260,1 Persistent Portal: 10.35.146.225:3260,1 ********** Interface: ********** Iface Name: enp3s0f1 Iface Transport: tcp Iface Initiatorname: iqn.1994-05.com.redhat:20179eabd3d Iface IPaddress: 10.35.71.93 Iface HWaddress: <empty> Iface Netdev: enp3s0f1 SID: 4 iSCSI Connection State: LOGGED IN iSCSI Session State: LOGGED_IN Internal iscsid Session State: NO CHANGE I can't verify this bug, due to the fact that its still impossible to separate iSCSI traffic from flowing over management interface. Tested on these components on hosts: rhvm-appliance-4.2-20180727.1.el7.noarch, Engine setup within the rhvm-appliance-4.2-20180727.1.el7.noarch is ovirt-engine-setup-4.2.5.2-0.1.el7ev.noarch. ovirt-hosted-engine-ha-2.2.16-1.el7ev.noarch ovirt-hosted-engine-setup-2.2.24-1.el7ev.noarch rhvm-appliance-4.2-20180727.1.el7.noarch ovirt-vmconsole-1.0.4-1.el7ev.noarch ovirt-vmconsole-host-1.0.4-1.el7ev.noarch Linux 3.10.0-862.el7.x86_64 #1 SMP Wed Mar 21 18:14:51 EDT 2018 x86_64 x86_64 x86_64 GNU/Linux Red Hat Enterprise Linux Server release 7.5 (Maipo) ------------- Summary: ------------- 1.At present, there is no way to completely separate HE's storage traffic between default management network and storage traffic dedicated iSCSI bonded networks. 2.There is no way to detach default management network interface from HE storage target without loosing HE-VM on hosts. 3.There is no way to choose during HE's deployment stage, which multiple interfaces should be used for MPIO (aka iSCSI Bond). 4.There is no prediction on which of the networks "iscsi dedicated network" or the default management network, the HE's storage traffic will be forwarded,once there were created separate storage dedicated networks for HE's storage domain iSCSI target. Due to the fact, that this RFE does not cover the customer requirements, I'm moving it back to assigned.
According to vdsm logs, ovirt-ha-agent sent: 2018-08-01 01:09:08,345+0300 INFO (jsonrpc/0) [vdsm.api] START connectStorageServer(domType=3, spUUID=u'00000000-0000-0000-0000-000000000000', conList=[{u'netIfaceName': u'enp3s0f1', u'id': u'59ead077-fcbc-4cd5-85f5-ddf2a88c7f6c', u'connection': u'10.35.146.225', u'iqn': u'iqn.2008-05.com.xtremio:xio00153500071-514f0c50023f6c05', u'user': u'', u'tpgt': u'1', u'ifaceName': u'enp3s0f1', u'password': '********', u'port': u'3260'}], options=None) from=::1,49818, task_id=2c39a45f-8f8e-4f86-8ef5-b757634e56b0 (api:46) 2018-08-01 01:09:09,666+0300 INFO (jsonrpc/0) [vdsm.api] FINISH connectStorageServer return={'statuslist': [{'status': 0, 'id': u'59ead077-fcbc-4cd5-85f5-ddf2a88c7f6c'}]} from=::1,49818, task_id=2c39a45f-8f8e-4f86-8ef5-b757634e56b0 (api:52) and the default interface is not in the connection list. The connectStorageServer are not that different from what the engine sent. 2018-07-31 17:43:33,035+0300 INFO (jsonrpc/0) [vdsm.api] START connectStorageServer(domType=3, spUUID=u'ff687f1e-93ea-11e8-a9b0-00163e7bb853', conList=[{u'netIfaceName': u'enp3s0f1', u'id': u'4b132b21-bad7-45ff-9006-0b3a69ac0bd4', u'connection': u'10.35.146.225', u'iqn': u'iqn.2008-05.com.xtremio:xio00153500071-514f0c50023f6c05', u'user': u'', u'tpgt': u'1', u'ifaceName': u'enp3s0f1', u'password': '********', u'port': u'3260'}], options=None) from=::ffff:10.35.92.51,37436, flow_id=438f4f82, task_id=e65f0604-d2fc-4440-87f9-e7e6bde45097 (api:46) 2018-07-31 17:43:34,823+0300 INFO (jsonrpc/0) [vdsm.api] FINISH connectStorageServer return={'statuslist': [{'status': 0, 'id': u'4b132b21-bad7-45ff-9006-0b3a69ac0bd4'}]} from=::ffff:10.35.92.51,37436, flow_id=438f4f82, task_id=e65f0604-d2fc-4440-87f9-e7e6bde45097 (api:52)
While the host is in maintenance mode a sequence of iscsiadm -m node -u iscsiadm -m node -I default --op=delete brings to the desidered result and ovirt-ha-broker will simply use the iSCSI bonding as configured by the engine with expected results. I fear that the issue is simply that the engine/vdsm never calls (or maybe it just fails because the ovirt-ha-broker keeps the iscsi sessions up?) iscsiadm -m node -I default --op=delete re-configuring the iSCSI bonding on the host and so all the bindings between iSCSI interfaces and portals used in the past will remain on the host.
Let's close this with current state and let's add another patch to automatically configure iscsi bonding from setup as for 1613763
Dear Simone, Do I understand correctly sessions default interface is legacy from initial configuration? I have situation with multiple path's duplicated originated from default interface. Can I remove them without impact, just leaving my 2 NICs in Multipath Bonding, right? they do not carry any service traffic or etc?
Yes, you should be able to do that.
Ok, clear. Thank you, Simone! Also one question. Is it mandatory to have both iSCSI Bonding members in same L3 network and same broadcast domain? I noticed that oVirt creates the file "vdsm.conf" under /etc/sysctl.d, with these contents net.ipv4.conf.default.arp_ignore = 1 net.ipv4.conf.default.arp_announce = 2 This indirectly gives me a sign that this is done for purpose of having IP addresses from same L3 subnet on iSCSI Bonding member interfaces. But can I join interfaces to iSCSI Bonding, which are members of different L3 subnets and having different broadcast domains? For example eth0 with 192.168.0.0/24 and eth1 with 192.168.1.0/24, connected to different switches. I do not find in RH and oVirt documentation straightforward answer if members of iSCSI Bonding can be splitted across different L3 subnets, not routed between each other. I would be very grateful for clarification.
AFAIK we're not limiting this to the same L3 or L2 network. * arp_ignore value set to 1 so that it will only reply to ARP requests on the interface which contains the target IP address. This acts much more like a traditional router and provides simplicity in troubleshooting and operation. ** By setting the arp_announce parameter to 2, Linux uses the best local address for each ARP request, preferring primary addresses on the interface used to send the ARP. This most closely matches traditional router ARP request behavior.If no suitable local address is found we select the first local address we have on the outgoing interface or on all other interfaces, with the hope we will receive reply for our request and even sometimes no matter the source IP address we announce. In general ARPs are working over L2. I see no issue using different unrouted L3 networks inside separate L2 domains, as long as they can reach all relevant iSCSI storage targets just like other source NICs do. https://www.ovirt.org/develop/release-management/features/storage/iscsi-multipath.html "The User selects logical networks and iSCSI targets. The iSCSI bond must contain at least one logical network related to the Data Center."