Bug 1225984

Summary: iscsistart -N doesn't honor VLAN tag
Product: Red Hat Enterprise Linux 6 Reporter: Jan Stodola <jstodola>
Component: iscsi-initiator-utilsAssignee: Chris Leech <cleech>
Status: CLOSED ERRATA QA Contact: Martin Hoyer <mhoyer>
Severity: high Docs Contact:
Priority: high    
Version: 6.7CC: agrover, bgoncalv, cleech, coughlan, ctatman, dracut-maint-list, harald, jstodola, mhoyer, salmy, sbueno
Target Milestone: rc   
Target Release: 6.8   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: iscsi-initiator-utils-6.2.0.873-15.el6 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-05-10 21:24:04 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1111358, 1179586, 1268411, 1286535    
Attachments:
Description Flags
content of /sys/firmware/ibft/
none
fix iscsistart -N when the iBFT includes a vlan tag
none
fix iscsistart -N when the iBFT includes a vlan tag none

Description Jan Stodola 2015-05-28 15:57:41 UTC
Description of problem:
If a VLAN tag is set in iBFT (tested using iPXE in a virtual machine), "iscsistart -N" doesn't honor this tag and configures the base device instead of the VLAN device.

This is what I got early in dracut shell during boot (booted with the "rdbreak=initqueue" option):

initqueue:/# cat /sys/firmware/ibft/ethernet0/vlan
1234
initqueue:/# ip a
1: lo: <LOOPBACK> mtu 65536 qdisc noop state DOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: ibft0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
    link/ether 52:54:00:12:91:ff brd ff:ff:ff:ff:ff:ff
initqueue:/# iscsistart -N
initqueue:/# ip a
1: lo: <LOOPBACK> mtu 65536 qdisc noop state DOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: ibft0: <BROADCAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 52:54:00:12:91:ff brd ff:ff:ff:ff:ff:ff
    inet 192.168.222.11/24 brd 192.168.222.255 scope global ibft0
initqueue:/#

Also tested when the VLAN interface was created before calling "iscsistart -N", IP address was assigned to the base device:

initqueue:/# ip a
1: lo: <LOOPBACK> mtu 65536 qdisc noop state DOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: ibft0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
    link/ether 52:54:00:12:91:ff brd ff:ff:ff:ff:ff:ff

initqueue:/# ip link add link ibft0 name ibft0.1234 type vlan id 1234
802.1Q VLAN Support v1.8 Ben Greear <greearb>
All bugs added by David S. Miller <davem>

initqueue:/# ip a
1: lo: <LOOPBACK> mtu 65536 qdisc noop state DOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: ibft0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
    link/ether 52:54:00:12:91:ff brd ff:ff:ff:ff:ff:ff
3: ibft0.1234@ibft0: <BROADCAST,MULTICAST,M-DOWN> mtu 1500 qdisc noop state DOWN 
    link/ether 52:54:00:12:91:ff brd ff:ff:ff:ff:ff:ff

initqueue:/# iscsistart -N
8021q: adding VLAN 0 to HW filter on device ibft0

initqueue:/# ip a
1: lo: <LOOPBACK> mtu 65536 qdisc noop state DOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: ibft0: <BROADCAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 52:54:00:12:91:ff brd ff:ff:ff:ff:ff:ff
    inet 192.168.222.11/24 brd 192.168.222.255 scope global ibft0
3: ibft0.1234@ibft0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP 
    link/ether 52:54:00:12:91:ff brd ff:ff:ff:ff:ff:ff

initqueue:/# ip -d link show
1: lo: <LOOPBACK> mtu 65536 qdisc noop state DOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: ibft0: <BROADCAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 52:54:00:12:91:ff brd ff:ff:ff:ff:ff:ff
3: ibft0.1234@ibft0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP 
    link/ether 52:54:00:12:91:ff brd ff:ff:ff:ff:ff:ff
    vlan id 1234 <REORDER_HDR> 

initqueue:/# ip r
192.168.222.1 dev ibft0  scope link 
192.168.222.0/24 dev ibft0  proto kernel  scope link  src 192.168.222.11 
initqueue:/#

Version-Release number of selected component (if applicable):
iscsi-initiator-utils-6.2.0.873-14.el6

How reproducible:
always

Steps to Reproduce:
1. Install system to iSCSI disk
2. configure iBFT with VLAN in iPXE:

vcreate --tag 1234 net0
dhcp net0-1234
set keep-san 1
sanboot iscsi:192.168.222.1:::1:iqn.2009-12.virt.nap:default

3. boot from iSCSI
4. system fails to boot since the VLAN device doesn't have IP address

Actual results:
the VLAN tag is not honored and the base device is configured with IP address, system fails to boot.

Expected results:
the VLAN device has IP address

Additional info:
This issue blocks verification of bug 1111358.

Comment 2 Jan Stodola 2015-05-28 16:12:23 UTC
Created attachment 1031343 [details]
content of /sys/firmware/ibft/

Comment 3 Chris Leech 2015-05-28 20:16:33 UTC
This looks like an issue in dracut in converting ibft entries into network configuration entries in the modified command line.  A vlan device is created, but the IP configuration is tagged to the physical interface.

Comment 4 Chris Leech 2015-05-28 20:22:03 UTC
Actually I'll take this back, I'm so used to relying on dracut to do the network configuration for boot that I ignored the fact that iscsistart does have code to do this - and it doesn't handle VLANs.  There may additionally be a dracut issue.

Comment 5 Chris Leech 2015-06-03 16:53:01 UTC
I don't think this qualifies as a blocker, I'm not sure why it would be needed to verify bz 1111358.

Using iscsistart to configure the networking directly is problematic, as there's no proper handover to NetworkManager after booting.  Dracut provides it's own iBFT parsing and network configuration code, it does not use iscsistart for that.

Jan, can you confirm why you think this is blocking testing of dracut?

Comment 6 Jan Stodola 2015-06-04 10:32:33 UTC
It is needed, because dracut does run "iscsistart -N", see modules.d/95iscsi/iscsiroot (dracut-004-386.el6)

Adding Harald to CC for additional comments.

Comment 7 Chris Leech 2015-06-05 04:25:22 UTC
(In reply to Jan Stodola from comment #6)
> It is needed, because dracut does run "iscsistart -N", see
> modules.d/95iscsi/iscsiroot (dracut-004-386.el6)
> 
> Adding Harald to CC for additional comments.

Thanks, I missed that change in the el6 code.  I'm not sure what it's doing there.

I can change iscsistart to match on vlan devices.  It's simpler if we require that the vlan device should already exist, but it could also create vlans.  There are some paths that match by MAC address where supporting vlans won't happen without additional work, like binding an iface record by MAC.

Comment 11 Chris Leech 2015-06-12 04:37:42 UTC
I have a patch to iscsistart to deal with this, it seems to work for the described test configuration at least.  It follow what I said in comment 7; it only configures vlan devices that already exists, and I believe there to be additional work to deal with things link binding an iface to hwaddress + vlan.

Although changing the dracut iscsi module to not call iscsistart -N works as well, the changes for bug 1111358 work fine if iscsistart -N isn't called at all resulting in changes to the routing table.

Comment 12 Chris Leech 2015-06-12 04:38:31 UTC
Created attachment 1037895 [details]
fix iscsistart -N when the iBFT includes a vlan tag

Comment 17 Jan Stodola 2015-06-16 09:47:33 UTC
Chris, thanks for the packages, here are my results:

First of all, it will be possible to verify bug 1111358 with those changes, since booting from iSCSI using iBFT + VLAN works fine.

I tried to run "iscsistart -N" in the following 4 cases:

1) iBFT with no VLAN configured:
   It works as expected, the ibft0 device is configured according to the settings in iBFT.

2) iBFT + VLAN, the vlan device doesn't exist yet:
   It works as expected, the settings in iBFT are not applied to the base device.

3) iBFT + VLAN, the vlan device exists, but the base device is down:
   iscsistart complains that network is down and fails to configure the vlan interface:

   initqueue:/# ip a
   1: lo: <LOOPBACK> mtu 65536 qdisc noop state DOWN 
       link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
   2: ibft0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
   link/ether 52:54:00:ce:b4:64 brd ff:ff:ff:ff:ff:ff

   initqueue:/# ip link add link ibft0 name ibft0.1234 type vlan id 1234
   802.1Q VLAN Support v1.8 Ben Greear <greearb>
   All bugs added by David S. Miller <davem>

   initqueue:/# ip a
   1: lo: <LOOPBACK> mtu 65536 qdisc noop state DOWN 
       link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
   2: ibft0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
       link/ether 52:54:00:ce:b4:64 brd ff:ff:ff:ff:ff:ff
   3: ibft0.1234@ibft0: <BROADCAST,MULTICAST,M-DOWN> mtu 1500 qdisc noop state DOWN 
       link/ether 52:54:00:ce:b4:64 brd ff:ff:ff:ff:ff:ff

   initqueue:/# iscsistart -N
   iscsistart: Could not bring up netdev ibft0.1234 (err 100 - Network is down)

   initqueue:/# ip a
   1: lo: <LOOPBACK> mtu 65536 qdisc noop state DOWN 
       link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
   2: ibft0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
       link/ether 52:54:00:ce:b4:64 brd ff:ff:ff:ff:ff:ff
   3: ibft0.1234@ibft0: <BROADCAST,M-DOWN> mtu 1500 qdisc noop state DOWN 
       link/ether 52:54:00:ce:b4:64 brd ff:ff:ff:ff:ff:ff
   initqueue:/#

4) iBFT + VLAN, the vlan device exists and the base device is up:
   it works as expected:

   initqueue:/# ip link add link ibft0 name ibft0.1234 type vlan id 1234
   initqueue:/# ip link set dev ibft0 up
   initqueue:/# iscsistart -N
   initqueue:/# ip a
   1: lo: <LOOPBACK> mtu 65536 qdisc noop state DOWN 
       link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
   2: ibft0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
       link/ether 52:54:00:ce:b4:64 brd ff:ff:ff:ff:ff:ff
   3: ibft0.1234@ibft0: <BROADCAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP 
       link/ether 52:54:00:ce:b4:64 brd ff:ff:ff:ff:ff:ff
       inet 192.168.222.12/24 brd 192.168.222.255 scope global ibft0.1234


So, only the case 3) didn't work as I expected.
Chris, is it intentional not to bring up the base device in case 3 and do not configure the vlan interface?

Comment 18 Chris Leech 2015-06-18 00:06:25 UTC
Created attachment 1040182 [details]
fix iscsistart -N when the iBFT includes a vlan tag

v2: bring physical interface up as well as vlan

Comment 19 Chris Leech 2015-06-18 00:08:24 UTC
(In reply to Jan Stodola from comment #17)
> So, only the case 3) didn't work as I expected.
> Chris, is it intentional not to bring up the base device in case 3 and do
> not configure the vlan interface?

Not intentional, I was just aiming for the minimum vlan support to unblock the current dracut use case.  I've updated the patch to bring up the physical interface as well as the vlan.  Updated test RPMs in the same place as before, alongside the first set with version vlan.2

Comment 20 Jan Stodola 2015-06-18 08:09:22 UTC
Retested with iscsi-initiator-utils-6.2.0.873-15.vlan.2.el6, all cases mentioned in comment 17 work as expected.

Comment 21 Bruno Goncalves 2015-06-19 13:39:02 UTC
Tested iscsi-initiator-utils-6.2.0.873-15.vlan.2.el6 in our regression test and it worked well.

Comment 22 Tom Coughlan 2015-06-19 22:24:23 UTC
We are going to move this to 6.8, because we do not want to add risk this late in the release cycle (i.e., the RC build). 

This BZ should enable the Dracut BZ to pass QE testing: 

Bug 1111358 - Add support for iscsiboot with VLAN configured in ibft

(The code for that is already in 6.7.)

This, in turn, should allow Anaconda to add support for this feature:

Bug 831002 - Add support for iscsiboot with VLAN configured in ibft

Comment 26 Martin Hoyer 2015-12-14 08:53:21 UTC
Hi Jan,
could You please test this?
I'm running regression tests now.

Comment 28 Jan Stodola 2016-01-20 11:21:32 UTC
I've retested the cases from comment 17, using iscsi-initiator-utils-6.2.0.873-15.el6, in a virtual machine.
All cases seem to work as I expect:

1) iBFT with no VLAN:
   The settings from iBFT are applied to the device and the device is brought up in case it was down before.

2) iBFT + VLAN, the vlan device doesn't exist:
   The iBFT settings are not applied to the base device and iscsistart warns that no matching vlan device was found.

3) iBFT + VLAN, the vlan device exists, but the base device is down:
   The iBFT settings are applied to the VLAN interface and both the base and the VLAN interfaces are brought up. 

4) iBFT + VLAN, the vlan device exists and the base device is up:
   The iBFT settings are applied to the VLAN interface and the device remains up

Comment 29 Martin Hoyer 2016-01-20 12:07:12 UTC
(In reply to Jan Stodola from comment #28)
Thanks Jan for testing this.
Regression tests did not found any new issue.

Comment 31 errata-xmlrpc 2016-05-10 21:24:04 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-0833.html