Bug 726387

Summary: [enic] VLAN-tagged frames received as untagged if tagged sub-interface not present
Product: Red Hat Enterprise Linux 6 Reporter: Tore Anderson <tore>
Component: kernelAssignee: Stefan Assmann <sassmann>
Status: CLOSED WORKSFORME QA Contact: Red Hat Kernel QE team <kernel-qe>
Severity: high Docs Contact:
Priority: unspecified    
Version: 6.1CC: lexer.wu, michael.hagmann, tore
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-03-11 11:57:28 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Attachments:
Description Flags
Output from "lspci -nnvvv"
none
"tcpdump -i eth0 -s 0 -c 10 -w ip6-precise.pcap ip6" on Ubuntu 12.04
none
"tcpdump -i eth0 -s 0 -c 10 -w ip6-rhel.pcap ip6" on RHEL 6.5
none
enic.patch none

Description Tore Anderson 2011-07-28 13:24:37 UTC
Description of problem:

When a virtual NIC in the Cisco UCS system set up with a native VLAN and a tagged one, and there's no VLAN-tagged sub-interfaces configured in the operating system (i.e. added manually with "ip link" or "vconfig"), the frames on the tagged VLAN are seen by the Linux kernel as untagged received straight on the main (untagged) interface eth0.

In my case I have ICMPv6 Router Advertisements on the tagged VLAN. Since these are incorrectly received by the main interface eth0, IPv6 addresses and an IPv6 default route is incorrectly configured straight on the main interface. This does of course not work, as any egress packets transmitted by the Linux host enters the native/untagged VLAN instead.

If I configure a VLAN-tagged interface, either with "vconfig add eth0 <tag>" or "ip link add link eth0 name foo type vlan id <tag>", the frames on the tagged VLAN instantly ceases to be visible as untagged frames in a tcpdump on the main interface eth0. If the VLAN-tagged interface I add has a different tag value than the tagged VLAN has in the UCS fabric, frames on this VLAN does not appear in the tcpdump on the main interface at all. If the tags match, I can see that the frames are received correctly with a 802.1Q VLAN header.


Version-Release number of selected component (if applicable):

Red Hat Enterprise Linux Server release 6.1 (Santiago)
kernel-2.6.32-131.6.1.el6.x86_64
enic: Cisco VIC Ethernet NIC Driver, ver 2.1.1.13


How reproducible:

100%


Steps to Reproduce:
1. Configure a Virtual NIC interface on a UCS server as eth0, having two VLANs active, of which one should be marked as the native VLAN. Ensure that the tagged VLAN has some broadcast traffic going on which is easily identifiable from any broadcast traffic on the native VLAN. IPv6 Router Advertisements are perfect as they will also mainfest themselves as bogus addresses/routes showing up, too. (In this example, the non-native VLAN has 802.1Q tag 116.)

2. Start a "tcpdump -i eth0 -e" on the UCS server.

3. Observe how the broadcast traffic on the tagged VLAN shows up in the tcpdump, while *not* being marked with «vlan 116» in the tcpdump output. In the case of ICMPv6 RAs, observe how an IPv6 address and default route has been auto-configured on eth0.

4. Add a bogus VLAN interface with "ip link add link eth0 name foo type vlan id 1024" or "vconfig add eth0 1024"

5. Observe how the broadcast traffic on the tagged VLAN no longer show up in the tcpdump output.

6. Delete the VLAN interface ("ip link del foo" or "vconfig rem eth0.1024")

7. (Like #3)

8. Add a matching VLAN interface "ip link add link eth0 name foo type vlan id 116" or "vconfig add eth0 116"

9. Observe in the tcpdump output how the packets on the tagged vlan now is marked with «vlan 116», and how ICMPv6 RAs received do not cause any addresses/routes to show up on eth0 (they will however correctly show up on the tagged interface foo/eth0.116 if it is additionally set to UP).
  

Actual results:

See above.


Expected results:

In step #3/#7 in the procedure above, the frames on the tagged VLAN should either not show up in the tcpdump at all (assuming filtering done by hardware, as appears to be the case in step #5), or they should show up marked with their correct 802.1Q tag (as in step #9).


Additional info:

Comment 2 RHEL Program Management 2011-10-07 15:42:49 UTC
Since RHEL 6.2 External Beta has begun, and this bug remains
unresolved, it has been rejected as it is not proposed as
exception or blocker.

Red Hat invites you to ask your support representative to
propose this request, if appropriate and relevant, in the
next release of Red Hat Enterprise Linux.

Comment 3 Weiping Pan 2013-12-10 07:59:15 UTC
Hi, Tore,

Since we do not have Cisco UCS system,
I can not reproduce this bug.

Can you try rhel6.5 and upstream to see if this bug still exists ?

thanks
Weiping Pan

Comment 4 Tore Anderson 2013-12-11 08:29:20 UTC
I will try do to so, although I doubt I will find time to do so this side of the Christmas holidays.

Tore

Comment 5 Tore Anderson 2014-01-10 12:01:15 UTC
Ok, I've now confirmed that the bug is still ocurring in RHEL 6.5,  kernel-2.6.32-431.3.1.el6.x86_64 (enic 2.1.1.39).

Tore

Comment 7 Stefan Assmann 2014-02-11 13:15:53 UTC
Tore,

would you please supply the output of "lspci -nnvvv"
Thanks!

Comment 8 Tore Anderson 2014-02-14 08:51:16 UTC
Created attachment 863180 [details]
Output from "lspci -nnvvv"

Here's the output you requested.

I've also noticed that this bug is *not* present in Ubuntu 12.04. I therefore suspect that this bug has been resolved in a newer upstream version of the enic driver. I will attach two PCAPs that demonstrate the difference beween RHEL 6.5 and Ubuntu 12.04 shortly.

I've also noticed that the bug only occurs when the interface is set in promiscuous mode (so running tcpdump is one way to trigger it).

I can arrange for log-in to the a test-blade where the bug can be reproduced freely, if you want. If you would like that, please send me your SSH public key.

Tore

Comment 9 Tore Anderson 2014-02-14 08:54:30 UTC
Created attachment 863182 [details]
"tcpdump -i eth0 -s 0 -c 10 -w ip6-precise.pcap ip6" on Ubuntu 12.04

Details of the Ubuntu server is as follows:

root@ucstest:~# lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 12.04.4 LTS
Release:	12.04
Codename:	precise
root@ucstest:~# uname -r
3.2.0-58-generic

Notably, the captured packets all include the 802.1Q VLAN header. Therefore, no IPv6 addressing or routing gets auto-configured on eth0.

Comment 10 Tore Anderson 2014-02-14 09:02:13 UTC
Created attachment 863184 [details]
"tcpdump -i eth0 -s 0 -c 10 -w ip6-rhel.pcap ip6" on RHEL 6.5

This PCAP is taken on the exact same system as the one in the previous comment, only that it has been re-installed with RHEL. There have been no changes whatsoever to the hardware, network configuration, UCSM service profile, or anything else - the environment is identical.

Notably, the 802.1Q VLAN tag is missing from the captured packets. Therefore, they cause the kernel to auto-configure IPv6 addressing and routing on the eth0 interface, even though the eth0 interface is attached to an untagged VLAN which has no IPv6 connectivity. The IPv6 connectivity included in the RA would only be usable on the VLAN-tagged interface "eth0.1908" (had it been configured).

OS details:

[root@ucstest ~]# lsb_release -a
LSB Version:	:base-4.0-amd64:base-4.0-noarch:core-4.0-amd64:core-4.0-noarch:graphics-4.0-amd64:graphics-4.0-noarch:printing-4.0-amd64:printing-4.0-noarch
Distributor ID:	RedHatEnterpriseServer
Description:	Red Hat Enterprise Linux Server release 6.5 (Santiago)
Release:	6.5
Codename:	Santiago
[root@ucstest ~]# uname -r
2.6.32-431.5.1.el6.x86_64

Comment 11 Stefan Assmann 2014-02-17 07:33:37 UTC
Created attachment 863972 [details]
enic.patch

Tore,
could you test the attached patch and report back whether it fixes your issue.
Thanks!

Comment 12 Tore Anderson 2014-02-17 08:25:06 UTC
Confirmed, this patch solves the problem. Now I can see the 802.1Q VLAN tag when running the exact same tcpdump command as in comment #10, and as a result no defective IPv6 connectivity gets configured on the untagged eth0 interface.

Tore

Comment 13 Stefan Assmann 2014-02-17 08:56:36 UTC
Thanks for the feedback. The fix is targeted at RHEL6.6 as well as RHEL6.5.z.

Comment 14 Stefan Assmann 2015-03-11 11:57:28 UTC
Fix is part of RHEL6.6. Closing.