Bug 656059

Summary: [Cisco 6.1 bug] udev wrongly renaming ethernet devices with zero macs
Product: Red Hat Enterprise Linux 6 Reporter: roprabhu
Component: udevAssignee: Harald Hoyer <harald>
Status: CLOSED ERRATA QA Contact: qe-baseos-daemons
Severity: high Docs Contact:
Priority: high    
Version: 6.0CC: azelinka, cward, jolsa, kvolny, nhorman, pknirsch, tgraf
Target Milestone: rcKeywords: OtherQA
Target Release: 6.1   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Some dynamic ethernet interfaces with zero MAC addresses were erroneously added to the list of persistent interfaces in /etc/udev/rules.d/70-persistent-net.rules. This resulted into wrongly named network interfaces. This udev update should fix the issue.
Story Points: ---
Clone Of: Environment:
Last Closed: 2011-05-19 11:50:22 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 580566    

Description roprabhu 2010-11-22 23:34:06 UTC
This is seen with enic devices on RHEL 6.0. All enic dynamic devices are created with all-zero mac addresses. 

- udev on first boot...creates almost similar entries for all enic dynamic devices. The change is only the device NAME attribute

#cat /etc/udev/rules.d/70-persistent-net.rules 

# PCI device 0x1137:0x0044 (enic)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:00:00:00:00:00", ATTR{type}=="1", KERNEL=="eth*", NAME="eth11"

# PCI device 0x1137:0x0044 (enic)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:00:00:00:00:00", ATTR{type}=="1", KERNEL=="eth*", NAME="eth15"

# PCI device 0x1137:0x0044 (enic)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:00:00:00:00:00", ATTR{type}=="1", KERNEL=="eth*", NAME="eth18"

# PCI device 0x1137:0x0044 (enic)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:00:00:00:00:00", ATTR{type}=="1", KERNEL=="eth*", NAME="eth31"


- During second boot, udev refers to the above persistent config file. For every enic dynamic device add  event it gets from kernel it matches the first rule in the persistent config file.
Since its different from the last one it renames the interface.
So all dynamic vnics except the first one get renamed. 
Something like this:
eth0  /* static enic. Non zero mac. Device name wont change */
eth1 /* static enic. Non zero mac. Device name wont change */
eth2 /* dynamic enic. matches first rule. Last time was eth2 so wont change */
eth3 /* dynamic enic. matches first rule. Last time was eth2, gets renamed as eth3-eth2 */
eth4 /* dynamic enic. matches first rule. Last time was eth2, gets renamed as eth4-eth2 */

It affects all VM's using these dynamic vnics in libvirt xml file for direct attach. Because after a reboot of the hypervisor all interface names are changed.

Is this expected behavior ?. We seem to be seeing it only in the last few snapshots of RHEL 6.0. 
Looks like something was introduced recently or something is broken.

we have worked around it by adding a rule in /lib/udev/rules.d/75-persistent-net-generator.rules to not generate persistent config entries for enic devices 

Version-Release number of selected component (if applicable):


How reproducible: consistently. On second boot.


Steps to Reproduce:
1. Deploy enic dynamic devices on RHEL 6.0 using UCSM
2. ifconfig -a will show all enic dynamic interfaces with zero mac addresses
3. reboot the host
4. All enic dynamic interfaces will get renamed  
  
Actual results:
ifconfig -a
eth2-eth10 Link encap:Ethernet  HWaddr 00:00:00:00:00:00  
          BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

eth3-eth10 Link encap:Ethernet  HWaddr 00:00:00:00:00:00  
          BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

eth4-eth10 Link encap:Ethernet  HWaddr 00:00:00:00:00:00  
          BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)



Expected results:
eth2  Link encap:Ethernet  HWaddr 00:00:00:00:00:00  
          BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

eth3  Link encap:Ethernet  HWaddr 00:00:00:00:00:00  
          BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

eth4  Link encap:Ethernet  HWaddr 00:00:00:00:00:00  
          BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

Additional info:

Comment 2 roprabhu 2010-12-11 05:05:28 UTC
Can the fix for this please be made available in RHEL 6.1 ? Thanks, Roopa

Comment 3 Andrius Benokraitis 2010-12-14 18:20:16 UTC
Roopa, you *gotta* let us know of bugzillas you create. Today is the first time Gary and I saw this, and we don't get automatically notified to drive these.

Comment 4 roprabhu 2010-12-14 18:31:30 UTC
ok sorry. I had kept chris wright in the loop for all these. Just forwarded you and gary an email with all new bugzilla's. Will keep you in the loop hereon. Thanks

Comment 5 Phil Knirsch 2010-12-17 14:08:43 UTC
Looking at the information in the description of the bug i don't see a unique way to differentiate the the enics:

# PCI device 0x1137:0x0044 (enic)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*",
ATTR{address}=="00:00:00:00:00:00", ATTR{type}=="1", KERNEL=="eth*",
NAME="eth11"

# PCI device 0x1137:0x0044 (enic)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*",
ATTR{address}=="00:00:00:00:00:00", ATTR{type}=="1", KERNEL=="eth*",
NAME="eth15"

# PCI device 0x1137:0x0044 (enic)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*",
ATTR{address}=="00:00:00:00:00:00", ATTR{type}=="1", KERNEL=="eth*",
NAME="eth18"

# PCI device 0x1137:0x0044 (enic)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*",
ATTR{address}=="00:00:00:00:00:00", ATTR{type}=="1", KERNEL=="eth*",
NAME="eth31"


All PCI device ids are the same, all have 00:00:00:00:00:00 as hw address and no other information there.

Harald is on PTO till next year though, but maybe he has an idea how to address this.

Thanks & regards, Phil

Comment 6 roprabhu 2010-12-20 14:43:10 UTC
yes thats correct. These devices are not initialized at this point. They get a unique mac address when they are initialized via a port profile. The idea is these devices get an identity only when a client like libvirt or iproute wants to use it and initializes it using a port profile (802.1Qbh).
THese devices get deinitialized and mac address resets to zero when the client on the host disassociates the port profile applied to them. This happens also on a reboot or a reload of the enic driver.

Since the mac address of these devices is dynamic, we were wondering if udev can ignore these devices ?.
We were hoping udev could ignore them based on the all zero mac address or by the pci id.

Thanks phil.

Comment 7 Harald Hoyer 2011-01-03 08:26:41 UTC
Are there any /etc/sysconfig/network-scripts/ifcfg-eth* files for the zero-mac interfaces?

Comment 8 roprabhu 2011-01-03 16:16:17 UTC
No. Usually the dynamic interfaces dont have ifcfg-* files. But i have occasionally seen a ifcfg file created for the first dynamic vnic on my test setup.

Comment 13 Harald Hoyer 2011-01-26 11:17:59 UTC
(In reply to comment #8)
> No. Usually the dynamic interfaces dont have ifcfg-* files. But i have
> occasionally seen a ifcfg file created for the first dynamic vnic on my test
> setup.

If you have one of these:
# fgrep 'HWADDR=00:00:00:00:00:00' /etc/sysconfig/network-scripts/ifcfg-*

Then the fix would be:
- remove all HWADDR=00:00:00:00:00:00 from /etc/sysconfig/network-scripts/ifcfg-*
- remove all rules with ATTR{address}=="00:00:00:00:00:00" from /etc/udev/rules.d/70-persistent-net.rules

Comment 15 Harald Hoyer 2011-02-16 14:18:43 UTC
    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
Some dynamic ethernet interfaces with zero MAC addresses were erroneously added to the list of persistent interfaces in /etc/udev/rules.d/70-persistent-net.rules. This resulted into wrongly named network interfaces. This udev update should fix the issue.

Comment 16 Chris Ward 2011-04-06 11:07:28 UTC
~~ Partners and Customers ~~

This bug was included in RHEL 6.1 Beta. Please confirm the status of this request as soon as possible.

If you're having problems accessing 6.1 bits, are delayed in your test execution or find in testing that the request was not addressed adequately, please let us know.

Thanks!

Comment 17 roprabhu 2011-04-06 18:34:23 UTC
we have verified this on RHEL 6.1 snapshot 1. It looks fine. Thanks!.

Comment 19 errata-xmlrpc 2011-05-19 11:50:22 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHBA-2011-0525.html