Bug 436435 - NETWORK DEVICE ORDER CHANGES
Summary: NETWORK DEVICE ORDER CHANGES
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: initscripts
Version: 5.1
Hardware: All
OS: Linux
low
low
Target Milestone: rc
: ---
Assignee: initscripts Maintenance Team
QA Contact: Brock Organ
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2008-03-07 07:02 UTC by Joe.Jin
Modified: 2008-05-06 03:13 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2008-03-07 18:35:05 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
patch file (8.70 KB, text/plain)
2008-03-07 07:02 UTC, Joe.Jin
no flags Details

Description Joe.Jin 2008-03-07 07:02:28 UTC
Component version:
    initscripts-8.45.16.EL-1.1.8
Description of the problem:
  Our target machines have 2 on-board NIC (bnx2) and 2 dual/quad Intel
NIC(e1000).
  By EL4 U5 and "installer" of OVS/EL5 maps bnx2 interfaces to eth[01].
  But once OVS/EL5 was installed, this order may change boot by boot.
  Sometime bnx2 may become the last 2 interface and other time may be in the
  middle. (e.g. eth0:e1000, eth[12]:bnx2, eth[3-]:e1000).
  We want to make sure to load bnx2 driver before e1000 is loaded.

How to reproduce:
  Reboot machine some times.
  Easy to reproduce:
    At /sbin/start_udev, add "modprobe eth[x]". 
    the x should will be corresponding to difference driver.
    For example, modprobe.conf have following content:
         alias eth0 e1000
         alias eth1 e1000
         alias eth2 bnx2
         alias eth3 bnx2
    then at /sbin/start_udev, add "modprobe eth0", then will get nic orders as
    defined if add "modprobe eth3", will get wrong orders.

Analisis:
    At AS5, there is not checking for modprobe.conf while boot startup,
    all modules loaded by udevd, so modprobe.conf just used for modprobe,
    mkinitrd but udevd, hwaddr bind is a solutions to deal with the issue
    however, while swap device, need rewrite udev rules or reconfig 
    ifcfg-eth[x] to rebind it, if have many server do that, will take
    a long time to do that.

I have create a patch for this issue, it will checking modprobe.conf
at rename_device -- it will execute when a new interface register. by
my testing looked worked fine.

Comment 1 Joe.Jin 2008-03-07 07:02:28 UTC
Created attachment 297142 [details]
patch file

Comment 2 Bill Nottingham 2008-03-07 18:35:05 UTC
If you want consistent naming, put HWADDR in your configuration files.

Comment 3 Joe.Jin 2008-03-10 01:18:13 UTC
(In reply to comment #2)
> If you want consistent naming, put HWADDR in your configuration files.

yes it also a solution, however, but when swap NIC to others, 
must redo that, if a batch of servers have to do that
it will take a long time to do.


Comment 4 Bill Nottingham 2008-03-10 17:09:35 UTC
modprobe.conf does not contain enough information to do reliable mapping -
that's why HWADDR is what is required.

Comment 5 Joe.Jin 2008-03-11 02:06:35 UTC
(In reply to comment #4)
> modprobe.conf does not contain enough information to do reliable mapping -
> that's why HWADDR is what is required.

Now modprobe.conf just provided alias of device driver, it used for 
modprobe.
But if no HWADDR option configured, it means OS would not promisee 
keep the devices's order, even at modprobe.conf have configured the 
corresponding alias. when the devices used difference switch, it maybe
changed OS's action.

At this patch, it will checking modprobe.conf and ifcfg-eth[x], if HWADDR
option have configured, it will re-allocate the device name as have defined
at ifcfg-eth[x] if necessary, or it will re-allocate devices name as 
modprobe.conf have defined. while mutil-device used same driver, it will 
allocate device name as the order as pci-id. 
So it means even without HWADDR option configured, OS will have the same 
name's order at anytime.


Comment 6 John Sobecki 2008-03-11 22:52:44 UTC
Here is the Business Justification:

Consider if you had a data center with 5000 servers.  One goes bad; you roll in
a replacement.  Upon restart you'll either have 2 problems, 1) NICs in assigned
in wrong order or, 2) NICs won't come online due to stale HWADDR info in the
ifcfg files.  

You won't get a chance to change HWADDRs in advance, if it's a CPU/Memory/root
fs disk failure.  Preserving the modprobe preference seemed like a reasonable
enhancement to udev.  Other ideas?   Thanks, John

Comment 7 Bill Nottingham 2008-03-11 23:00:37 UTC
modprobe *is not enough information* to be reliable, especially in the case
where you swap multiple cards at once. The reason we do HWADDR only is because
attempts to use modprobe aliases was done in the past. It didn't work.

Comment 8 Harald Hoyer 2008-03-12 16:58:42 UTC
HWADDR is "by-id" .. what you may need is "by-path"

RHEL currently has no mechanism to identify network cards by the e.g. pci-slot
number


Note You need to log in before you can comment on or make changes to this bug.