Bug 611418

(In reply to comment #0)
> Actual results:
> passthrough/hotplug VF to guest, log in guest, lspci will show the device, but
> it cannot get ip address with no link error, but the device is carried
> actually.

Is the ethX associated with the PF up?  I think you'll find the same is true on the host if you try to use an ethX associated with a VF and the ethX for the PF is down.  On my system with 82576 NICs, I have network entries like this to bring the PF interface up, but not assign an IP, then the VF NICs work fine:

# cat /etc/sysconfig/network-scripts/ifcfg-eth1
DEVICE=eth1
HWADDR=00:1B:21:60:5C:34
ONBOOT=yes
BOOTPROTO=static
NM_CONTROLLED=no

> Expected results:
> Network configuration should list the right info after modprobe igb and could
> work well with VF passthrough/hotplug.

What is actually wrong with the network configuration between steps 1 & 3?

FYI, I confirmed with an Intel contact that the PF NIC must be up in order for the VF NIC to receive packets.  AIUI, even numbered VFs are associated with PF 0, odd number VFs are associated with PF 1.  Please retest with PF NICs up and explain in more detail if there is still a problem with the ip link configuration.

Sorry for the uncertainty issue description.

step 1:
2: **eth1**: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state **UP** qlen
1000
    link/ether **00:1b:21:39:8b:18** brd ff:ff:ff:ff:ff:ff

--passthrough/hotplug PF to guest and get IP successfully. 

step 2: 
# modprobe -r igb
# modprobe igbvf
# modprobe igb max_vfs=1

--after step 2, the state changed from 'UP' to 'DOWN' and the 'eth1' changed to 'eth2_rename'.

step 3:
8: **eth2_rename**: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state **DOWN** qlen
1000
    link/ether **00:1b:21:39:8b:18** brd ff:ff:ff:ff:ff:ff

--No matter I passthrough/hotplug PF or VF, it cannot get ip address with no link error.

Ok, thanks for the clarification.  It looks like there are some problems with VFs claiming names that we try to save for PFs via udev.  Adding Stefan Assmann to this bz since I saw a recent patch from him attempting to fix this.  Stefan, if this bz is a duplicate of one you're already working on, please let us know the number.

I can suggest a couple workarounds since this is marked as a test blocker:

1) You don't need to load igbvf if you only intend to use the VFs as pass through devices for KVM guests.  You can do this by blacklisting the driver:

echo blacklist igbvf >> /etc/modprobe.d/blacklist.conf

Now if you unload igbvf, all those extra ethX interfaces go away and won't come back if you reload igb.

2) You can create a udev rule to make sure the eth devices created for igbvf don't conflict with those for PF devices.  Here's one example, add this to /etc/udev/rules.d/70-persistent-net.rules:

SUBSYSTEM=="net", ACTION=="add", DRIVERS=="igbvf", KERNEL=="eth*", NAME="eth%b"

Now your VF eth devices will show up with names that match the PCI function they're associated with.  ip link will show things like:

175: eth0000:01:10.0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP ioctl(SIOCGIFXQLEN) failed: No such device

    link/ether 62:7c:0c:8b:cf:9b brd ff:ff:ff:ff:ff:ff

Either of these should be a sufficient work around to test with, also remember to make sure the PF eth device is UP before expecting packets to be seen in a VF assigned to a guest.

Hi Alex,

thanks for letting me know. I had opened a bug for this problem against udev. But that got closed because it's not a really a udev bug. So let's use this bug to keep track of it.

Both your workarounds are fine for now. As you already mentioned I'm trying to get a fix accepted upstream.

link to the upstream discussion http://patchwork.ozlabs.org/patch/57383/

Test without modprobe igbvf, hotplug and passthrough can be done successfully.
So remove the TestBlocker.

This problem is currently being solved in bug #614786, I'm not going to close this as a dup to make sure it doesn't get lost. I'll update this bug as soon as we have agreed on a solution so that you can verify your problem has been addressed properly.

While I'm pushing a patch upstream could you try the following and see if it helps to solve your problem:

- install udev packages from
http://people.redhat.com/harald/downloads/udev/udev-147-2.21.0.hh.24.el6
- Remove /etc/udev/rules.d/70-persistent-net.rules
- try again
(Beware of bug 614943 and make sure DEVICE= has no "" in the ifcfg files.)

Upgrade udev from https://brewweb.devel.redhat.com/buildinfo?buildID=138203
udev-147-2.22.el6
kernel-2.6.32-44.2.el6
The network configuration is not messed up after 'modprobe igbvf'.

1. # mv 70-persistent-net.rules /tmp/

2. remove "" from ifcfg files
# cat ifcfg-eth0 
DEVICE=eth0
BOOTPROTO="dhcp"
HWADDR="00:1B:21:39:8B:18"
NM_CONTROLLED="yes"
ONBOOT="yes"

3. modprobe igbvf

ip link configuration looks fine and all eth* can get ip successfully.
#ip link
17: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
    link/ether 00:1b:21:39:8b:18 brd ff:ff:ff:ff:ff:ff
    vf 0 MAC c2:06:2d:2d:01:f9
18: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
    link/ether 00:1b:21:39:8b:19 brd ff:ff:ff:ff:ff:ff
    vf 0 MAC 3a:25:28:49:01:c3
19: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether c2:06:2d:2d:01:f9 brd ff:ff:ff:ff:ff:ff
20: eth3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 3a:25:28:49:01:c3 brd ff:ff:ff:ff:ff:ff


but I don't try it with "" in ifcfg files.
Maybe it should be supported, NM quote it by default.

The quote issue is already being addressed in bug #614943.

Finally marking as duplicate of #614786.

*** This bug has been marked as a duplicate of bug 614786 ***