Bug 612788 - 82576 physical function device assignment doesn't work with window os when both two ports are pass-through to guest.
82576 physical function device assignment doesn't work with window os when bo...
Status: CLOSED WONTFIX
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: qemu-kvm (Show other bugs)
6.0
All Linux
low Severity medium
: beta
: 6.1
Assigned To: Don Dutile
Virtualization Bugs
:
Depends On:
Blocks: 580953
  Show dependency treegraph
 
Reported: 2010-07-08 22:26 EDT by juzhang
Modified: 2013-01-09 17:50 EST (History)
9 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
A dual function, 82576 interface (codename: Kawela, PCI Vendor/Device ID: 8086:10c9)cannot have both physical functions (PF's) device-assigned to a Windows 2008 guest. Either physical function can be device assigned to a Windows 2008 guest (PCI function 0 or function 1), but not both.
Story Points: ---
Clone Of:
Environment:
Last Closed: 2011-01-26 13:47:31 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description juzhang 2010-07-08 22:26:38 EDT
Description of problem:
There is an dual-port 82576 NIC in my system.If just pass through one port to guest,It's ok.however,If both two ports  pass-through to guest.In guest,two device are claimed, but just one get ip and works,the other device doesn't get an IP.


Version-Release number of selected component (if applicable):
#rpm -q qemu-kvm
qemu-kvm-0.12.1.2-2.91.el6.x86_64
#uname -r
2.6.32-37.el6.x86_64


How reproducible:

I tested window2008r2 and win7 64, take window2008r2 for example.

Steps to Reproduce:
1. Unbind device from host kernel driver (PCI device 03:00.0 and device 03:00.1)
1.1 lspci | grep Ethernet
lspci | grep Ethernet
01:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5764M Gigabit
Ethernet PCIe (rev 10)
03:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection
(rev 01)
03:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection
(rev 01)
28:00.0 Ethernet controller: Intel Corporation 82572EI Gigabit Ethernet
Controller (Copper) (rev 06)

2 lspci -n | grep 03:00.0
03:00.0 0200: 8086:10c9 (rev 01)
echo "8086 10c9" >/sys/bus/pci/drivers/pci-stub/new_id
echo 0000:03:00.0 >/sys/bus/pci/devices/0000\:03\:00.0/driver/unbind
echo 0000:03:00.0 >/sys/bus/pci/drivers/pci-stub/bind

3.lspci -n | grep 03:00.1
#03:00.1 0200: 8086:10c9 (rev 01)
echo "8086 10c9" >/sys/bus/pci/drivers/pci-stub/new_id
echo 0000:03:00.1 >/sys/bus/pci/devices/0000\:03\:00.1/driver/unbind
echo 0000:03:00.1 >/sys/bus/pci/drivers/pci-stub/bind

4.Boot guest just attach physical(82576) dual-port NIC card.
#/usr/libexec/qemu-kvm -m 4G -smp 4 -drive file=/root/zhangjunyi/win2k8.qcow2,if=none,id=test,boot=on,cache=none,format=qcow2 -device ide-drive,drive=test -cpu qemu64,+sse2,+x2apic -monitor tcp:0:5678,server,nowait -vnc :10 -qmp tcp:0:4445,server,nowait -drive file=/root/zhangjunyi/boot.iso,if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw -device ide-drive,drive=drive-ide0-1-0 -boot c -net none -device pci-assign,host=03:00.1,id=zjy -device pci-assign,host=03:00.0,id=zjy1
  
Actual results:
two device are claimed, but just one get ip and works,the other device doesn't get an IP.


Expected results:
Both devices works.


Additional info:
driver version shows up in guest

Intel(R) Gigabit ET Dual Port Server Adapter
Driver Provider: Intel
Driver Data: 12/4/2009
Driver Version: 11.4.7.0
Digital Signer: Microsoft Windows Hardware Compatibility Publisher


Intel(R) Gigabit ET Dual Port Server Adapter #2
Driver Provider: Intel
Driver Data: 12/4/2009
Driver Version: 11.4.7.0
Digital Signer: Microsoft Windows Hardware Compatibility Publisher

Please Note:
I also tested RHEL5.5 and RHEL6,It's all works.
Comment 2 Don Dutile 2010-07-12 16:17:33 EDT
could you try the following:

(1) specify 03:00.0 in the qemu-kvm cmdline first, then 03:00.1;
     by (PCI) definition, fcn 0 must appear before fcn1, but not sure
     if qem-kvm cares or not.  
    The devices should appear in the windows guest as devices having
     zero-based functions with different device numbers (like 00:4.0, 00:5.0,
     or the Windows-equiv), but interesting to know

(2) Could you do the assignment using a properly qualified xml spec
    and use virsh define <guest-name>.xml & virst start <guest-name> ?
  e.g., 
    (a) start kvm guest as listed above _without_ the pci dev assignments.
    (b) exec: virsh dumpxml <guest-name> > <guest-name>.xml
        i) edit <guest-name>.xml and add:
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
      </source>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0000' bus='0x03' slot='0x00' function='0x1'/>
      </source>
    </hostdev>

   (c) stop <guest-name> : virsh shutdown <guest-name>
   (d) virsh undefine <guest-name>
   (e) virsh define <guest-name> < <guest-name>.xml
   (f) virsh start <guest-name>


Note, also, that Windows reports it as a 'dual-port server adapter',
yet qemu-kvm is probably exporting it as two, single devices.
Not sure if Windows driver is having 'heartburn' that a dual-port is showing up as a single port.

What does Device Manager show/say about each port -- does it show the first one functional, and a yellow/warning label on the second?

Finally, can you try kernel-2.6.32-44 and qemu-kvm 0.12.1.2-2.93 ?
Minor tweaks were done in this area around MSI-X mapping, which this adapter performs.  Seen as a multi-VF problem, but wondering if multi-PF has same problem.
Comment 3 Don Dutile 2010-07-14 15:46:38 EDT
I dug up a dual Kawela 82576, id 8086:10c9.

Able to duplicate this bz.

note that dual Kawela have shared eeprom access, and when assigning
function 1 of PF's to a windows guest, it appears as a unique function 0 device.

I dont know how Linux drivers work (as claimed in description; I should try duping those cases), but this looks like a Windows driver 'feature'.

I can assign either of the PF's (PF.0 or PF.1), but cannot assign both to the
guest as the same time.

Other things found while testing:
(a) if I swap the order of assignment, i.e., PF.0 first, PF.1 second, the first-spec'd dev-assignment is the one that connects successfully in win2k8.
(b) The successfully configured dev in win2k8 is always named
     "Intel(R) Gigabit ET Dual Port Server Adapter #2"
   The failing one is always "Intel(R) Gigabit ET Dual Port Server Adapter"
(c) the successfully configured dev is always pci dev 00:04.0;
    the failing one is always pci dev 00:05.0  (in the guest/win2k8 vm).

Adding Intel Partner & Ron Pacheco to see if we should be using
an updated driver on win2k8 for this situation, since Linux drivers (5.5 & 6.0)
work.
Comment 4 RHEL Product and Program Management 2010-07-15 10:06:07 EDT
This issue has been proposed when we are only considering blocker
issues in the current Red Hat Enterprise Linux release. It has
been denied for the current Red Hat Enterprise Linux release.

** If you would still like this issue considered for the current
release, ask your support representative to file as a blocker on
your behalf. Otherwise ask that it be considered for the next
Red Hat Enterprise Linux release. **
Comment 11 Don Dutile 2010-07-23 15:54:07 EDT
Moving to rhel6.1, & Tier3 for now.

Intel claims device assigning any dual-fcn, Kawela PF to a Windows guest is not supported.

Still trying to debug to get to final resolution, but current belief is w2k8 driver isnt as clean/robust as linux driver.
Comment 13 Don Dutile 2010-07-27 11:25:24 EDT
Technical Note:
===============

A dual function, 82576 (codename: Kawela, PCI Vendor/Device ID: 8086:10c9) interface cannot have both physical functions (PF's) device-assigned to a Window 2008 guest.  Either function can be device assigned to a Windows 2008 guest (PCI function 0 or function 1), but not both. Since dual-function device assignment works for Linux guests that support the 82576, it appears that there is a bug in the Windows 2008 driver that affects dual-function operation when device-assigned to a KVM guest.
Comment 14 Ryan Lerch 2010-08-30 20:05:24 EDT
    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
A dual function, 82576 (codename: Kawela, PCI Vendor/Device ID: 8086:10c9)
interface cannot have both physical functions (PF's) device-assigned to a
Window 2008 guest.  Either function can be device assigned to a Windows 2008
guest (PCI function 0 or function 1), but not both. Since dual-function device
assignment works for Linux guests that support the 82576, it appears that there
is a bug in the Windows 2008 driver that affects dual-function operation when
device-assigned to a KVM guest.
Comment 16 Ryan Lerch 2010-10-05 02:01:26 EDT
    Technical note updated. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    Diffed Contents:
@@ -1,7 +1 @@
-A dual function, 82576 (codename: Kawela, PCI Vendor/Device ID: 8086:10c9)
+A dual function, 82576 interface (codename: Kawela, PCI Vendor/Device ID: 8086:10c9)cannot have both physical functions (PF's) device-assigned to a Windows 2008 guest.  Either physical function can be device assigned to a Windows 2008 guest (PCI function 0 or function 1), but not both.-interface cannot have both physical functions (PF's) device-assigned to a
-Window 2008 guest.  Either function can be device assigned to a Windows 2008
-guest (PCI function 0 or function 1), but not both. Since dual-function device
-assignment works for Linux guests that support the 82576, it appears that there
-is a bug in the Windows 2008 driver that affects dual-function operation when
-device-assigned to a KVM guest.
Comment 18 Don Dugger 2010-11-22 16:37:13 EST
Note that Intel does not support assignment of PF devices to a guest (that's what VFs were designed for).  Even though assigning 2 PFs works for a Linux guest that is still an unsupported configuration and not recommended.
Comment 20 Don Dutile 2011-01-26 13:47:31 EST
Closing due to lack of Windows driver support.

Note You need to log in before you can comment on or make changes to this bug.