Description of problem: When loading a driver for a PCIe device which supports SR-IOV, the BARs associated with the SR-IOV capability may not be reliably requested, even if there is space. The bug is similar (identical?) to 523341 (https://bugzilla.redhat.com/show_bug.cgi?id=523341) Version-Release number of selected component (if applicable): 2.6.18-190.el5xen How reproducible: Platform dependent, but when it happens it happens 100% of the time. Steps to Reproduce: 1. Enable SR-IOV in system BIOS 2. Boot RHEL-5.5 2.6.18-190.el5xen 3. Load relevant driver (for SR-IOV capable device) Actual results: Driver modprobe fails and produces an error like: “PCI: Failed to allocate mem resource #12:1800000@b4800000 for 0000:02:00.0 vxge 0000:02:00.0: not enough MMIO resources for SR-IOV” This means the physical device (aka Physical Function or PF) will not work, and, of course, no virtual devices (aka Virtual Function or VF) associated w/ that PF can be allocated. Expected results: Driver is successfully loaded and VFs are successfully allocated making both the PF and its VFs functional. Additional info:
The info in comment #1 mention Xen, however this is not a Xen specific bug. Neterion have shown same issue on bare metal case, and both bare metal and Xen are using the same PCI enumeration and resource reservation code. I have updated the bug to be against kernel, and made it dependent on BZ# 523341 which is a generic entry for this resource allocation issue. Masroor, can you please verify here in this BZ that the patched kernel available at: http://et.redhat.com/~chrisw/rhel5/5.5/bz523341/ fixes the issue for you?
Hi Chris, I have upgraded to kernel-xen-2.6.18-190.el5.bz523341.x86_64.rpm which fixes this issue. Thanks, Masroor
Hi Chris, Can this fix be included as part of a 5.5.z errata and 5.6? This will enable users to run the X3100 with SRIOV enabled. Thanks, Ram
Chris, is this patch already included in the kernel planned for 5.6? If not, are there plans to submit it and we will likely require an exception to have it considered. This bug fix is a pre-req to a driver update required my some high-profile customers that plan to use the Exar NIC in their Red Hat certified servers.
Per Ram Vepa in a telecon 6/14, patches submitted and accepted upstream. They have been running a patched kernel from Chris Wright since last September.
Patches need to be attached to this bug by 8/01.
Chris, do we have the patches POSTed to rhkernel and approved for 5.6? Note that the patch submission date is Aug 13 (not Aug 1 above) but they must be reviewed and approved by that date.
Requesting exception until NEEDINFO satisfied.
In response to comment #7, Don Dutile has 3-part patch set he's working for 5.6. Removing NEEDINFO.
Installed RHEL 5.6 Beta (snapshot1) image and verified that the driver (version 2.0.8.20182-k) loads successfully in SRIOV mode. PF and all the VFs get enumerated properly and are functional. Also verified that the passthrough configuration works by attaching the PCI functions to 2 VMs and running traffic between the VMs and also to the remote machine. [root@linux-rhel-5 ~]# uname -a Linux linux-rhel-5.6-181 2.6.18-231.el5xen #1 SMP Mon Nov 8 18:45:02 EST 2010 x86_64 x86_64 x86_64 GNU/Linux [root@linux-rhel-5 ~]# lspci -d17d5:* 02:00.0 Ethernet controller: Exar Corp. X3100 Series 10 Gigabit Ethernet PCIe (rev 02) 02:00.1 Ethernet controller: Exar Corp. X3100 Series 10 Gigabit Ethernet PCIe (rev 02) 02:00.2 Ethernet controller: Exar Corp. X3100 Series 10 Gigabit Ethernet PCIe (rev 02) 02:00.3 Ethernet controller: Exar Corp. X3100 Series 10 Gigabit Ethernet PCIe (rev 02) 02:00.4 Ethernet controller: Exar Corp. X3100 Series 10 Gigabit Ethernet PCIe (rev 02) 02:00.5 Ethernet controller: Exar Corp. X3100 Series 10 Gigabit Ethernet PCIe (rev 02) 02:00.6 Ethernet controller: Exar Corp. X3100 Series 10 Gigabit Ethernet PCIe (rev 02) 02:00.7 Ethernet controller: Exar Corp. X3100 Series 10 Gigabit Ethernet PCIe (rev 02) 02:01.0 Ethernet controller: Exar Corp. X3100 Series 10 Gigabit Ethernet PCIe (rev 02) 02:01.1 Ethernet controller: Exar Corp. X3100 Series 10 Gigabit Ethernet PCIe (rev 02) 02:01.2 Ethernet controller: Exar Corp. X3100 Series 10 Gigabit Ethernet PCIe (rev 02) 02:01.3 Ethernet controller: Exar Corp. X3100 Series 10 Gigabit Ethernet PCIe (rev 02) 02:01.4 Ethernet controller: Exar Corp. X3100 Series 10 Gigabit Ethernet PCIe (rev 02) 02:01.5 Ethernet controller: Exar Corp. X3100 Series 10 Gigabit Ethernet PCIe (rev 02) 02:01.6 Ethernet controller: Exar Corp. X3100 Series 10 Gigabit Ethernet PCIe (rev 02) 02:01.7 Ethernet controller: Exar Corp. X3100 Series 10 Gigabit Ethernet PCIe (rev 02) 02:02.0 Ethernet controller: Exar Corp. X3100 Series 10 Gigabit Ethernet PCIe (rev 02)
An advisory has been issued which should help the problem described in this bug report. This report is therefore being closed with a resolution of ERRATA. For more information on therefore solution and/or where to find the updated files, please follow the link below. You may reopen this bug report if the solution does not work for you. http://rhn.redhat.com/errata/RHSA-2011-0017.html