Bug 577182 - vxge: not enough MMIO resources for SR-IOV error
vxge: not enough MMIO resources for SR-IOV error
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: kernel (Show other bugs)
5.5
All Linux
high Severity high
: beta
: 5.6
Assigned To: Don Dutile
Network QE
: FutureFeature, HardwareEnablement, OtherQA
Depends On: 523341
Blocks: 557597 570372 580413 Rhel5KvmTier1
  Show dependency treegraph
 
Reported: 2010-03-26 08:03 EDT by Masroor
Modified: 2011-01-13 16:21 EST (History)
12 users (show)

See Also:
Fixed In Version: kernel-2.6.18-219.el5
Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2011-01-13 16:21:49 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Masroor 2010-03-26 08:03:12 EDT
Description of problem:

When loading a driver for a PCIe device which supports SR-IOV, the BARs
associated with the SR-IOV capability may not be reliably requested, even if
there is space.
The bug is similar (identical?) to 523341
(https://bugzilla.redhat.com/show_bug.cgi?id=523341)

Version-Release number of selected component (if applicable):

2.6.18-190.el5xen

How reproducible:

Platform dependent, but when it happens it happens 100% of the time.

Steps to Reproduce:
1. Enable SR-IOV in system BIOS
2. Boot RHEL-5.5 2.6.18-190.el5xen
3. Load relevant driver (for SR-IOV capable device)

Actual results:

Driver modprobe fails and produces an error like: 
“PCI: Failed to allocate mem resource #12:1800000@b4800000 for 0000:02:00.0
vxge 0000:02:00.0: not enough MMIO resources for SR-IOV”


This means the physical device (aka Physical Function or PF) will not work,
and, of course, no virtual devices (aka Virtual Function or VF) associated w/
that PF can be allocated.

Expected results:

Driver is successfully loaded and VFs are successfully allocated making both
the PF and its VFs functional.


Additional info:
Comment 1 Chris Wright 2010-03-26 14:25:12 EDT
The info in comment #1 mention Xen, however this is not a Xen specific bug.  Neterion have shown same issue on bare metal case, and both bare metal and Xen are using the same PCI enumeration and resource reservation code.  I have updated the bug to be against kernel, and made it dependent on BZ# 523341 which is a generic entry for this resource allocation issue.

Masroor, can you please verify here in this BZ that the patched kernel available at:

http://et.redhat.com/~chrisw/rhel5/5.5/bz523341/

fixes the issue for you?
Comment 2 Masroor 2010-03-29 08:11:00 EDT
Hi Chris,

I have upgraded to kernel-xen-2.6.18-190.el5.bz523341.x86_64.rpm which fixes this issue.

Thanks,
Masroor
Comment 3 Ramkrishna Vepa 2010-04-13 22:49:37 EDT
Hi Chris,
Can this fix be included as part of a 5.5.z errata and 5.6? This will enable users to run the X3100 with SRIOV enabled.

Thanks,
Ram
Comment 4 Larry Troan 2010-05-19 12:34:58 EDT
Chris, is this patch already included in the kernel planned for 5.6? If not, are there plans to submit it and we will likely require an exception to have it considered.

This bug fix is a pre-req to a driver update required my some high-profile customers that plan to use the Exar NIC in their Red Hat certified servers.
Comment 5 Larry Troan 2010-06-15 11:38:22 EDT
Per Ram Vepa in a telecon 6/14, patches submitted and accepted upstream. They have been running a patched kernel from Chris Wright since last September.
Comment 6 Larry Troan 2010-07-27 17:12:34 EDT
Patches need to be attached to this bug by 8/01.
Comment 7 Larry Troan 2010-07-27 21:34:05 EDT
Chris, do we have the patches POSTed to rhkernel and approved for 5.6?

Note that the patch submission date is Aug 13 (not Aug 1 above) but they must be reviewed and approved by that date.
Comment 8 Larry Troan 2010-07-29 18:07:29 EDT
Requesting exception until NEEDINFO satisfied.
Comment 11 Larry Troan 2010-08-29 21:38:31 EDT
In response to comment #7,
Don Dutile has 3-part patch set he's working for 5.6. Removing NEEDINFO.
Comment 16 Srirama 2010-11-25 07:16:48 EST
Installed RHEL 5.6 Beta (snapshot1) image and verified that the driver (version
2.0.8.20182-k) loads successfully in SRIOV mode. PF and all the VFs get
enumerated properly and are functional. Also verified that the passthrough
configuration works by attaching the PCI functions to 2 VMs and running traffic
between the VMs and also to the remote machine.

[root@linux-rhel-5 ~]# uname -a
Linux linux-rhel-5.6-181 2.6.18-231.el5xen #1 SMP Mon Nov 8 18:45:02 EST 2010
x86_64 x86_64 x86_64 GNU/Linux
[root@linux-rhel-5 ~]# lspci -d17d5:*
02:00.0 Ethernet controller: Exar Corp. X3100 Series 10 Gigabit Ethernet PCIe
(rev 02)
02:00.1 Ethernet controller: Exar Corp. X3100 Series 10 Gigabit Ethernet PCIe
(rev 02)
02:00.2 Ethernet controller: Exar Corp. X3100 Series 10 Gigabit Ethernet PCIe
(rev 02)
02:00.3 Ethernet controller: Exar Corp. X3100 Series 10 Gigabit Ethernet PCIe
(rev 02)
02:00.4 Ethernet controller: Exar Corp. X3100 Series 10 Gigabit Ethernet PCIe
(rev 02)
02:00.5 Ethernet controller: Exar Corp. X3100 Series 10 Gigabit Ethernet PCIe
(rev 02)
02:00.6 Ethernet controller: Exar Corp. X3100 Series 10 Gigabit Ethernet PCIe
(rev 02)
02:00.7 Ethernet controller: Exar Corp. X3100 Series 10 Gigabit Ethernet PCIe
(rev 02)
02:01.0 Ethernet controller: Exar Corp. X3100 Series 10 Gigabit Ethernet PCIe
(rev 02)
02:01.1 Ethernet controller: Exar Corp. X3100 Series 10 Gigabit Ethernet PCIe
(rev 02)
02:01.2 Ethernet controller: Exar Corp. X3100 Series 10 Gigabit Ethernet PCIe
(rev 02)
02:01.3 Ethernet controller: Exar Corp. X3100 Series 10 Gigabit Ethernet PCIe
(rev 02)
02:01.4 Ethernet controller: Exar Corp. X3100 Series 10 Gigabit Ethernet PCIe
(rev 02)
02:01.5 Ethernet controller: Exar Corp. X3100 Series 10 Gigabit Ethernet PCIe
(rev 02)
02:01.6 Ethernet controller: Exar Corp. X3100 Series 10 Gigabit Ethernet PCIe
(rev 02)
02:01.7 Ethernet controller: Exar Corp. X3100 Series 10 Gigabit Ethernet PCIe
(rev 02)
02:02.0 Ethernet controller: Exar Corp. X3100 Series 10 Gigabit Ethernet PCIe
(rev 02)
Comment 19 errata-xmlrpc 2011-01-13 16:21:49 EST
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHSA-2011-0017.html

Note You need to log in before you can comment on or make changes to this bug.