Bug 560540

Summary: Reserve PNP enumerated system board iomem resources
Product: Red Hat Enterprise Linux 5 Reporter: Myron Stowe <myron.stowe>
Component: kernelAssignee: Prarit Bhargava <prarit>
Status: CLOSED ERRATA QA Contact: Red Hat Kernel QE team <kernel-qe>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 5.4.zCC: adam.vinsh, adaora.onyia, dhoward, eguan, jpirko, jwilson, prarit, shawn.pagan, tao
Target Milestone: rcKeywords: ZStream
Target Release: 5.6   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Previously, system board iomem resources, which were enumerated using the PNP Motherboard resource descriptions, were not recognized and taken into consideration when gathering resource information. This could have caused MMIO-based requests to receive allocations that were not valid. With this update, system board iomem resources are correctly recognized when gathering resource information.
Story Points: ---
Clone Of: Environment:
Last Closed: 2011-01-13 21:02:37 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
Bug Depends On:    
Bug Blocks: 562684, 629861    
Attachments:
Description Flags
RHEL5 fix for this issue part 1
none
RHEL5 fix for this issue part 2 none

Description Myron Stowe 2010-02-01 04:25:33 UTC
Description of problem: System board iomem resources enumerated using the PNP Motherboard resource descriptions are currently not recognized and taken into consideration when gathering resource information.  What this means is that MMIO based requests may end up receiving allocations that are not valid. 


Version-Release number of selected component (if applicable):


How reproducible: 100%


Steps to Reproduce:
The steps will be platform specific.  What we have found is
1. Boot an Intel x56 chipset based system whose BIOS utilizes PNP Motherboard resource descriptions.
2. Enable an I/O device's option ROM and try to access it (an  Intel Corporation 82571EB Gigabit Ethernet Controller; e1000 NIC - is what we have been using).
3. The option ROM access may fail if the allocation given conflicts with other resources.  We are encountering failures where the allocation given conflicts with the VTBAR related register window.
  
Actual results: The option ROM access fails due to resource conflicts.


Expected results: The option ROM access succeeds because the kernel's resources have been constructed taking into account PNP Motherboard resource descriptions.


Additional info: The RHEL5.x stream needs to incorporate/backport git commit a8c78f7fb1571764f48b8af5459abdd2c66a765f as it resolves this issue.

Comment 1 Prarit Bhargava 2010-02-16 19:01:43 UTC
I assume this effects HP DL series systems.  Assigning to tcamuso.

P.

Comment 3 Myron Stowe 2010-02-24 00:02:14 UTC
Tony - any update?  Any chance of this getting into RHEL5.5 and RHEL6?

Comment 4 Issue Tracker 2010-02-25 20:08:01 UTC
Event posted on 02-19-2010 05:18pm EST by myron.stowe

The main scenarios where I see this is needed currently are PCI hotplug and
option ROM programming.  Obviously the OS needs to know what resources are
available under a given IOH in order to open bridge windows to accommodate
a
PCI device hotplugged into the system.  BIOS plays no part in actively
allocating resources for the device in this scenario.  For devices
installed
from boot, BIOS will program memory and I/O port resources, but does not
program PCI option ROMs.  Linux prefers to map ROMs into prefetchable
space and
will open up windows on the bridge to accommodate the device if needed. 
In a
multi-IOH system, resource routing considerations and consequences
increase.  Without this, Linux is only guessing which address might go to
which IOH and will sometimes get lucky, and other times not.  

In the hotplug case, incorrect guessing means the device probably doesn't
work
at all, in the option ROM case, it means the option ROM doesn't work (ie.
the
data read doesn't hit the card, so the return is probably -1).  An
improperly
programmed option may mean that device firmware required to make use of
the
card is inaccessible.  These cards may work in some slots, but not others,
and
may also be dependent on the resource usage of other cards in the system
as to
whether the option ROM gets mapped to a valid location.

With Intel's introduction of VT-d, the ramifications mentioned above are
magnified.  The address window containing the set of registers related to
VT-d is denoted by the VTBAR.  In Intel's chipset(s), the VTBAR must
reside within the system's Local MMIOL resource.  This introduces the
possibility that a range assigned by Linux for an option ROM may conflict
(overlap) with the VT-d window which is how we recently stumbled upon
this.  It is very hard to debug when a ROM gets mapped behind a VTBAR and
makes the device unusable in house.  When a customer encounters, and
reports such, it will be much harder to diagnose.

Without the patch mentioned, RHEL5.x is susceptible to these issues on any
vendor's Intel based x86 system that incorporates VT-d functionality. 
This is already true today with existing single IOH architectures.  With
multi-IOH systems that are now beginning to appear, the susceptibility to
these issues increases dramatically.


This event sent from IssueTracker by dwa 
 issue 537293

Comment 5 Myron Stowe 2010-02-25 20:35:51 UTC
With respect to Prarit's comments in "Comment 1" - yes, it does effect HP systems but it is *not* limited to HP systems.  The issue will occur on any vendor's Intel based x86 system that incorporates VT-d (which I believe includes most, if not all, recent Intel based systems.

Comment 8 Myron Stowe 2010-06-25 18:04:02 UTC
Repeating comment 3: any updates here.  Any chance this will get into 5.5 or 6.0?
The patch was incorporated upstream over three years ago.

Comment 9 Prarit Bhargava 2010-07-29 19:11:04 UTC
Created attachment 435372 [details]
RHEL5 fix for this issue part 1

Comment 10 Prarit Bhargava 2010-07-29 19:11:46 UTC
Created attachment 435373 [details]
RHEL5 fix for this issue part 2

Comment 11 RHEL Product and Program Management 2010-08-27 18:30:30 UTC
This request was evaluated by Red Hat Product Management for inclusion in a Red
Hat Enterprise Linux maintenance release.  Product Management has requested
further review of this request by Red Hat Engineering, for potential
inclusion in a Red Hat Enterprise Linux Update release for currently deployed
products.  This request is not yet committed for inclusion in an Update
release.

Comment 15 Jarod Wilson 2010-09-10 21:38:49 UTC
in kernel-2.6.18-219.el5
You can download this test kernel from http://people.redhat.com/jwilson/el5

Detailed testing feedback is always welcomed.

Comment 17 Eryu Guan 2010-11-01 11:34:00 UTC
pnp entries are present in /proc/iomem on 2.6.18-194.230.el5 kernel
[root@ibm-x3950m2-01 ~]# uname -a
Linux ibm-x3950m2-01.rhts.eng.bos.redhat.com 2.6.18-230.el5PAE #1 SMP Thu Oct 28 17:13:02 EDT 2010 i686 i686 i386 GNU/Linux
[root@ibm-x3950m2-01 ~]# cat /proc/iomem | grep pnp
00000400-000004ff : pnp 00:08
fda00000-fdbfffff : pnp 00:06
fde84000-fde843ff : pnp 00:0a
fde86000-fdf05fff : pnp 00:06
[root@ibm-x3950m2-01 ~]#

On unpatched kernel, no such entries
[root@ibm-x3950m2-01 ~]# uname -a
Linux ibm-x3950m2-01.rhts.eng.bos.redhat.com 2.6.18-194.el5PAE #1 SMP Tue Mar 16 22:00:21 EDT 2010 i686 i686 i386 GNU/Linux
[root@ibm-x3950m2-01 ~]#  cat /proc/iomem | grep pnp
[root@ibm-x3950m2-01 ~]#

Comment 18 Martin Prpič 2010-11-11 13:57:53 UTC
    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
Previously, system board iomem resources, which were enumerated using the PNP Motherboard resource descriptions, were not recognized and taken into consideration when gathering resource information. This could have caused MMIO-based requests to receive allocations that were not valid. With this update, system board iomem resources are correctly recognized when gathering resource information.

Comment 20 errata-xmlrpc 2011-01-13 21:02:37 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHSA-2011-0017.html