Bug 486185 - pci_setup_bridge() clears the Prefetchable Memory Base and Limit Upper 32 Bits registers
pci_setup_bridge() clears the Prefetchable Memory Base and Limit Upper 32 Bit...
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: kernel (Show other bugs)
All Linux
high Severity medium
: rc
: ---
Assigned To: Prarit Bhargava
Red Hat Kernel QE team
: OtherQA
Depends On:
  Show dependency treegraph
Reported: 2009-02-18 14:36 EST by Issue Tracker
Modified: 2010-10-23 03:46 EDT (History)
9 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2009-09-02 04:33:59 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
RHEL5 fix for this issue (1.86 KB, patch)
2009-03-03 13:47 EST, Prarit Bhargava
no flags Details | Diff
RHEL5 fix for this issue (3.57 KB, patch)
2009-03-04 13:48 EST, Prarit Bhargava
no flags Details | Diff

  None (edit)
Description Issue Tracker 2009-02-18 14:36:58 EST
Escalated to Bugzilla from IssueTracker
Comment 1 Issue Tracker 2009-02-18 14:37:00 EST
Description of problem:

The problem is that pci_setup_bridge() clears the Prefetchable
Memory Base and Limit Upper 32 Bits registers, i.e. it
potentially moves the address range defined by these and the
Prefetchable Memory Base and Limit registers. The function needs
to preserve the upper 32 bits of the 64-bit start/end.

The fix needed in RHEL5's 2.6.18-92.1.22.el5 kernel (the most
recent I could find) is (a subset of the) linux-2.6 git
commit c40a22e0ce5eb400f27449e59e43d021bee58b8d from the end of

Please let us know if more information is needed.

This event sent from IssueTracker by fleitner  [Support Engineering Group]
 issue 258146
Comment 2 Issue Tracker 2009-02-18 14:37:01 EST

This event sent from IssueTracker by fleitner  [Support Engineering Group]
 issue 258146
Comment 3 Issue Tracker 2009-02-18 14:37:03 EST
As-is, pci_setup_brige() always clears the upper 32 bits of the
prefetchable-memory-behind-bridge base and limit registers. This much
should be obvious by looking at the code. A subset of the patch that we
referenced fixes this by considering the upper 32 bits in the associated
PCI resource object, i.e. the values written to PCI_PREF_BASE_UPPER32 and
PCI_PREF_LIMIT_UPPER32 are derived from the same region the value written
to PCI_PREF_MEMORY_BASE is derived from. In order to test the code, RedHat
will either need to find a system with an SBIOS willing to map a
prefetchable 64-bit memory BAR (such as our BAR1) above 4GB and verify
that the upper half of the address ranges isn't lost, or you'll need to
hack your own kernel to move a suitable BAR above 4GB before same kernel
runs pci_setup_bridge().

This event sent from IssueTracker by fleitner  [Support Engineering Group]
 issue 258146
Comment 4 Prarit Bhargava 2009-03-02 08:40:05 EST
IIRC, SGI Altix boxes map all PCI mem above 4G.

The real problem then is finding a card that has a prefetch memory value defined ... but I can always hack around that ;)

Comment 5 Prarit Bhargava 2009-03-03 13:47:13 EST
Created attachment 333911 [details]
RHEL5 fix for this issue
Comment 6 RHEL Product and Program Management 2009-03-03 14:04:50 EST
This request was evaluated by Red Hat Product Management for inclusion in a Red
Hat Enterprise Linux maintenance release.  Product Management has requested
further review of this request by Red Hat Engineering, for potential
inclusion in a Red Hat Enterprise Linux Update release for currently deployed
products.  This request is not yet committed for inclusion in an Update
Comment 7 Prarit Bhargava 2009-03-04 13:48:03 EST
Created attachment 334029 [details]
RHEL5 fix for this issue

bmaly suggested further changes to the patch.
Comment 8 Don Zickus 2009-04-27 11:59:34 EDT
in kernel-2.6.18-141.el5
You can download this test kernel from http://people.redhat.com/dzickus/el5

Please do NOT transition this bugzilla state to VERIFIED until our QE team
has sent specific instructions indicating when to do so.  However feel free
to provide a comment indicating that this fix has been verified.
Comment 10 Chris Ward 2009-07-03 14:24:59 EDT
~~ Attention - RHEL 5.4 Beta Released! ~~

RHEL 5.4 Beta has been released! There should be a fix present in the Beta release that addresses this particular request. Please test and report back results here, at your earliest convenience. RHEL 5.4 General Availability release is just around the corner!

If you encounter any issues while testing Beta, please describe the issues you have encountered and set the bug into NEED_INFO. If you encounter new issues, please clone this bug to open a new issue and request it be reviewed for inclusion in RHEL 5.4 or a later update, if it is not of urgent severity.

Please do not flip the bug status to VERIFIED. Only post your verification results, and if available, update Verified field with the appropriate value.

Questions can be posted to this bug or your customer or partner representative.
Comment 16 Issue Tracker 2009-08-17 16:39:23 EDT
Event posted on 08-17-2009 04:39pm EDT by jkachuck

Per call today NVIDIA has tested it and it looks good. I have requested
NVIDIA to update this issue with this information as well.

Thank You
Joe Kachuck

This event sent from IssueTracker by jkachuck 
 issue 258146
Comment 17 errata-xmlrpc 2009-09-02 04:33:59 EDT
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

Comment 19 Garrison Wu 2009-09-02 15:29:58 EDT

NVIDIA engineering verified with the the two test kernels on a system on which a BAR is mapped above 4GB by the SBIOS.  On both kernels BAR1 of the cards tested were placed above 4G and the prefetchable Memory window of the PCI bridge just above these cards were above 4G accordingly.  Full functionality of the test cards were verified

Garrison Wu

Note You need to log in before you can comment on or make changes to this bug.