Bug 463525 - [LTC 6.0 FEAT] 200970:Include CalIOC2 IOMMU Support (follow on to Calgary)
[LTC 6.0 FEAT] 200970:Include CalIOC2 IOMMU Support (follow on to Calgary)
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: kernel (Show other bugs)
All All
high Severity high
: alpha
: 6.0
Assigned To: James Takahashi (IBM)
Red Hat Kernel QE team
: FutureFeature
Depends On:
Blocks: 356741 554559 555224 555228
  Show dependency treegraph
Reported: 2008-09-23 16:11 EDT by IBM Bug Proxy
Modified: 2010-11-11 10:54 EST (History)
6 users (show)

See Also:
Fixed In Version:
Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2010-11-11 10:54:23 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
patch corrects max number of phbs (1.39 KB, text/plain)
2010-07-07 17:01 EDT, IBM Bug Proxy
no flags Details

  None (edit)
Description IBM Bug Proxy 2008-09-23 16:11:30 EDT
=Comment: #0=================================================
Emily J. Ratliff <emilyr@us.ibm.com> - 2008-09-16 17:54 EDT
1. Feature Overview:
Feature Id:	[200970]
a. Name of Feature:	Include CalIOC2 IOMMU Support (follow on to Calgary)
b. Feature Description
CalIOC2 is the High Speed SerDes to PCIe (PCI-Express)  Bridge controller for High End System x
servers. It performs similar functionality to the Calgary PCI-X host controller in earlier System x
servers. The CalIOC2 includes IOMMU support, similar to Calgary. IBM has extended the existing
calgary IOMMU code (already upstream in the x86_64 arch) to support IOMMU functionality for CalIOC2.

2. Feature Details:
Sponsor:	xSeries

Arch Specificity: Purely Arch Specific Code
Affects Core Kernel: Yes
Delivery Mechanism: Direct from community
Category:	xSeries
Request Type:	Kernel - Enhancement from IBM
d. Upstream Acceptance:	Accepted
Sponsor Priority	1
f. Severity: High
IBM Confidential:	no
Code Contribution:	IBM code
g. Component Version Target:	2.6.23

3. Business Case
Provides protection from errant DMAs on bare metal. Only available to IBM high-end customers.

Improves I/O performance in virtualized environments. Provides a level of protection against errant

4. Primary contact at Red Hat: 
John Jarvis

5. Primary contacts at Partner:
Project Management Contact:
Monte Knutson, mknutson@us.ibm.com, 877-894-1495

Technical contact(s):
Luvella McFadden, luvella@us.ibm.com
Chris McDermott, mcdermoc@us.ibm.com
Comment 1 Bill Nottingham 2008-10-02 15:18:44 EDT
RHEL 6 will include a newer kernel than 2.6.23, so this shouldn't be an issue.

The feature requested has already been accepted into the upstream code base
planned for the next major release of Red Hat Enterprise Linux.

When the next milestone release of Red Hat Enterprise Linux 6 is available,
please verify that the feature requested is present and functioning as
Comment 2 IBM Bug Proxy 2008-10-03 12:22:57 EDT
in Red Hat CVS Tree Marking as Accepted
Comment 3 IBM Bug Proxy 2008-10-13 15:01:35 EDT
There are additional critical changes to the calgary IOMMU code post 2.6.23. But, since these are already upstream, that shouldn't be a problem for RHEL6.

There are 2 additional issues that need to be addressed, but have not be submitted upstream yet:

1) There is an assumption made regarding the number of supported PCI-X Host Bridge buses in a Calgary
based system that doesn't hold true for CalIOC2 based systems. The CalIOC2 based servers have more PCI
bridges (PCIe root ports) per node and are (more) sparsely allocated. There are currently 2 ways to work
around this problem, but neither is very desirable, since it implies disabling the calgary iommu.

a) To workaround this problem without any changes to the kernel, use the 'iommu=soft' boot option.

b) To workaround this problem with a kernel change, modify the CALGARY_IOMMU_ENABLED_BY_DEFAULT config option from 'Y' to 'N'. This will allow the Calgary/CalioC2 code to still be enabled when the 'iommu=calgary' boot option is suppled, and will prevent the kernel from oopsing without any boot options on the large multi-node CalIOC2 based systems (x3950 M2). This is how RHEL5 configures the Calgary IOMMU today.

2) cagary_init() uses dev->bus->number to index into bus_info[] before the bus number is verified as valid (in calgary_init_one()).
Comment 5 IBM Bug Proxy 2010-07-07 17:01:39 EDT
------- Comment From coschult@us.ibm.com 2010-07-07 16:54 EDT-------
The patch fixing the number of phbs has been accepted upstream. I'm attaching a backport here. It should apply against rhel 6 beta2.
Comment 6 IBM Bug Proxy 2010-07-07 17:01:45 EDT
Created attachment 430174 [details]
patch corrects max number of phbs

------- Comment (attachment only) From coschult@us.ibm.com 2010-07-07 16:55 EDT-------
Comment 7 IBM Bug Proxy 2010-07-27 14:51:47 EDT
------- Comment From pbogdanovic@us.ibm.com 2010-07-27 14:41 EDT-------
This feature is not used anyone and is only relevant to old hardware.
Comment 8 IBM Bug Proxy 2010-09-28 12:30:48 EDT
------- Comment From tpnoonan@us.ibm.com 2010-09-28 12:21 EDT-------
(In reply to comment #12)
> We propose to add a release note that this feature is disabled and advise not
> to enable it on the kernel command line, since the patch is not in rhel6.

this feature will not be verified on rhel6.0 because of the above
Comment 9 releng-rhel@redhat.com 2010-11-11 10:54:23 EST
Red Hat Enterprise Linux 6.0 is now available and should resolve
the problem described in this bug report. This report is therefore being closed
with a resolution of CURRENTRELEASE. You may reopen this bug report if the
solution does not work for you.

Note You need to log in before you can comment on or make changes to this bug.