Bug 463525

Summary: [LTC 6.0 FEAT] 200970:Include CalIOC2 IOMMU Support (follow on to Calgary)
Product: Red Hat Enterprise Linux 6 Reporter: IBM Bug Proxy <bugproxy>
Component: kernelAssignee: James Takahashi (IBM) <nobody+PNT0273897>
Status: CLOSED CURRENTRELEASE QA Contact: Red Hat Kernel QE team <kernel-qe>
Severity: high Docs Contact:
Priority: high    
Version: 6.0CC: ejratl, emcnabb, jjarvis, notting, peterm, snagar
Target Milestone: alphaKeywords: FutureFeature
Target Release: 6.0   
Hardware: All   
OS: All   
Whiteboard:
Fixed In Version: Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2010-11-11 15:54:23 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 356741, 554559, 555224, 555228    
Attachments:
Description Flags
patch corrects max number of phbs none

Description IBM Bug Proxy 2008-09-23 20:11:30 UTC
=Comment: #0=================================================
Emily J. Ratliff <emilyr.com> - 2008-09-16 17:54 EDT
1. Feature Overview:
Feature Id:	[200970]
a. Name of Feature:	Include CalIOC2 IOMMU Support (follow on to Calgary)
b. Feature Description
CalIOC2 is the High Speed SerDes to PCIe (PCI-Express)  Bridge controller for High End System x
servers. It performs similar functionality to the Calgary PCI-X host controller in earlier System x
servers. The CalIOC2 includes IOMMU support, similar to Calgary. IBM has extended the existing
calgary IOMMU code (already upstream in the x86_64 arch) to support IOMMU functionality for CalIOC2.

2. Feature Details:
Sponsor:	xSeries
Architectures:
x86_64

Arch Specificity: Purely Arch Specific Code
Affects Core Kernel: Yes
Delivery Mechanism: Direct from community
Category:	xSeries
Request Type:	Kernel - Enhancement from IBM
d. Upstream Acceptance:	Accepted
Sponsor Priority	1
f. Severity: High
IBM Confidential:	no
Code Contribution:	IBM code
g. Component Version Target:	2.6.23

3. Business Case
Provides protection from errant DMAs on bare metal. Only available to IBM high-end customers.

Benefits
Improves I/O performance in virtualized environments. Provides a level of protection against errant
DMAs.

4. Primary contact at Red Hat: 
John Jarvis
jjarvis

5. Primary contacts at Partner:
Project Management Contact:
Monte Knutson, mknutson.com, 877-894-1495

Technical contact(s):
Luvella McFadden, luvella.com
Chris McDermott, mcdermoc.com

Comment 1 Bill Nottingham 2008-10-02 19:18:44 UTC
RHEL 6 will include a newer kernel than 2.6.23, so this shouldn't be an issue.

The feature requested has already been accepted into the upstream code base
planned for the next major release of Red Hat Enterprise Linux.

When the next milestone release of Red Hat Enterprise Linux 6 is available,
please verify that the feature requested is present and functioning as
desired.

Comment 2 IBM Bug Proxy 2008-10-03 16:22:57 UTC
in Red Hat CVS Tree Marking as Accepted

Comment 3 IBM Bug Proxy 2008-10-13 19:01:35 UTC
There are additional critical changes to the calgary IOMMU code post 2.6.23. But, since these are already upstream, that shouldn't be a problem for RHEL6.

There are 2 additional issues that need to be addressed, but have not be submitted upstream yet:

1) There is an assumption made regarding the number of supported PCI-X Host Bridge buses in a Calgary
based system that doesn't hold true for CalIOC2 based systems. The CalIOC2 based servers have more PCI
bridges (PCIe root ports) per node and are (more) sparsely allocated. There are currently 2 ways to work
around this problem, but neither is very desirable, since it implies disabling the calgary iommu.

a) To workaround this problem without any changes to the kernel, use the 'iommu=soft' boot option.

b) To workaround this problem with a kernel change, modify the CALGARY_IOMMU_ENABLED_BY_DEFAULT config option from 'Y' to 'N'. This will allow the Calgary/CalioC2 code to still be enabled when the 'iommu=calgary' boot option is suppled, and will prevent the kernel from oopsing without any boot options on the large multi-node CalIOC2 based systems (x3950 M2). This is how RHEL5 configures the Calgary IOMMU today.

2) cagary_init() uses dev->bus->number to index into bus_info[] before the bus number is verified as valid (in calgary_init_one()).

Comment 5 IBM Bug Proxy 2010-07-07 21:01:39 UTC
------- Comment From coschult.com 2010-07-07 16:54 EDT-------
The patch fixing the number of phbs has been accepted upstream. I'm attaching a backport here. It should apply against rhel 6 beta2.

Comment 6 IBM Bug Proxy 2010-07-07 21:01:45 UTC
Created attachment 430174 [details]
patch corrects max number of phbs


------- Comment (attachment only) From coschult.com 2010-07-07 16:55 EDT-------

Comment 7 IBM Bug Proxy 2010-07-27 18:51:47 UTC
------- Comment From pbogdanovic.com 2010-07-27 14:41 EDT-------
This feature is not used anyone and is only relevant to old hardware.

Comment 8 IBM Bug Proxy 2010-09-28 16:30:48 UTC
------- Comment From tpnoonan.com 2010-09-28 12:21 EDT-------
(In reply to comment #12)
> We propose to add a release note that this feature is disabled and advise not
> to enable it on the kernel command line, since the patch is not in rhel6.

this feature will not be verified on rhel6.0 because of the above

Comment 9 releng-rhel@redhat.com 2010-11-11 15:54:23 UTC
Red Hat Enterprise Linux 6.0 is now available and should resolve
the problem described in this bug report. This report is therefore being closed
with a resolution of CURRENTRELEASE. You may reopen this bug report if the
solution does not work for you.