Bug 463183 - [LTC 6.0 FEAT] 201700:Support Intel 10Gb Ethernet (Knox) with MSI-X interrupt enabled.
[LTC 6.0 FEAT] 201700:Support Intel 10Gb Ethernet (Knox) with MSI-X interrupt...
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: kernel (Show other bugs)
ppc64 All
high Severity high
: alpha
: 6.0
Assigned To: Ameet Paranjape
Martin Jenner
: FutureFeature
Depends On:
Blocks: 356741
  Show dependency treegraph
Reported: 2008-09-22 11:10 EDT by IBM Bug Proxy
Modified: 2010-10-18 10:26 EDT (History)
7 users (show)

See Also:
Fixed In Version:
Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2009-11-12 18:15:46 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

  None (edit)
Description IBM Bug Proxy 2008-09-22 11:10:57 EDT
=Comment: #0=================================================
Emily J. Ratliff <emilyr@us.ibm.com> - 2008-09-16 18:22 EDT
1. Feature Overview:
Feature Id:	[201700]
a. Name of Feature:	Support Intel 10Gb Ethernet (Knox) with MSI-X interrupt enabled.
b. Feature Description
Update the ixgbe driver in order to enable the MSI-X interrupt support for the Intel-based 10Gb
PCI-E adapter on p6 servers.

2. Feature Details:
Sponsor:	PPC

Arch Specificity: Purely Common Code
Affects Kernel Modules: Yes
Delivery Mechanism: Direct from community
Category:	Device Drivers and IO
Request Type:	Driver - Update Version
d. Upstream Acceptance:	In Progress
Sponsor Priority	1
f. Severity: High
IBM Confidential:	no
Code Contribution:	3rd party code
g. Component Version Target:	TBD

3. Business Case
The MSI-X interrupt provides better performance and throughput out of the high bandwidth (10Gb)
networking device.

4. Primary contact at Red Hat: 
John Jarvis

5. Primary contacts at Partner:
Project Management Contact:
Stephanie Glass, sglass@us.ibm.com, 512-838-9284

Technical contact(s):
Daisy Chang, daisyc@us.ibm.com
Paul Larson, plars@us.ibm.com

IBM Manager:
Wendel Voigt, wvoigt@us.ibm.com
Comment 1 Bill Nottingham 2008-10-01 16:38:32 EDT
What kernel version is this currently targeted for release in?
Comment 2 IBM Bug Proxy 2009-03-03 11:02:27 EST
The ixgbe driver in the kernel already has msi-x, and Breno tested this with linux-next that should go in (with kernel msi-x support patches) to 2.6.30 and found no problems.
Comment 3 Bill Nottingham 2009-03-03 17:13:16 EST
OK, marking as MODIFIED.

The feature requested has already been accepted into the upstream code base
planned for the next major release of Red Hat Enterprise Linux.

When the next milestone release of Red Hat Enterprise Linux 6 is available,
please verify that the feature requested is present and functioning as
Comment 4 IBM Bug Proxy 2009-03-04 07:41:19 EST
Moving feature to Accepted state since it is in RH cvs tree.
Comment 5 Kevin W Monroe 2009-11-12 18:15:46 EST
Closing as support is available in current release.
Comment 6 IBM Bug Proxy 2010-05-07 15:01:58 EDT
------- Comment From leitao@linux.vnet.ibm.com 2010-05-07 14:59 EDT-------
I just had chance to verify this feature, and MSI-X is not enabled on ixgbe.
It seems that only MSI is running on ixgbe.

Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit+
Capabilities: [60] MSI-X: Enable- Count=18 Masked-

Adding Andr?? as the owner fo this bug, since Paul left IBM.
Comment 7 IBM Bug Proxy 2010-05-10 19:01:14 EDT
------- Comment From adetsch@br.ibm.com 2010-05-10 18:51 EDT-------
This patch made MSI-X stop to work in ixgbe + powerpc.

After this change, we request 3 msi-x vectors, but pci_enable_msix returns only 2, which is less than MIN_MSIX_COUNT (== 3), and leads to a MSI fallback.

Trying vectors = 4,  pci_enable_msix returns the requested number, 4, and MSI-X works fine.

In summary: the commit seems to expose a problem with the pci_enable_msix logic in powerpc.

I'll keep working on that.
Comment 8 IBM Bug Proxy 2010-05-10 19:50:39 EDT
------- Comment From adetsch@br.ibm.com 2010-05-10 19:40 EDT-------
Enabling debug on rtas msi code, this is the point where we fail:
rtas_msi: ibm,change_msi(func=4,num=3), got 2 rc = 2
rtas_msi: rtas_change_msi() failed

"num=3" corresponds to the msi-x vector size. "got 2" means that the rtas call returned only 2 irqs and not 3.
Comment 9 IBM Bug Proxy 2010-05-21 13:30:54 EDT
------- Comment From adetsch@br.ibm.com 2010-05-21 13:20 EDT-------
Feature verified.
There was a recent "regression" in the driver, that made MSI-X not be enabled on machine with 2 or less cpu cores. And this problem is present in the version in RHEL6.
Not a driver regression actually, but a driver change that exposed a firmware limitation. This is still yet to be handled, and will be the subject of another bug.

Note You need to log in before you can comment on or make changes to this bug.