Bug 633388

Summary: sfc: creates too many queues
Product: Red Hat Enterprise Linux 5 Reporter: Michal Schmidt <mschmidt>
Component: kernelAssignee: Michal Schmidt <mschmidt>
Status: CLOSED ERRATA QA Contact: Network QE <network-qe>
Severity: medium Docs Contact:
Priority: low    
Version: 5.6CC: bhutchings, cward, lzheng
Target Milestone: rc   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2011-01-13 21:17:47 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
[PATCH] topology: rename topology_{thread,core}_cpumask back to *_siblings none

Description Michal Schmidt 2010-09-13 17:09:54 UTC 
Description of problem:
As noted by Ben Hutchings in https://bugzilla.redhat.com/show_bug.cgi?id=556476#c15:

 The patch "linux-2.6-misc-add-thread-core-_siblings_list-to-sys.patch"
 renames the macros topology_core_siblings and topology_thread_siblings
 to topology_core_cpumask and topology_thread_cpumask.

 sfc attempts to allocate one set of queues per package, falling back
 to one per core. It uses the macro topology_core_siblings, if defined,
 and therefore will always use the fallback now. Also,
 the topology_*_cpumask macros return a different type from their
 mainline implementations, so I don't think the renaming makes sense.

Version-Release number of selected component (if applicable):
kernel-2.6.18-220.el5

How reproducible:
always

Steps to Reproduce:
1. use a machine with a Solarflare NIC (sfc driver)
2. cat /proc/interrupts
  
Actual results:
The card (eth2 in my case) uses one MSI-X interrupt (associated with a queue) per processor. E.g. with CPU Intel Xeon W3520 (4 cores + hyperthreading):

 98:          0          ...          0       PCI-MSI-X  eth2-0
106:          0          ...          0       PCI-MSI-X  eth2-1
114:          0          ...          0       PCI-MSI-X  eth2-2
122:          0          ...          0       PCI-MSI-X  eth2-3
130:          0          ...          0       PCI-MSI-X  eth2-4
138:          0          ...          0       PCI-MSI-X  eth2-5
146:          0          ...          0       PCI-MSI-X  eth2-6
154:          0          ...          0       PCI-MSI-X  eth2-7

Expected results:
The card should use one MSI-X interrupt per CPU package:

 90:          0          ...          0       PCI-MSI-X  eth2-0
Comment 1 Michal Schmidt 2010-09-13 17:13:34 UTC 
Created attachment 446999 [details]
[PATCH] topology: rename topology_{thread,core}_cpumask back to *_siblings

Brew build: https://brewweb.devel.redhat.com/taskinfo?taskID=2754167
Comment 2 Michal Schmidt 2010-09-13 17:17:26 UTC 
Notes for QE:

It can be tested that the patch does not change the values in
/sys/devices/system/cpu/cpu*/topology/*_siblings*

I have Solarflare cards and can test the sfc driver.
Comment 3 RHEL Program Management 2010-09-13 17:20:44 UTC 
This request was evaluated by Red Hat Product Management for inclusion in a Red
Hat Enterprise Linux maintenance release.  Product Management has requested
further review of this request by Red Hat Engineering, for potential
inclusion in a Red Hat Enterprise Linux Update release for currently deployed
products.  This request is not yet committed for inclusion in an Update
release.
Comment 4 Michal Schmidt 2010-09-13 17:22:01 UTC 
http://post-office.corp.redhat.com/archives/rhkernel-list/2010-September/msg00640.html
Comment 7 Jarod Wilson 2010-09-17 14:03:36 UTC 
in kernel-2.6.18-222.el5
You can download this test kernel from http://people.redhat.com/jwilson/el5

Detailed testing feedback is always welcomed.
Comment 9 Michal Schmidt 2010-09-17 15:10:23 UTC 
On my machine with kernel-2.6.18-221.el5 an sfc interface took 8 interrupts (one per logical processor).

With kernel-2.6.18-222.el5 I get allocated one interrupt for an sfc interface. I have one CPU package, so this is as expected.
Comment 10 Ben Hutchings 2010-10-07 15:26:50 UTC 
I'm satisfied that this is fixed in kernel-2.6.18-222.el5.
Comment 13 errata-xmlrpc 2011-01-13 21:17:47 UTC 
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHSA-2011-0017.html