Bug 1010264 - smp_affinity broken?
smp_affinity broken?
Status: CLOSED NOTABUG
Product: Fedora
Classification: Fedora
Component: kernel (Show other bugs)
19
x86_64 Linux
unspecified Severity medium
: ---
: ---
Assigned To: Neil Horman
Fedora Extras Quality Assurance
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2013-09-20 08:09 EDT by Markus Stockhausen
Modified: 2013-09-20 11:27 EDT (History)
8 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2013-09-20 11:27:55 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Markus Stockhausen 2013-09-20 08:09:10 EDT
Description of problem:

my knowledge about IRQ SMP affinity may be a little bit outdated but I found no newer documentation about the topic. In Fedora 19 I'm no longer able to change the interrupt affinity for my intel network card so it gets served by two or more CPUs:

Version-Release number of selected component (if applicable):

Fedora 19 kernel 3.l1.1

How reproducible:

100%

Steps to Reproduce:

1. stop irqbalance daemon
2. get interrupt of network card

[root@colovn1 ~]# cat /proc/interrupts | grep p1
 71:    2542650          0          0         55  PCI-MSI-edge      p49p1-rx-0
 72:         11    4324139          0          0  PCI-MSI-edge      p49p1-tx-0

3. set SMP affinity of IRQ 71 to CPU0 & CPU1

 echo 3 > /proc/irq/71/smp_affinity

4. transfer data over the network and watch the interrupt values. 
 
Actual results:

only CPU0 serves interrupts

Expected results:

CPU0 and CPU1 get interrupts of network card

Additional info:

/sys/class/net/p49p1/queues/rx-0/rps_cpus is not set:

cat /sys/class/net/p49p1/queues/rx-0/rps_cpus
00000000,00000000,00000000,00000000

Card is

[root@colovn1 ~]# lspci | grep -i net
01:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection
02:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection
Comment 1 Neil Horman 2013-09-20 11:27:55 EDT
Setting affinity for a given irq to multiple cpus does not imply that irq will be equally serviced by those cpus (quite the opposite in fact).  Setting smp affinity just marks the respective cpus in the mask as elligible to service that cpu.  The actual decision as to which cpu handles the irq is made by hardware, and the hardware policy is to have a given irq serviced by the same cpu (as determined by a bios configured priority value for each cpu in the system).  Only if the highest priority cpu of the elligible cpu set in smp_affinity is busy (handling another interrupt), will another cpu in the set be selected.  This increases the likelyhood of L2 cache hits for instructions in the asserted irqs software handler, which is where the performance increase in setting smp affinity is found.

Unless you want to do something specfic for some very uncommon requirements, just run irqbalance, and the right things should happen.

Note You need to log in before you can comment on or make changes to this bug.