Bug 1010264 - smp_affinity broken?
Summary: smp_affinity broken?
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Fedora
Classification: Fedora
Component: kernel
Version: 19
Hardware: x86_64
OS: Linux
unspecified
medium
Target Milestone: ---
Assignee: Neil Horman
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2013-09-20 12:09 UTC by Markus Stockhausen
Modified: 2013-09-20 15:27 UTC (History)
8 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2013-09-20 15:27:55 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Markus Stockhausen 2013-09-20 12:09:10 UTC
Description of problem:

my knowledge about IRQ SMP affinity may be a little bit outdated but I found no newer documentation about the topic. In Fedora 19 I'm no longer able to change the interrupt affinity for my intel network card so it gets served by two or more CPUs:

Version-Release number of selected component (if applicable):

Fedora 19 kernel 3.l1.1

How reproducible:

100%

Steps to Reproduce:

1. stop irqbalance daemon
2. get interrupt of network card

[root@colovn1 ~]# cat /proc/interrupts | grep p1
 71:    2542650          0          0         55  PCI-MSI-edge      p49p1-rx-0
 72:         11    4324139          0          0  PCI-MSI-edge      p49p1-tx-0

3. set SMP affinity of IRQ 71 to CPU0 & CPU1

 echo 3 > /proc/irq/71/smp_affinity

4. transfer data over the network and watch the interrupt values. 
 
Actual results:

only CPU0 serves interrupts

Expected results:

CPU0 and CPU1 get interrupts of network card

Additional info:

/sys/class/net/p49p1/queues/rx-0/rps_cpus is not set:

cat /sys/class/net/p49p1/queues/rx-0/rps_cpus
00000000,00000000,00000000,00000000

Card is

[root@colovn1 ~]# lspci | grep -i net
01:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection
02:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection

Comment 1 Neil Horman 2013-09-20 15:27:55 UTC
Setting affinity for a given irq to multiple cpus does not imply that irq will be equally serviced by those cpus (quite the opposite in fact).  Setting smp affinity just marks the respective cpus in the mask as elligible to service that cpu.  The actual decision as to which cpu handles the irq is made by hardware, and the hardware policy is to have a given irq serviced by the same cpu (as determined by a bios configured priority value for each cpu in the system).  Only if the highest priority cpu of the elligible cpu set in smp_affinity is busy (handling another interrupt), will another cpu in the set be selected.  This increases the likelyhood of L2 cache hits for instructions in the asserted irqs software handler, which is where the performance increase in setting smp affinity is found.

Unless you want to do something specfic for some very uncommon requirements, just run irqbalance, and the right things should happen.


Note You need to log in before you can comment on or make changes to this bug.