Description of problem: We have to servers running Fedora Core 5 with Xen kernel and several guest domains. Lately, the irqbalance daemons running on zero domains pm those servers have started consuming all available CPU. It probably occured after some update, but we cannot determine that - we'd have to reboot those machines, and they are constantly in use. Here's the output from xentop: xentop - 15:11:52 Xen 3.0.3-0-1.2316.fc5 12 domains: 1 running, 11 blocked, 0 paused, 0 crashed, 0 dying, 0 shutdown Mem: 6119000k total, 5739120k used, 379880k free CPUs: 4 @ 1804MHz NAME STATE CPU(sec) CPU(%) MEM(k) MEM(%) MAXMEM(k) MAXMEM(%) VCPUS NETS NETTX(k) NETRX(k) VBDS VBD_OO VBD_RD VBD_WR SSID Domain-0 -----r 247351 100.5 523160 8.5 no limit n/a 4 4 2580863 327865 0 0 0 0 0 .... then the guest domains - they consume 1.6% CPU max. Version-Release number of selected component (if applicable): irqbalance-1.12-1.25 xen-3.0.3-5.fc5 kernel-xen0-2.6.20-1.2316.fc5 on one machine, kernel-xen0-2.6.18-1.2239.fc5 on the other How reproducible: Immediately after reboot, 100% of the time
Can you please try upgrading to irqbalance-0.55-3.fc7? Its a complete rewrite and should correct lots of these sorts of problems. If it works I'll try to get it into FC-5 if I'm able. Note the version number is lower than the current fc5 release. I've set the epoch to make the upgrade transparent in the fc7 version.
should I install it on the domain 0, or on the guests as well?
you shouldn't be running irqbalance on the guests, just on the dom0
I couldn't find irqbalance-0.55-3.fc7 anywhere yet, so I've compiled and installed irqbalance-0.55-2.fc7 from http://download.fedora.redhat.com/pub/fedora/linux/core/development/source/SRPMS/irqbalance-0.55-2.fc7.src.rpm. Then stopped and started irqbalance service (no reboot since the machine hosts some minor services). It doesn't consume all CPU anymore. But how can I verify that it in fact does its job, balancing IRQ?
the same way you would have verified it previously: look at /proc/irq/<irqnum>/smp_affinity and look to see that the value contained in those files is something other than all f's
Seems it works: # find /proc/irq -type f -name smp_affinity | xargs cat | sort | uniq -c 500 01 4 02 3 04 3 08 2 0f Doesn't excessively consume CPU now - I think it can be released into FC5.
done
This report targets FC5, which is now end-of-life. Please re-test against Fedora 7 or later, and if the issue persists, open a new bug. Thanks