Red Hat Bugzilla – Bug 584581
dom0_max_vcpus=2 is not enough to allow pinning dom0's VCPUs
Last modified: 2011-04-06 12:50:12 EDT
Description of problem: Xen 'xm vcpu-pin 0 0 0' does not work: # /usr/sbin/xm vcpu-pin 0 0 0 Error: (22, 'Invalid argument') Usage: xm vcpu-pin <Domain<VCPU<CPUs> However, the customer tested upstream Xen 3.4.2 and it worked fine: # /usr/sbin/xm vcpu-pin 0 0 0 # /usr/sbin/xm vcpu-pin 0 1 2 # /usr/sbin/xm vcpu-list Name ID VCPU CPU State Time(s) CPU Affinity Domain-0 0 0 0 -b- 2293.2 0 Domain-0 0 1 2 r-- 492.6 2 . . . Version-Release number of selected component (if applicable): 5.5, 2.6.18-194.el5xen Customer is able to reproduce the problem on a RHEL 5.4 xen system
Try booting with dom0_vcpus_pin=0, it was changed by commit c41ed05dc7a3078fe2234dc8c65577c0695689c7 Author: Rik van Riel <riel@redhat.com> Date: Thu Dec 13 15:13:52 2007 -0500 [xen] disable cpu freq scaling when vcpus is small
1) The option goes on the xen kernel line. Sorry for not being more specific. 2) "small" means actually "not all of the host CPUs". This can happen at boot time due to the "dom0_max_vcpus" option, or at runtime due to "xm vcpu-pin". The hypervisor does not support disabling/enabling CPU frequency scaling at run-time, so it has to make a decision at boot time: whether to enable CPU frequency scaling, or whether to allow pinning dom0's VCPUs. What the patch does then is the following: - add a command line option dom0_vcpus_pin - if the option is 1, disable dom0 VCPU pinning *and* enable CPU frequency scaling The option's default value depending on "dom0_max_vcpus". If it is not found or equal to the number of physical CPUs, dom0_vcpus_pin will be 1 by default. If it is different from the number of physical CPUs (which typically means "small"), dom0_vcpus_pin will be 0 by default. So, an alternative (and if applicable, better) fix will be to set dom0_max_vcpus in the Xen kernel line. 3) I agree that this should be documented. The behavior changed in 5.2 by the way.
I must admit I wrote everything from reading the code rather than really trying it out. This code that actually uses the value of opt_dom0_vcpus_pin if ( (domid == 0) && opt_dom0_vcpus_pin ) d->is_pinned = 1; runs after "Brought up 4 CPUs" is printed. This is before the value of dom0_max_vcpus is taken into account (which is just after "Dom0 has maximum 2 VCPUs" is printed). So indeed dom0_vcpus_pin=0 is necessary too, this is a minor bug.
I haven't really looked at it. Since there is a workaround, we didn't give it a high priority. However, it is a bug and it's going to be fixed, if not for 5.6.
Created attachment 487594 [details] patch to fix the bug One-line patch, will ask for exception