Bug 584581 - dom0_max_vcpus=2 is not enough to allow pinning dom0's VCPUs
Summary: dom0_max_vcpus=2 is not enough to allow pinning dom0's VCPUs
Alias: None
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: kernel-xen
Version: 5.5
Hardware: All
OS: Linux
Target Milestone: rc
: ---
Assignee: Paolo Bonzini
QA Contact: Virtualization Bugs
Depends On:
Blocks: 514491
TreeView+ depends on / blocked
Reported: 2010-04-21 21:21 UTC by Bill Braswell
Modified: 2018-11-14 17:42 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Last Closed: 2011-04-05 15:49:02 UTC
Target Upstream Version:

Attachments (Terms of Use)
patch to fix the bug (1.35 KB, patch)
2011-03-25 16:30 UTC, Paolo Bonzini
no flags Details | Diff

Description Bill Braswell 2010-04-21 21:21:04 UTC
Description of problem:
Xen 'xm vcpu-pin 0 0 0' does not work:

# /usr/sbin/xm vcpu-pin 0 0 0
Error: (22, 'Invalid argument')
Usage: xm vcpu-pin <Domain<VCPU<CPUs>

However, the customer tested upstream Xen 3.4.2 and it worked fine:

# /usr/sbin/xm vcpu-pin 0 0 0
# /usr/sbin/xm vcpu-pin 0 1 2
# /usr/sbin/xm vcpu-list
Name                                ID  VCPU   CPU State   Time(s) CPU
Domain-0                             0     0     0   -b-    2293.2 0
Domain-0                             0     1     2   r--     492.6 2

Version-Release number of selected component (if applicable):
5.5, 2.6.18-194.el5xen
Customer is able to reproduce the problem on a RHEL 5.4 xen system

Comment 2 Paolo Bonzini 2010-04-22 07:30:06 UTC
Try booting with dom0_vcpus_pin=0, it was changed by

commit c41ed05dc7a3078fe2234dc8c65577c0695689c7
Author: Rik van Riel <riel@redhat.com>
Date:   Thu Dec 13 15:13:52 2007 -0500

    [xen] disable cpu freq scaling when vcpus is small

Comment 4 Paolo Bonzini 2010-04-28 11:28:45 UTC
1) The option goes on the xen kernel line.  Sorry for not being more specific.

2) "small" means actually "not all of the host CPUs".  This can happen at boot time due to the "dom0_max_vcpus" option, or at runtime due to "xm vcpu-pin".  The hypervisor does not support disabling/enabling CPU frequency scaling at run-time, so it has to make a decision at boot time: whether to enable CPU frequency scaling, or whether to allow pinning dom0's VCPUs.

What the patch does then is the following:

- add a command line option dom0_vcpus_pin

- if the option is 1, disable dom0 VCPU pinning *and* enable CPU frequency scaling

The option's default value depending on "dom0_max_vcpus".  If it is not found or equal to the number of physical CPUs, dom0_vcpus_pin will be 1 by default.  If it is different from the number of physical CPUs (which typically means "small"), dom0_vcpus_pin will be 0 by default.

So, an alternative (and if applicable, better) fix will be to set dom0_max_vcpus in the Xen kernel line.

3) I agree that this should be documented.  The behavior changed in 5.2 by the way.

Comment 6 Paolo Bonzini 2010-04-29 16:32:09 UTC
I must admit I wrote everything from reading the code rather than really trying it out.

This code that actually uses the value of opt_dom0_vcpus_pin

    if ( (domid == 0) && opt_dom0_vcpus_pin )
        d->is_pinned = 1;

runs after "Brought up 4 CPUs" is printed.  This is before the value of dom0_max_vcpus is taken into account (which is just after "Dom0 has maximum 2 VCPUs" is printed).

So indeed dom0_vcpus_pin=0 is necessary too, this is a minor bug.

Comment 8 Paolo Bonzini 2010-08-02 09:52:35 UTC
I haven't really looked at it.  Since there is a workaround, we didn't give it a high priority.  However, it is a bug and it's going to be fixed, if not for 5.6.

Comment 11 Paolo Bonzini 2011-03-25 16:30:19 UTC
Created attachment 487594 [details]
patch to fix the bug

One-line patch, will ask for exception

Note You need to log in before you can comment on or make changes to this bug.