Bug 584581 - dom0_max_vcpus=2 is not enough to allow pinning dom0's VCPUs
dom0_max_vcpus=2 is not enough to allow pinning dom0's VCPUs
Status: CLOSED NOTABUG
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: kernel-xen (Show other bugs)
5.5
All Linux
medium Severity medium
: rc
: ---
Assigned To: Paolo Bonzini
Virtualization Bugs
:
Depends On:
Blocks: 514491
  Show dependency treegraph
 
Reported: 2010-04-21 17:21 EDT by Bill Braswell
Modified: 2011-04-06 12:50 EDT (History)
7 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2011-04-05 11:49:02 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
patch to fix the bug (1.35 KB, patch)
2011-03-25 12:30 EDT, Paolo Bonzini
no flags Details | Diff

  None (edit)
Description Bill Braswell 2010-04-21 17:21:04 EDT
Description of problem:
Xen 'xm vcpu-pin 0 0 0' does not work:

# /usr/sbin/xm vcpu-pin 0 0 0
Error: (22, 'Invalid argument')
Usage: xm vcpu-pin <Domain<VCPU<CPUs>

However, the customer tested upstream Xen 3.4.2 and it worked fine:

# /usr/sbin/xm vcpu-pin 0 0 0
# /usr/sbin/xm vcpu-pin 0 1 2
# /usr/sbin/xm vcpu-list
Name                                ID  VCPU   CPU State   Time(s) CPU
Affinity
Domain-0                             0     0     0   -b-    2293.2 0
Domain-0                             0     1     2   r--     492.6 2
 .
 .
 .

Version-Release number of selected component (if applicable):
5.5, 2.6.18-194.el5xen
Customer is able to reproduce the problem on a RHEL 5.4 xen system
Comment 2 Paolo Bonzini 2010-04-22 03:30:06 EDT
Try booting with dom0_vcpus_pin=0, it was changed by

commit c41ed05dc7a3078fe2234dc8c65577c0695689c7
Author: Rik van Riel <riel@redhat.com>
Date:   Thu Dec 13 15:13:52 2007 -0500

    [xen] disable cpu freq scaling when vcpus is small
Comment 4 Paolo Bonzini 2010-04-28 07:28:45 EDT
1) The option goes on the xen kernel line.  Sorry for not being more specific.


2) "small" means actually "not all of the host CPUs".  This can happen at boot time due to the "dom0_max_vcpus" option, or at runtime due to "xm vcpu-pin".  The hypervisor does not support disabling/enabling CPU frequency scaling at run-time, so it has to make a decision at boot time: whether to enable CPU frequency scaling, or whether to allow pinning dom0's VCPUs.

What the patch does then is the following:

- add a command line option dom0_vcpus_pin

- if the option is 1, disable dom0 VCPU pinning *and* enable CPU frequency scaling

The option's default value depending on "dom0_max_vcpus".  If it is not found or equal to the number of physical CPUs, dom0_vcpus_pin will be 1 by default.  If it is different from the number of physical CPUs (which typically means "small"), dom0_vcpus_pin will be 0 by default.

So, an alternative (and if applicable, better) fix will be to set dom0_max_vcpus in the Xen kernel line.


3) I agree that this should be documented.  The behavior changed in 5.2 by the way.
Comment 6 Paolo Bonzini 2010-04-29 12:32:09 EDT
I must admit I wrote everything from reading the code rather than really trying it out.

This code that actually uses the value of opt_dom0_vcpus_pin

    if ( (domid == 0) && opt_dom0_vcpus_pin )
        d->is_pinned = 1;

runs after "Brought up 4 CPUs" is printed.  This is before the value of dom0_max_vcpus is taken into account (which is just after "Dom0 has maximum 2 VCPUs" is printed).

So indeed dom0_vcpus_pin=0 is necessary too, this is a minor bug.
Comment 8 Paolo Bonzini 2010-08-02 05:52:35 EDT
I haven't really looked at it.  Since there is a workaround, we didn't give it a high priority.  However, it is a bug and it's going to be fixed, if not for 5.6.
Comment 11 Paolo Bonzini 2011-03-25 12:30:19 EDT
Created attachment 487594 [details]
patch to fix the bug

One-line patch, will ask for exception

Note You need to log in before you can comment on or make changes to this bug.