Bug 488924 - [RHEL5.3 Xen]: cpuspeed lies about frequency scaling
Summary: [RHEL5.3 Xen]: cpuspeed lies about frequency scaling
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: cpuspeed
Version: 5.3
Hardware: All
OS: Linux
low
medium
Target Milestone: rc
: ---
Assignee: Jarod Wilson
QA Contact: BaseOS QE
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2009-03-06 08:27 UTC by Chris Lalancette
Modified: 2013-04-12 20:07 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
If a system has more physical CPUs than the number of virtual CPUs in dom0, dom0's processes can be migrated from one physical CPU to another, resulting in frequency scaling happening on the wrong physical CPU. Therefore, frequency scaling cannot be done reliably if the number of physical CPUs in a given system and the number of virtual CPUs allocated to dom0 are not the same. To avoid this unpredictable behavior, CPU frequency scaling is now disabled if the number of virtual CPUs in dom0 does not equal the number of physical CPUs in the system, and cpuspeed alerts the user with the message "CPU freq scaling under Xen when vcpus != pcpus is not supported". Disabling CPU frequency scaling under these circumstances ensures the stability of the system.
Clone Of:
Environment:
Last Closed: 2009-04-15 10:05:24 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2009:0424 0 normal SHIPPED_LIVE cpuspeed bug fix update 2009-05-20 10:05:02 UTC

Description Chris Lalancette 2009-03-06 08:27:22 UTC
Description of problem:
In order for P-state frequency scaling to work under a Xen kernel, dom0 *must* have access to all physical CPUs on the system.  This is because dom0 is the one responsible for making decisions about when and how to frequency scale.  Additionally, dom0 also *must* have it's vcpus pinned to pcpus; otherwise, dom0 could make a decision to frequency scale processor 2, but have been migrated by the hypervisor to processor 4 by the time it gets around to writing the MSR.  In normal operation, both of these conditions are true; we automatically setup dom0 to have as many cpus as are in the system, and we pin dom0 vcpus to pcpus.

However, there are various situations that can cause this not to be true.  One example is the hypervisor command-line argument "dom0_max_vcpus"; if you set this to a number less than the total number of pcpus in the system, then the hypervisor disables frequency scaling.  Also, because of an ABI limitation, if you have > 32 cores on the system, the hypervisor also disables frequency scaling.  Other examples exist.

The problem is that even when the hypervisor has disabled frequency scaling, the cpuspeed service in the dom0 still starts successfully, and still says something like "Frequency scaling enabled using ondemand governor", which is a lie.  In the Xen case, I think we need to make the cpuspeed service a bit smarter, and fail starting the service when our dom0 vcpus are not pinned to all pcpus, with a message like "Frequency scaling disabled because Xen pcpus are not pinned to vcpus".

Comment 7 Ruediger Landmann 2009-03-25 02:49:12 UTC
Release note added. If any revisions are required, please set the 
"requires_release_notes" flag to "?" and edit the "Release Notes" field accordingly.
All revisions will be proofread by the Engineering Content Services team.

New Contents:
If a system has more physical CPUs than the number of virtual CPUs in dom0, dom0's processes can be migrated from one physical CPU to another, resulting in frequency scaling happening on the wrong physical CPU. Therefore, frequency scaling cannot be done reliably if the number of physical CPUs in a given system and the number of virtual CPUs allocated to dom0 are not the same. To avoid this unpredictable behavior, CPU frequency scaling is now disabled if the number of virtual CPUs in dom0 does not equal the number of physical CPUs in the system, and cpuspeed alerts the user with the message "CPU freq scaling under Xen when vcpus != pcpus is not supported". Disabling CPU frequency scaling under these circumstances ensures the stability of the system.

Comment 11 errata-xmlrpc 2009-04-15 10:05:24 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHBA-2009-0424.html


Note You need to log in before you can comment on or make changes to this bug.