Bug 463560

Summary: [LTC 6.0 FEAT] 201132:Dynamic CPU hotplug daemon for System z
Product: Red Hat Enterprise Linux 6 Reporter: IBM Bug Proxy <bugproxy>
Component: s390utilsAssignee: Dan Horák <dhorak>
Severity: high Docs Contact:
Priority: high    
Version: 6.0CC: ejratl, hpicht, jjarvis, notting, swells
Target Milestone: alphaKeywords: FutureFeature, OtherQA
Target Release: 6.0   
Hardware: s390x   
OS: All   
Fixed In Version: Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2009-09-17 18:05:42 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
Bug Depends On: 462976    
Bug Blocks: 519799, 356741    

Description IBM Bug Proxy 2008-09-23 21:21:19 UTC
=Comment: #0=================================================
Emily J. Ratliff <emilyr@us.ibm.com> - 2008-09-16 18:03 EDT
1. Feature Overview:
Feature Id:	[201132]
a. Name of Feature:	Dynamic CPU hotplug daemon for System z
b. Feature Description
A daemon monitoring workload and plugs and unplugs (virtual) CPUs driven by current demand for
processing units.  This dynamic hotplug daemon for workload optimization in a SMP environment -
should be part of s390-tools.

2. Feature Details:
Sponsor:	zSeries

Arch Specificity: Purely Arch Specific Code
Delivery Mechanism: Direct from community
Category:	zSeries
Request Type:	Package - Feature from IBM
d. Upstream Acceptance:	Accepted
Sponsor Priority	1
f. Severity: High
IBM Confidential:	no
Code Contribution:	IBM code
g. Component Version Target:	s390tools 1.6.3
Performance Assistance:	yes

3. Business Case
This is a requirement from customers that want to run game servers on Linux on System z.

4. Primary contact at Red Hat: 
John Jarvis

5. Primary contacts at Partner:
Project Management Contact:
Hans-Georg Markgraf, mgrf@de.ibm.com, Boeblingen 49-7031-16-3978

Technical contact(s):
Gonzalo Muelas Serrano, gmuelas@de.ibm.com

IBM Manager:
Thomas Schwarz, t.schwarz@de.ibm.com

Comment 1 Bill Nottingham 2008-10-02 19:54:47 UTC
Is there a reason this is s390 specific, and not able to be used on other virtualization frameworks?

Comment 2 Dan Horák 2008-10-06 08:11:05 UTC
from the cpuplugd manpage:
This program is useful  when executed within a z/VM or LPAR Linux environment. The CMM feature (memory hotplug) can not be used inside an LPAR environment.

The memory hotplug requires CMM which is available only when running under z/VM (but other virtualization environments could provide similar features?), but IMHO the cpu hotplug can be used generally.

Adding Hans into CC as he is the developer of this utility.

Comment 3 Hans-Joachim Picht 2008-10-06 10:38:07 UTC
This feature is currently only useful on s390. Whereas a generic interface for cpu hotplug is used, the cmm feature (for memory hotplug) is only available on s390.

The cpu hotplug functionality can also be used in other virtualization environments. we're currently testing if there'll also be a similar performance advantage as the one we've observed while running inside the z/VM oder LPAR hypervisor.

Whereas existing cpu hotplug tools are typically used to enable or disable cpus based on static system limits, this tool includes a rule based setup which allows a very fine grained configuration. 

The primary reason for you to accept this feature should be the fact that this daemon can be used (with a descent configuration) to reduce hardware requirements and to result in a increased system performance.

The following is a quote form our performance department related to this program:

<<Up to 40% more throughput, up to 40% CPU cost savings>>


With best regards,


Comment 4 Bill Nottingham 2008-10-06 18:58:52 UTC
Right, but the idea is to do things in a generic way as possible, so we don't have

- "configure your x86 servers to do XYZ"
- "but configure your s390 servers to do ABC instead"

Memory hotplug is also in the generic kernel as well, so I'm still confused as to why it's s390 specific.

Comment 5 IBM Bug Proxy 2008-10-07 11:31:20 UTC
(In reply to comment #6)
> ------- Comment From notting@redhat.com 2008-10-06 14:58:52 EDT-------
> Right, but the idea is to do things in a generic way as possible, so we don't
> have
> - "configure your x86 servers to do XYZ"
> - "but configure your s390 servers to do ABC instead"
> Memory hotplug is also in the generic kernel as well, so I'm still confused as
> to why it's s390 specific.

The problem here is that a mainframe can hardly be compared with a x86 box...
When it comes to memory hotplug this daemon uses the CONFIG_CMM kernel option which is
architecture specific.

With best regards,


Comment 6 Bill Nottingham 2008-10-07 14:28:22 UTC
See https://bugzilla.redhat.com/show_bug.cgi?id=463635#c2, which implies otherwise.

Comment 7 Dan Horák 2008-10-07 14:47:26 UTC
But I have "Access denied" for #463635 ...

Comment 9 Dan Horák 2008-10-07 16:06:36 UTC
Couldn't be the CMM compared to controlling the memory balloon driver in Xen, where z/VM is Dom0 and Linux/s390 in a VM is DomU? And kernel memory hotplug mechanism is then only reacting on changes in "virtual physical" memory?

My understanding is following:
- guest is configured with 512 MB of memory
- during operation no more then 384 MB are used
- guest (or better the cpuplugd daemon) decides to return 128 MB to its hypervisor
- if memory utilization gets higher it asks the hypervisor for additional memory

or I am wrong?

Comment 10 Dan Horák 2008-10-16 13:40:56 UTC
CPU hotplug in XEN is being added into kernel 2.6.28 - http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=d68d82afd4c8

Comment 11 Shawn Wells 2009-01-15 17:51:41 UTC
Is there an update on this?  

From reviewing the comments thus far, my understanding is that this code is upstream (per comment #0).  It looks like the s390x code was in 2.6.27 (comment #8), however x86 additions were made and committed into 2.6.28 (comment #10). 

Based upon that, it looks like this is on schedule for RHEL6.0 inclusion.  What are the next steps?

Reason for asking:
- IBM System z management is asking for status
- Customer interest from US Gov't

Just want to make sure I convey the proper messages & status.

Comment 12 IBM Bug Proxy 2009-03-03 19:41:07 UTC
The code for this feature get delivered in s390-tools with feature request:
48095  - RHBZ 462976  [LTC 6.0 FEAT] 201674: Pick up latest version of

Comment 13 John Jarvis 2009-09-17 18:05:42 UTC
duping to package rebase BZ.

*** This bug has been marked as a duplicate of bug 462976 ***