RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 523122 - [RHEL-6 Xen]: Cannot balloon a Xen domU guest above the initial starting memory
Summary: [RHEL-6 Xen]: Cannot balloon a Xen domU guest above the initial starting memory
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: kernel
Version: 6.1
Hardware: All
OS: Linux
low
medium
Target Milestone: rc
: ---
Assignee: Igor Mammedov
QA Contact: Virtualization Bugs
URL:
Whiteboard:
: 608610 (view as bug list)
Depends On: 725519
Blocks: 653816 GSS_6_2_PROPOSED
TreeView+ depends on / blocked
 
Reported: 2009-09-14 08:15 UTC by Chris Lalancette
Modified: 2018-11-14 15:04 UTC (History)
21 users (show)

Fixed In Version: kernel-2.6.32-176.el6
Doc Type: Bug Fix
Doc Text:
Clone Of: 499587
Environment:
Last Closed: 2011-12-06 12:23:40 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)
01/21 xen: release unused free memory (3.26 KB, patch)
2011-07-27 12:28 UTC, Igor Mammedov 		no flags Details | Diff
02/21 xen: make sure pages are really part of domain before freeing (4.09 KB, patch)
2011-07-27 12:30 UTC, Igor Mammedov 		no flags Details | Diff
03/21 xen: Rename the balloon lock (3.89 KB, patch)
2011-07-27 12:31 UTC, Igor Mammedov 		no flags Details | Diff
04/21 xen: don't map missing memory (2.18 KB, patch)
2011-07-27 12:32 UTC, Igor Mammedov 		no flags Details | Diff
05/21 xen: Use host-provided E820 map (3.78 KB, patch)
2011-07-27 12:33 UTC, Igor Mammedov 		no flags Details | Diff
06/21 xen: implement "extra" memory to reserve space for pages not present at boot (3.42 KB, patch)
2011-07-27 12:34 UTC, Igor Mammedov 		no flags Details | Diff
07/21 xen: make sure xen_extra_mem_start is beyond all non-RAM e820 (1.13 KB, patch)
2011-07-27 12:36 UTC, Igor Mammedov 		no flags Details | Diff
08/21 xen: add extra pages for E820 RAM regions, even if beyond mem_end (1.49 KB, patch)
2011-07-27 12:38 UTC, Igor Mammedov 		no flags Details | Diff
09/21 xen: limit extra memory to a certain ratio of base (2.81 KB, patch)
2011-07-27 12:39 UTC, Igor Mammedov 		no flags Details | Diff
10/21 xen: make sure xen_max_p2m_pfn is up to date (3.12 KB, patch)
2011-07-27 12:40 UTC, Igor Mammedov 		no flags Details | Diff
11/21 xen: don't add extra_pages for RAM after mem_end (2.62 KB, patch)
2011-07-27 12:41 UTC, Igor Mammedov 		no flags Details | Diff
12/21 xen: add extra pages to balloon (2.27 KB, patch)
2011-07-27 12:41 UTC, Igor Mammedov 		no flags Details | Diff
13/21 xen/balloon: make sure we only include remaining extra ram (1.77 KB, patch)
2011-07-27 12:42 UTC, Igor Mammedov 		no flags Details | Diff
14/21 xen/balloon: the balloon_lock is useless (2.22 KB, patch)
2011-07-27 12:43 UTC, Igor Mammedov 		no flags Details | Diff
15/21 xen: clean up "extra" memory handling some more (2.00 KB, patch)
2011-07-27 12:44 UTC, Igor Mammedov 		no flags Details | Diff
16/21 xen: Mark all initial reserved pages for the balloon as INVALID_P2M_ENTRY. (4.24 KB, patch)
2011-07-27 12:45 UTC, Igor Mammedov 		no flags Details | Diff
17/21 xen/balloon: Removal of driver_pages (2.58 KB, patch)
2011-07-27 12:46 UTC, Igor Mammedov 		no flags Details | Diff
18/21 xen/balloon: Use PageHighMem() for high memory page detection (1.25 KB, patch)
2011-07-27 12:47 UTC, Igor Mammedov 		no flags Details | Diff
19/21 xen/balloon: Move dec_totalhigh_pages() from __balloon_append() to balloon_append() (2.26 KB, patch)
2011-07-27 12:48 UTC, Igor Mammedov 		no flags Details | Diff
20/21 xen: prevent crashes with non-HIGHMEM 32-bit kernels with largeish memory (1.72 KB, patch)
2011-07-27 12:48 UTC, Igor Mammedov 		no flags Details | Diff
21/21 xen: x86_32: Ignore not present at boot time HIGHMEM pages (2.47 KB, patch)
2011-07-27 12:50 UTC, Igor Mammedov 		no flags Details | Diff
View All


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2011:1530 0 normal SHIPPED_LIVE Moderate: Red Hat Enterprise Linux 6 kernel security, bug fix and enhancement update 2011-12-06 01:45:35 UTC

Internal Links: 969601

Description Chris Lalancette 2009-09-14 08:15:00 UTC 
+++ This bug was initially created as a clone of Bug #499587 +++

Created an attachment (id=342822)
dmesg from the guest, showing the e820 map

Description of problem:
I have a F-11 domU configured with the following memory parameters (full libvirt XML is attached):

  <memory>15360000</memory>
  <currentMemory>786432</currentMemory>

With this configuration in place, and the dom0 appropriately ballooned, I should be able to set the amount of memory in the guest to any number between 0 and 15GB.  However, I can only balloon down below the "currentMemory" target, and back up to "currentMemory".  Anything about 768M (the starting value) just fails to allocate the memory to the domU.

This appears to at least partially be a problem with the e820 map.  I'll attach a full dmesg from the guest, but this snippet:

BIOS-provided physical RAM map:
 Xen: 0000000000000000 - 00000000000a0000 (usable)
 Xen: 00000000000a0000 - 0000000000100000 (reserved)
 Xen: 0000000000100000 - 0000000001fea000 (usable)
 Xen: 0000000001fea000 - 000000000216d000 (reserved)
 Xen: 000000000216d000 - 0000000030000000 (usable)

Shows that the guest only sees the e820 map up to 0x30000000, which is ~768M.  To be able to balloon, the guest would need to have the e820 map be all the way up to "memory" from the above XML.

--- Additional comment from clalance on 2009-05-07 06:54:56 EDT ---

Created an attachment (id=342823)
Libvirt XML from the affected domain

--- Additional comment from clalance on 2009-05-08 11:22:53 EDT ---

Miroslav,
     Another important piece of functionality that really should work.

Chris Lalancette

--- Additional comment from fedora-triage-list on 2009-06-09 11:18:18 EDT ---


This bug appears to have been reported against 'rawhide' during the Fedora 11 development cycle.
Changing version to '11'.

More information and reason for this action is here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping
Comment 1 RHEL Program Management 2009-09-14 08:19:38 UTC 
This request was evaluated by Red Hat Product Management for inclusion in a Red
Hat Enterprise Linux major release.  Product Management has requested further
review of this request by Red Hat Engineering, for potential inclusion in a Red
Hat Enterprise Linux Major release.  This request is not yet committed for
inclusion.
Comment 3 Andrew Jones 2010-06-17 15:43:07 UTC 
For an update, this problem still exists upstream. I tested with Jeremy's stable branches 

jeremy/xen/stable-2.6.32.x
jeremy/xen/stable-2.6.33.x

Jeremy,

Is there another branch I should test with? Have you seen ballooning up within maxmem's limit work?

Andrew
Comment 4 Andrew Jones 2010-06-28 09:59:02 UTC 
*** Bug 608610 has been marked as a duplicate of this bug. ***
Comment 5 Andrew Jones 2010-06-28 10:08:06 UTC 
Technical note added. If any revisions are required, please edit the "Technical Notes" field
accordingly. All revisions will be proofread by the Engineering Content Services team.

New Contents:
RHEL 6.0 XenDomU guests do not support memory ballooning.
Comment 6 Paolo Bonzini 2010-06-28 11:42:17 UTC 
Technical note updated. If any revisions are required, please edit the "Technical Notes" field
accordingly. All revisions will be proofread by the Engineering Content Services team.

Diffed Contents:
@@ -1 +1 @@
-RHEL 6.0 XenDomU guests do not support memory ballooning.+RHEL 6.0 paravirtualized Xen guests do not support memory ballooning.
Comment 7 Ryan Lerch 2010-06-29 02:24:56 UTC 
Technical note updated. If any revisions are required, please edit the "Technical Notes" field
accordingly. All revisions will be proofread by the Engineering Content Services team.

Diffed Contents:
@@ -1 +1 @@
-RHEL 6.0 paravirtualized Xen guests do not support memory ballooning.+Memory Ballooning is not a supported by Red Hat Enterprise Linux 6 paravirtualized Xen guests.
Comment 8 Andrew Jones 2010-08-24 14:13:45 UTC 
The current state upstream is

http://lists.xensource.com/archives/html/xen-devel/2010-08/msg01468.html

this is true for rhel6 as well.
Comment 9 RHEL Program Management 2010-11-18 16:20:22 UTC 
This request was evaluated by Red Hat Product Management for inclusion
in a Red Hat Enterprise Linux maintenance release. Product Management has 
requested further review of this request by Red Hat Engineering, for potential
inclusion in a Red Hat Enterprise Linux Update release for currently deployed 
products. This request is not yet committed for inclusion in an Update release.
Comment 10 Paolo Bonzini 2011-01-26 12:57:14 UTC 
See series at

http://permalink.gmane.org/gmane.comp.emulators.xen.devel/95987
http://permalink.gmane.org/gmane.comp.emulators.xen.devel/95988
http://permalink.gmane.org/gmane.comp.emulators.xen.devel/95989
http://permalink.gmane.org/gmane.comp.emulators.xen.devel/95990
http://permalink.gmane.org/gmane.comp.emulators.xen.devel/95991
http://permalink.gmane.org/gmane.comp.emulators.xen.devel/95992
http://permalink.gmane.org/gmane.comp.emulators.xen.devel/95993
http://permalink.gmane.org/gmane.comp.emulators.xen.devel/95994
Comment 11 Paolo Bonzini 2011-01-26 13:26:19 UTC 
... and these patches too:

2) boot-time ballooning series

c2d08791 xen: clean up "extra" memory handling some more
d2a81713 xen: re-enable boot-time ballooning
66946f67 xen/balloon: make sure we only include remaining extra ram
2f70e0ac xen/balloon: the balloon_lock is useless
9be4d457 xen: add extra pages to balloon
419db274 x86, memblock: Fix early_node_mem with big reserved region.
2f7acb20 xen: make sure xen_max_p2m_pfn is up to date
698bb8d1 xen: limit extra memory to a certain ratio of base
b5b43ced xen: add extra pages for E820 RAM regions, even if beyond mem_end
36bc251b xen: make sure xen_extra_mem_start is beyond all non-RAM e820
42ee1471 xen: implement "extra" memory to reserve space for pages not present
at boot
35ae11fd xen: Use host-provided E820 map
cfd8951e xen: don't map missing memory
fef5ba79 xen: Cope with unmapped pages when initializing kernel pagetable
Comment 12 Andrew Jones 2011-01-31 16:22:03 UTC 
It's too late to post/test this many patches for 6.1. Moving to 6.2.
Comment 16 RHEL Program Management 2011-05-31 16:11:16 UTC 
This request was evaluated by Red Hat Product Management for inclusion
in a Red Hat Enterprise Linux maintenance release. Product Management has 
requested further review of this request by Red Hat Engineering, for potential
inclusion in a Red Hat Enterprise Linux Update release for currently deployed 
products. This request is not yet committed for inclusion in an Update release.
Comment 27 Igor Mammedov 2011-07-27 12:28:43 UTC 
Created attachment 515491 [details]
01/21 xen: release unused free memory
Comment 28 Igor Mammedov 2011-07-27 12:30:08 UTC 
Created attachment 515492 [details]
02/21 xen: make sure pages are really part of domain before freeing
Comment 29 Igor Mammedov 2011-07-27 12:31:10 UTC 
Created attachment 515493 [details]
03/21 xen: Rename the balloon lock
Comment 30 Igor Mammedov 2011-07-27 12:32:56 UTC 
Created attachment 515494 [details]
04/21 xen: don't map missing memory
Comment 31 Igor Mammedov 2011-07-27 12:33:55 UTC 
Created attachment 515496 [details]
05/21 xen: Use host-provided E820 map
Comment 32 Igor Mammedov 2011-07-27 12:34:50 UTC 
Created attachment 515498 [details]
06/21 xen: implement "extra" memory to reserve space for pages not present at boot
Comment 33 Igor Mammedov 2011-07-27 12:36:11 UTC 
Created attachment 515499 [details]
07/21 xen: make sure xen_extra_mem_start is beyond all non-RAM e820
Comment 34 Igor Mammedov 2011-07-27 12:38:19 UTC 
Created attachment 515500 [details]
08/21 xen: add extra pages for E820 RAM regions, even if beyond mem_end
Comment 35 Igor Mammedov 2011-07-27 12:39:31 UTC 
Created attachment 515502 [details]
09/21 xen: limit extra memory to a certain ratio of base
Comment 36 Igor Mammedov 2011-07-27 12:40:24 UTC 
Created attachment 515503 [details]
10/21 xen: make sure xen_max_p2m_pfn is up to date
Comment 37 Igor Mammedov 2011-07-27 12:41:14 UTC 
Created attachment 515504 [details]
11/21 xen: don't add extra_pages for RAM after mem_end
Comment 38 Igor Mammedov 2011-07-27 12:41:57 UTC 
Created attachment 515505 [details]
12/21 xen: add extra pages to balloon
Comment 39 Igor Mammedov 2011-07-27 12:42:44 UTC 
Created attachment 515506 [details]
13/21 xen/balloon: make sure we only include remaining extra ram
Comment 40 Igor Mammedov 2011-07-27 12:43:33 UTC 
Created attachment 515507 [details]
14/21 xen/balloon: the balloon_lock is useless
Comment 41 Igor Mammedov 2011-07-27 12:44:30 UTC 
Created attachment 515508 [details]
15/21 xen: clean up "extra" memory handling some more
Comment 42 Igor Mammedov 2011-07-27 12:45:28 UTC 
Created attachment 515510 [details]
16/21 xen: Mark all initial reserved pages for the balloon as INVALID_P2M_ENTRY.
Comment 43 Igor Mammedov 2011-07-27 12:46:12 UTC 
Created attachment 515511 [details]
17/21 xen/balloon: Removal of driver_pages
Comment 44 Igor Mammedov 2011-07-27 12:47:08 UTC 
Created attachment 515512 [details]
18/21 xen/balloon: Use PageHighMem() for high memory page detection
Comment 45 Igor Mammedov 2011-07-27 12:48:05 UTC 
Created attachment 515514 [details]
19/21 xen/balloon: Move dec_totalhigh_pages() from  __balloon_append() to balloon_append()
Comment 46 Igor Mammedov 2011-07-27 12:48:53 UTC 
Created attachment 515515 [details]
20/21 xen: prevent crashes with non-HIGHMEM 32-bit kernels  with largeish memory
Comment 47 Igor Mammedov 2011-07-27 12:50:42 UTC 
Created attachment 515517 [details]
21/21 xen: x86_32: Ignore not present at boot time HIGHMEM  pages
Comment 48 Aristeu Rozanski 2011-08-02 13:58:36 UTC 
Patch(es) available on kernel-2.6.32-176.el6
Comment 54 Yuyu Zhou 2011-08-25 07:48:36 UTC 
test with RHEL6.2 x86_64 pv guest, (Kernel 2.6.32-191.el6), now failed in Intel W3520 and Intel Q9400, Pass in amd B95 and amd 2427.

In Intel platform:
Set memory below the initial starting memory - pass
Set memory above the initial starting memory -- fail

In AMD paltform:
Set memory below the initial starting memory - pass
Set memory above the initial starting memory -- pass
Comment 55 Igor Mammedov 2011-08-25 08:32:15 UTC 
Can you(In reply to comment #54)
> test with RHEL6.2 x86_64 pv guest, (Kernel 2.6.32-191.el6), now failed in Intel
> W3520 and Intel Q9400, Pass in amd B95 and amd 2427.
> 
> In Intel platform:
> Set memory below the initial starting memory - pass
> Set memory above the initial starting memory -- fail

Can you provide access to one of this boxes where test fails?
Comment 57 Igor Mammedov 2011-08-25 11:07:19 UTC 
Guest wasn't provided with enough memory:
--------------------
Aug 23 05:42:54 virtlab-66-84-79 kernel: BIOS-provided physical RAM map:
Aug 23 05:42:54 virtlab-66-84-79 kernel: Xen: 0000000000000000 - 00000000000a0000 (usable)
0 (reserved)
Aug 23 05:42:54 virtlab-66-84-79 kernel: Xen: 0000000000100000 - 0000000040800000 (usable)
Aug 23 05:42:54 virtlab-66-84-79 kernel: DMI not present or invalid.
Aug 23 05:42:54 virtlab-66-84-79 kernel: last_pfn = 0x40800 max_arch_pfn = 0x400000000
---------------------
0x40800 is roughly 1032Mb and it is host's problem.

If you execute xm info on this host you'll see:
total_memory           : 8125
free_memory            : 1035

if you balloon down dom0 to free enough memory guest will be provided with correct map and you'll be able to balloon it up and down.

something like this:
# xm mem-set 0 4096
# xm info|grep mem
total_memory           : 8125
free_memory            : 3823

now you can boot guest and verify if ballooning works.

PS:
Maybe you should compare versions of xen tools used on this box and on amd box and free_memory param on both boxes as well.
Comment 58 Yuyu Zhou 2011-08-26 02:57:32 UTC 
Hello, Igor
Pass in all conditions after enable 'auto-balloon-dom0' in xend-config.sxp
Thanks for your help.
Yuyu Zhou
Comment 59 Laszlo Ersek 2011-08-26 07:46:06 UTC 
Hi Yuyu,

(In reply to comment #58)

> Pass in all conditions after enable 'auto-balloon-dom0' in xend-config.sxp

would it be appropriate to set this BZ to VERIFIED then? Thanks!
Comment 60 Yuyu Zhou 2011-08-26 08:17:58 UTC 
Verified with RHEL6.2 x86_64 pv guest, (Kernel 2.6.32-191.el6).

In Intel platform:
Set memory below the initial starting memory - pass
Set memory above the initial starting memory -- pass

In AMD paltform:
Set memory below the initial starting memory - pass
Set memory above the initial starting memory -- pass
Comment 62 Qixiang Wan 2011-08-26 09:39:38 UTC 
Deleted Technical Notes Contents.

Old Contents:
Memory Ballooning is not a supported by Red Hat Enterprise Linux 6 paravirtualized Xen guests.
Comment 63 errata-xmlrpc 2011-12-06 12:23:40 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHSA-2011-1530.html
Note You need to log in before you can comment on or make changes to this bug.