Bug 743590

Summary: x86_64 xen guest crash when booting with maxmem = 128Gb
Product: Red Hat Enterprise Linux 6 Reporter: Igor Mammedov <imammedo>
Component: kernelAssignee: Igor Mammedov <imammedo>
Status: CLOSED ERRATA QA Contact: Virtualization Bugs <virt-bugs>
Severity: high Docs Contact:
Priority: high    
Version: 6.2CC: drjones, leiwang, mjenner, qguan, qwan, xen-maint, yuzhou
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: kernel-2.6.32-209.el6 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2011-12-06 14:15:56 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 653816, 716498, 748554    
Attachments:
Description Flags
[RHEL6.2 PATCH] xen: use maximum reservation to limit amount of usable RAM none

Description Igor Mammedov 2011-10-05 13:14:52 UTC 
x86_64 xen guest will hit BUG_ON at arch/x86/xen/mmu.c:185
xen_memory_setup=>xen_add_extra_mem=>__set_phys_to_machine=>p2m_top_index

   BUG_ON(pfn >= MAX_DOMAIN_PAGES);

Affects RHEL6.2 since 27109d2 commit due to introduction of PV ballooning code "xen_add_extra_mem=>...".

Problem description:

Xen starts pv guest setting guest's e820 RAM entry size to maxmem + 8Mb.
When memory == maxmem = 128Gb, the guest will see HV supplied e820 map with 128Gb+8Mb entry (0x2000800 pages). But xen_start_info->nr_pages will have value 0x2000000 (128Gb in pages). So guest assumes that extra 0x800 pages are ballooning memory and tries to initialize it. However this extra pages are located beyond of guest supported MAX_DOMAIN_PAGES. Which leads to BUG_ON.


Expected behaviour:
 Guest should not crash if host provided e820 map has more RAM then supported by guest.
Comment 1 Igor Mammedov 2011-10-05 13:23:17 UTC 
Could be fixed by capping the RAM region size to supported limit:

Relevant upstream commits to backport:
  d312ae878b6 xen: use maximum reservation to limit amount of usable RAM
  e3b73c4a25e xen/e820: if there is no dom0_mem=, don't tweak extra_pages
Comment 2 RHEL Program Management 2011-10-05 13:31:43 UTC 
This request was evaluated by Red Hat Product Management for inclusion
in a Red Hat Enterprise Linux maintenance release. Product Management has 
requested further review of this request by Red Hat Engineering, for potential
inclusion in a Red Hat Enterprise Linux Update release for currently deployed 
products. This request is not yet committed for inclusion in an Update release.
Comment 3 Igor Mammedov 2011-10-06 08:02:35 UTC 
Created attachment 526642 [details]
[RHEL6.2 PATCH] xen: use maximum reservation to limit amount of usable RAM
Comment 4 Aristeu Rozanski 2011-10-13 15:27:10 UTC 
Patch(es) available on kernel-2.6.32-209.el6
Comment 8 Qin Guan 2011-10-31 04:08:48 UTC 
Verify this problem with kernel-2.6.32-211.el6. 

Host:
2.6.18-294.el5xen
xen-3.0.3-135.el5

Guest:
kernel-2.6.32-211.el6

Steps:
1. Create a RHEL6.2 PV guest with memory set as 128G(131072M)
2. Check dmesg after guest started up 
3. Do some sanity check for the guest, such as reboot, halt, pause/unpause
4. Save and restore the guest
5. Do guest live migration with "xenheap_megabytes=48" set in xen.gz cmd line (rhbz #728877 comment 13)

No any problem found during the testing.
Comment 9 errata-xmlrpc 2011-12-06 14:15:56 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHSA-2011-1530.html