Bug 459033 - [Feat] (Public) Have Xen automatically set parameters for 256g system
Summary: [Feat] (Public) Have Xen automatically set parameters for 256g system
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: kernel-xen
Version: 5.2
Hardware: All
OS: Linux
high
high
Target Milestone: rc
: ---
Assignee: Xen Maintainance List
QA Contact: Martin Jenner
URL:
Whiteboard:
Depends On:
Blocks: 448753
TreeView+ depends on / blocked
 
Reported: 2008-08-13 20:19 UTC by Russell Doty
Modified: 2009-01-28 13:30 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2009-01-28 13:30:12 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
meminfo after applying patch (682 bytes, text/plain)
2008-08-26 19:54 UTC, Sandy Garza
no flags Details

Description Russell Doty 2008-08-13 20:19:10 UTC
This bug was originally filed and worked as a private issue. This public bug is being created to allow testing and feedback by all vendors.

Please report results - success or failure - of testing with Xen on 256GB or larger systems

*****************************************************************

Original Description of problem:
hang when boot RHEL 5.2 xen on 256g/32cores system

How reproducible:
install that on RHEL 5.2 and boot 

Steps to Reproduce:
install that on RHEL 5.2 and boot 

Actual results:
hang

Expected results:
booted

Additional info:
mem=224g get more info

mem=224g

 http://www.cl.cam.ac.uk/netos/xeng
 University of Cambridge Computer Laboratorytes]

 Xen version 3.1.2-92.el5 (brewbuilder.com) (gcc version 4.1.2 20071124 (Red Hat 4.1.2-41)) Tue Apr 29 13:12:52 EDT 2008
 Latest ChangeSet: unavailable

(XEN) Command line: com1=115200,8n1 console=com1 mem=224g
(XEN) Video information:
(XEN)  VGA is text mode 80x25, font 8x16
(XEN)  VBE/DDC methods: V2; EDID transfer time: 2 seconds
(XEN) Disc information:
(XEN)  Found 3 MBR signatures                                                   (XEN)  Found 3 EDD information structures
(XEN) Truncating memory map to 234881024kB
(XEN) Xen-e820 RAM map:
(XEN)  0000000000000000 - 0000000000095800 (usable)
(XEN)  0000000000095800 - 00000000000a0000 (reserved)
(XEN)  00000000000e6000 - 0000000000100000 (reserved)
(XEN)  0000000000100000 - 00000000dffa0000 (usable)
(XEN)  00000000dffae000 - 00000000dffb0000 type 9
(XEN)  00000000dffb0000 - 00000000dffbe000 (ACPI data)
(XEN)  00000000dffbe000 - 00000000dfff0000 (ACPI NVS)
(XEN)  00000000dfff0000 - 00000000dfffe000 (reserved)
(XEN)  00000000e0000000 - 00000000f0000000 (reserved)
(XEN)  00000000fec00000 - 00000000fec01000 (reserved)
(XEN)  00000000fee00000 - 00000000fee01000 (reserved)
(XEN)  00000000ff700000 - 0000000100000000 (reserved)
(XEN)  0000000100000000 - 0000003800000000 (usable)
(XEN) System RAM: 228863MB (234355924kB)
(XEN) Xen heap: 7MB (7872kB)
(XEN) Domain heap initialised: DMA width 32 bits
(XEN) Processor #4 0:2 APIC version 16
(XEN) Processor #5 0:2 APIC version 16
(XEN) Processor #6 0:2 APIC version 16
(XEN) Processor #7 0:2 APIC version 16
(XEN) Processor #8 0:2 APIC version 16
(XEN) Processor #9 0:2 APIC version 16
(XEN) Processor #10 0:2 APIC version 16
(XEN) Processor #11 0:2 APIC version 16
(XEN) Processor #12 0:2 APIC version 16
(XEN) Processor #13 0:2 APIC version 16
(XEN) Processor #14 0:2 APIC version 16
(XEN) Processor #15 0:2 APIC version 16
(XEN) Processor #16 0:2 APIC version 16
(XEN) Processor #17 0:2 APIC version 16
(XEN) Processor #18 0:2 APIC version 16
(XEN) Processor #19 0:2 APIC version 16
(XEN) Processor #20 0:2 APIC version 16
(XEN) Processor #21 0:2 APIC version 16
(XEN) Processor #22 0:2 APIC version 16
(XEN) Processor #23 0:2 APIC version 16
(XEN) Processor #24 0:2 APIC version 16
(XEN) Processor #25 0:2 APIC version 16
(XEN) Processor #26 0:2 APIC version 16
(XEN) Processor #27 0:2 APIC version 16
(XEN) Processor #28 0:2 APIC version 16
(XEN) Processor #29 0:2 APIC version 16
(XEN) Processor #30 0:2 APIC version 16
(XEN) Processor #31 0:2 APIC version 16
(XEN) Processor #32 0:2 APIC version 16
(XEN) Processor #33 0:2 APIC version 16
(XEN) Processor #34 0:2 APIC version 16
(XEN) Processor #35 0:2 APIC version 16
(XEN) IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23
(XEN) IOAPIC[1]: apic_id 1, version 17, address 0xfdefd000, GSI 48-54
(XEN) IOAPIC[2]: apic_id 2, version 17, address 0xfdefc000, GSI 56-62
(XEN) IOAPIC[3]: apic_id 3, version 17, address 0xfeaff000, GSI 24-47
(XEN) Enabling APIC mode:  Phys.  Using 4 I/O APICs
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Detected 2493.385 MHz processor.
(XEN) AMD SVM: ASIDs enabled.
(XEN) HVM: SVM enabled
(XEN) HVM: Hardware Assisted Paging detected and enabled.
(XEN) CPU0: AMD Quad-Core AMD Opteron(tm) Processor 8360 SE stepping 03
(XEN) Mapping cpu 0 to node 255
(XEN) Booting processor 1/5 eip 90000
(XEN) Mapping cpu 1 to node 255
(XEN) AMD SVM: ASIDs enabled.
(XEN) CPU1: AMD Quad-Core AMD Opteron(tm) Processor 8360 SE stepping 03
(XEN) Booting processor 2/6 eip 90000
(XEN) Mapping cpu 2 to node 255
(XEN) AMD SVM: ASIDs enabled.
(XEN) CPU2: AMD Quad-Core AMD Opteron(tm) Processor 8360 SE stepping 03
(XEN) Booting processor 3/7 eip 90000
(XEN) Mapping cpu 3 to node 255
(XEN) AMD SVM: ASIDs enabled.
(XEN) CPU3: AMD Quad-Core AMD Opteron(tm) Processor 8360 SE stepping 03
(XEN) Booting processor 4/8 eip 90000
(XEN) Mapping cpu 4 to node 255
(XEN) AMD SVM: ASIDs enabled.
(XEN) CPU4: AMD Quad-Core AMD Opteron(tm) Processor 8360 SE stepping 03
(XEN) Booting processor 5/9 eip 90000
(XEN) Mapping cpu 5 to node 255
(XEN) AMD SVM: ASIDs enabled.
(XEN) CPU5: AMD Quad-Core AMD Opteron(tm) Processor 8360 SE stepping 03
(XEN) Booting processor 6/10 eip 90000
(XEN) Mapping cpu 6 to node 255
(XEN) AMD SVM: ASIDs enabled.
(XEN) CPU6: AMD Quad-Core AMD Opteron(tm) Processor 8360 SE stepping 03
(XEN) Booting processor 7/11 eip 90000
(XEN) Mapping cpu 7 to node 255
(XEN) AMD SVM: ASIDs enabled.
(XEN) CPU7: AMD Quad-Core AMD Opteron(tm) Processor 8360 SE stepping 03
(XEN) Booting processor 8/12 eip 90000
(XEN) Mapping cpu 8 to node 255
(XEN) AMD SVM: ASIDs enabled.
(XEN) CPU8: AMD Quad-Core AMD Opteron(tm) Processor 8360 SE stepping 03
(XEN) Booting processor 9/13 eip 90000
(XEN) Mapping cpu 9 to node 255
(XEN) AMD SVM: ASIDs enabled.
(XEN) CPU9: AMD Quad-Core AMD Opteron(tm) Processor 8360 SE stepping 03
(XEN) Booting processor 10/14 eip 90000
(XEN) Mapping cpu 10 to node 255
(XEN) AMD SVM: ASIDs enabled.
(XEN) CPU10: AMD Quad-Core AMD Opteron(tm) Processor 8360 SE stepping 03
(XEN) Booting processor 11/15 eip 90000
(XEN) Mapping cpu 11 to node 255
(XEN) AMD SVM: ASIDs enabled.
(XEN) CPU11: AMD Quad-Core AMD Opteron(tm) Processor 8360 SE stepping 03
(XEN) Booting processor 12/16 eip 90000
(XEN) Mapping cpu 12 to node 255
(XEN) AMD SVM: ASIDs enabled.
(XEN) CPU12: AMD Quad-Core AMD Opteron(tm) Processor 8360 SE stepping 03
(XEN) Booting processor 13/17 eip 90000
(XEN) Mapping cpu 13 to node 255
(XEN) AMD SVM: ASIDs enabled.
(XEN) CPU13: AMD Quad-Core AMD Opteron(tm) Processor 8360 SE stepping 03
(XEN) Booting processor 14/18 eip 90000
(XEN) Mapping cpu 14 to node 255
(XEN) AMD SVM: ASIDs enabled.
(XEN) CPU14: AMD Quad-Core AMD Opteron(tm) Processor 8360 SE stepping 03
(XEN) Booting processor 15/19 eip 90000
(XEN) Mapping cpu 15 to node 255
(XEN) AMD SVM: ASIDs enabled.
(XEN) CPU15: AMD Quad-Core AMD Opteron(tm) Processor 8360 SE stepping 03
(XEN) Booting processor 16/20 eip 90000
(XEN) Mapping cpu 16 to node 255
(XEN) AMD SVM: ASIDs enabled.
(XEN) CPU16: AMD Quad-Core AMD Opteron(tm) Processor 8360 SE stepping 03
(XEN) Booting processor 17/21 eip 90000
(XEN) Mapping cpu 17 to node 255
(XEN) AMD SVM: ASIDs enabled.
(XEN) CPU17: AMD Quad-Core AMD Opteron(tm) Processor 8360 SE stepping 03
(XEN) Booting processor 18/22 eip 90000
(XEN) Mapping cpu 18 to node 255
(XEN) AMD SVM: ASIDs enabled.
(XEN) CPU18: AMD Quad-Core AMD Opteron(tm) Processor 8360 SE stepping 03
(XEN) Booting processor 19/23 eip 90000
(XEN) Mapping cpu 19 to node 255
(XEN) AMD SVM: ASIDs enabled.
(XEN) CPU19: AMD Quad-Core AMD Opteron(tm) Processor 8360 SE stepping 03
(XEN) Booting processor 20/24 eip 90000
(XEN) Mapping cpu 20 to node 255
(XEN) AMD SVM: ASIDs enabled.
(XEN) CPU20: AMD Quad-Core AMD Opteron(tm) Processor 8360 SE stepping 03
(XEN) Booting processor 21/25 eip 90000
(XEN) Mapping cpu 21 to node 255
(XEN) AMD SVM: ASIDs enabled.
(XEN) CPU21: AMD Quad-Core AMD Opteron(tm) Processor 8360 SE stepping 03
(XEN) Booting processor 22/26 eip 90000
(XEN) Mapping cpu 22 to node 255
(XEN) AMD SVM: ASIDs enabled.
(XEN) CPU22: AMD Quad-Core AMD Opteron(tm) Processor 8360 SE stepping 03
(XEN) Booting processor 23/27 eip 90000
(XEN) Mapping cpu 23 to node 255
(XEN) AMD SVM: ASIDs enabled.
(XEN) CPU23: AMD Quad-Core AMD Opteron(tm) Processor 8360 SE stepping 03
(XEN) Booting processor 24/28 eip 90000
(XEN) Mapping cpu 24 to node 255
(XEN) AMD SVM: ASIDs enabled.
(XEN) CPU24: AMD Quad-Core AMD Opteron(tm) Processor 8360 SE stepping 03
(XEN) Booting processor 25/29 eip 90000
(XEN) Mapping cpu 25 to node 255
(XEN) AMD SVM: ASIDs enabled.
(XEN) CPU25: AMD Quad-Core AMD Opteron(tm) Processor 8360 SE stepping 03
(XEN) Booting processor 26/30 eip 90000
(XEN) Mapping cpu 26 to node 255
(XEN) AMD SVM: ASIDs enabled.
(XEN) CPU26: AMD Quad-Core AMD Opteron(tm) Processor 8360 SE stepping 03
(XEN) Booting processor 27/31 eip 90000
(XEN) Mapping cpu 27 to node 255
(XEN) AMD SVM: ASIDs enabled.
(XEN) CPU27: AMD Quad-Core AMD Opteron(tm) Processor 8360 SE stepping 03
(XEN) Booting processor 28/32 eip 90000
(XEN) Mapping cpu 28 to node 255
(XEN) AMD SVM: ASIDs enabled.
(XEN) CPU28: AMD Quad-Core AMD Opteron(tm) Processor 8360 SE stepping 03
(XEN) Booting processor 29/33 eip 90000
(XEN) Mapping cpu 29 to node 255
(XEN) AMD SVM: ASIDs enabled.
(XEN) CPU29: AMD Quad-Core AMD Opteron(tm) Processor 8360 SE stepping 03
(XEN) Booting processor 30/34 eip 90000
(XEN) Mapping cpu 30 to node 255
(XEN) AMD SVM: ASIDs enabled.
(XEN) CPU30: AMD Quad-Core AMD Opteron(tm) Processor 8360 SE stepping 03
(XEN) Booting processor 31/35 eip 90000
(XEN) Mapping cpu 31 to node 255
(XEN) AMD SVM: ASIDs enabled.
(XEN) CPU31: AMD Quad-Core AMD Opteron(tm) Processor 8360 SE stepping 03
(XEN) Total of 32 processors activated.
(XEN) ENABLING IO-APIC IRQs
(XEN)  -> Using new ACK method
(XEN) Platform timer overflows in 8589 jiffies.
(XEN) Platform timer is 25.000MHz HPET
(XEN) Brought up 32 CPUs
(XEN) *** LOADING DOMAIN 0 ***
(XEN) elf_parse_binary: phdr: paddr=0xffffffff80200000 memsz=0x2d5d20
(XEN) elf_parse_binary: phdr: paddr=0xffffffff804d5d80 memsz=0x1161d0
(XEN) elf_parse_binary: phdr: paddr=0xffffffff805ec000 memsz=0xc08
(XEN) elf_parse_binary: phdr: paddr=0xffffffff805ee000 memsz=0x1134e4
(XEN) elf_parse_binary: memory: 0xffffffff80200000 -> 0xffffffff807014e4
(XEN) elf_xen_parse_note: GUEST_OS = "linux"
(XEN) elf_xen_parse_note: GUEST_VERSION = "2.6"
(XEN) elf_xen_parse_note: XEN_VERSION = "xen-3.0"
(XEN) elf_xen_parse_note: VIRT_BASE = 0xffffffff80000000
(XEN) elf_xen_parse_note: PADDR_OFFSET = 0xffffffff80000000
(XEN) elf_xen_parse_note: ENTRY = 0xffffffff80200000
(XEN) elf_xen_parse_note: HYPERCALL_PAGE = 0xffffffff80206000
(XEN) elf_xen_parse_note: FEATURES = "writable_page_tables|writable_descriptor_tables|auto_translated_physmap|pae_pgdir_above_4gb|supervisor_mode_kernel"
(XEN) elf_xen_parse_note: LOADER = "generic"
(XEN) elf_xen_addr_calc_check: addresses:
(XEN)     virt_base        = 0xffffffff80000000
(XEN)     elf_paddr_offset = 0xffffffff80000000
(XEN)     virt_offset      = 0x0
(XEN)     virt_kstart      = 0xffffffff80200000
(XEN)     virt_kend        = 0xffffffff807014e4
(XEN)     virt_entry       = 0xffffffff80200000
(XEN)  Xen  kernel: 64-bit, lsb, compat32
(XEN)  Dom0 kernel: 64-bit, lsb, paddr 0xffffffff80200000 -> 0xffffffff807014e4
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN)  Dom0 alloc.:   0000003720000000->0000003740000000 (57675185 pages to be allocated)
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN)  Loaded kernel: ffffffff80200000->ffffffff807014e4
(XEN)  Init. ramdisk: ffffffff80702000->ffffffff80ea1800
(XEN)  Phys-Mach map: ffffffff80ea2000->ffffffff9c7a8d88
(XEN)  Start info:    ffffffff9c7a9000->ffffffff9c7a949c
(XEN)  Page tables:   ffffffff9c7aa000->ffffffff9c893000
(XEN)  Boot stack:    ffffffff9c893000->ffffffff9c894000
(XEN)  TOTAL:         ffffffff80000000->ffffffff9cc00000
(XEN)  ENTRY ADDRESS: ffffffff80200000
(XEN) Dom0 has maximum 32 VCPUs
(XEN) elf_load_binary: phdr 0 at 0xffffffff80200000 -> 0xffffffff804d5d20
(XEN) elf_load_binary: phdr 1 at 0xffffffff804d5d80 -> 0xffffffff805ebf50
(XEN) elf_load_binary: phdr 2 at 0xffffffff805ec000 -> 0xffffffff805ecc08
(XEN) elf_load_binary: phdr 3 at 0xffffffff805ee000 -> 0xffffffff80628388
(XEN) Initrd len 0x79f800, start at 0xffffffff80702000
(XEN) Scrubbing Free RAM: .done.
(XEN) Xen trace buffers: disabled
(XEN) Std. Loglevel: Errors and warnings
(XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to Xen).
(XEN) Freed 100kB init memory.
Bootdata ok (command line is ro root=LABEL=/1 earlyprintk=ttyS0,115200 console=ttyS0,115200)
Linux version 2.6.18-92.el5xen (brewbuilder.redhat.com) (gcc version 4.1.2 20071124 (Red Hat 4.1.2-41)) #1 SMP Tue Apr 29 13:31:30 EDT 2008
BIOS-provided physical RAM map:
 Xen: 0000000000000000 - 00000037215b1000 (usable)
kernel direct mapping tables up to 37215b1000 @ 1c96f000-53d41000
DMI present.
ACPI: LAPIC (acpi_id[0x01] lapic_id[0x04] enabled)
ACPI: LAPIC (acpi_id[0x02] lapic_id[0x05] enabled)
ACPI: LAPIC (acpi_id[0x03] lapic_id[0x06] enabled)
ACPI: LAPIC (acpi_id[0x04] lapic_id[0x07] enabled)
ACPI: LAPIC (acpi_id[0x05] lapic_id[0x08] enabled)
ACPI: LAPIC (acpi_id[0x06] lapic_id[0x09] enabled)
ACPI: LAPIC (acpi_id[0x07] lapic_id[0x0a] enabled)
ACPI: LAPIC (acpi_id[0x08] lapic_id[0x0b] enabled)
ACPI: LAPIC (acpi_id[0x09] lapic_id[0x0c] enabled)
ACPI: LAPIC (acpi_id[0x0a] lapic_id[0x0d] enabled)
ACPI: LAPIC (acpi_id[0x0b] lapic_id[0x0e] enabled)
ACPI: LAPIC (acpi_id[0x0c] lapic_id[0x0f] enabled)
ACPI: LAPIC (acpi_id[0x0d] lapic_id[0x10] enabled)
ACPI: LAPIC (acpi_id[0x0e] lapic_id[0x11] enabled)
ACPI: LAPIC (acpi_id[0x0f] lapic_id[0x12] enabled)
ACPI: LAPIC (acpi_id[0x10] lapic_id[0x13] enabled)
ACPI: LAPIC (acpi_id[0x11] lapic_id[0x14] enabled)
ACPI: LAPIC (acpi_id[0x12] lapic_id[0x15] enabled)
ACPI: LAPIC (acpi_id[0x13] lapic_id[0x16] enabled)
ACPI: LAPIC (acpi_id[0x14] lapic_id[0x17] enabled)
ACPI: LAPIC (acpi_id[0x15] lapic_id[0x18] enabled)
ACPI: LAPIC (acpi_id[0x16] lapic_id[0x19] enabled)
ACPI: LAPIC (acpi_id[0x17] lapic_id[0x1a] enabled)
ACPI: LAPIC (acpi_id[0x18] lapic_id[0x1b] enabled)
ACPI: LAPIC (acpi_id[0x19] lapic_id[0x1c] enabled)
ACPI: LAPIC (acpi_id[0x1a] lapic_id[0x1d] enabled)
ACPI: LAPIC (acpi_id[0x1b] lapic_id[0x1e] enabled)
ACPI: LAPIC (acpi_id[0x1c] lapic_id[0x1f] enabled)
ACPI: LAPIC (acpi_id[0x1d] lapic_id[0x20] enabled)
ACPI: LAPIC (acpi_id[0x1e] lapic_id[0x21] enabled)
ACPI: LAPIC (acpi_id[0x1f] lapic_id[0x22] enabled)
ACPI: LAPIC (acpi_id[0x20] lapic_id[0x23] enabled)
ACPI: IOAPIC (id[0x00] address[0xfec00000] gsi_base[0])
IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23
ACPI: IOAPIC (id[0x01] address[0xfdefd000] gsi_base[48])
IOAPIC[1]: apic_id 1, version 17, address 0xfdefd000, GSI 48-54
ACPI: IOAPIC (id[0x02] address[0xfdefc000] gsi_base[56])
IOAPIC[2]: apic_id 2, version 17, address 0xfdefc000, GSI 56-62
ACPI: IOAPIC (id[0x03] address[0xfeaff000] gsi_base[24])
IOAPIC[3]: apic_id 3, version 17, address 0xfeaff000, GSI 24-47
ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
Setting APIC routing to xen
Using ACPI (MADT) for SMP configuration information
Allocating PCI resources starting at f1000000 (gap: f0000000:ec00000)
Built 1 zonelists.  Total pages: 57808305
Kernel command line: ro root=LABEL=/1 earlyprintk=ttyS0,115200 console=ttyS0,115200
Initializing CPU#0
PID hash table entries: 4096 (order: 12, 32768 bytes)
Xen reported: 2493.374 MHz processor.
disabling early console
Bootdata ok (command line is ro root=LABEL=/1 earlyprintk=ttyS0,115200 console=ttyS0,115200)
Linux version 2.6.18-92.el5xen (brewbuilder.redhat.com) (gcc version 4.1.2 20071124 (Red Hat 4.1.2-41)) #1 SMP Tue Apr 29 13:31:30 EDT 2008
BIOS-provided physical RAM map:
 Xen: 0000000000000000 - 00000037215b1000 (usable)
DMI present.
ACPI: LAPIC (acpi_id[0x01] lapic_id[0x04] enabled)
ACPI: LAPIC (acpi_id[0x02] lapic_id[0x05] enabled)
ACPI: LAPIC (acpi_id[0x03] lapic_id[0x06] enabled)
ACPI: LAPIC (acpi_id[0x04] lapic_id[0x07] enabled)
ACPI: LAPIC (acpi_id[0x05] lapic_id[0x08] enabled)
ACPI: LAPIC (acpi_id[0x06] lapic_id[0x09] enabled)
ACPI: LAPIC (acpi_id[0x07] lapic_id[0x0a] enabled)
ACPI: LAPIC (acpi_id[0x08] lapic_id[0x0b] enabled)
ACPI: LAPIC (acpi_id[0x09] lapic_id[0x0c] enabled)
ACPI: LAPIC (acpi_id[0x0a] lapic_id[0x0d] enabled)
ACPI: LAPIC (acpi_id[0x0b] lapic_id[0x0e] enabled)
ACPI: LAPIC (acpi_id[0x0c] lapic_id[0x0f] enabled)
ACPI: LAPIC (acpi_id[0x0d] lapic_id[0x10] enabled)
ACPI: LAPIC (acpi_id[0x0e] lapic_id[0x11] enabled)
ACPI: LAPIC (acpi_id[0x0f] lapic_id[0x12] enabled)
ACPI: LAPIC (acpi_id[0x10] lapic_id[0x13] enabled)
ACPI: LAPIC (acpi_id[0x11] lapic_id[0x14] enabled)
ACPI: LAPIC (acpi_id[0x12] lapic_id[0x15] enabled)
ACPI: LAPIC (acpi_id[0x13] lapic_id[0x16] enabled)
ACPI: LAPIC (acpi_id[0x14] lapic_id[0x17] enabled)
ACPI: LAPIC (acpi_id[0x15] lapic_id[0x18] enabled)
ACPI: LAPIC (acpi_id[0x16] lapic_id[0x19] enabled)
ACPI: LAPIC (acpi_id[0x17] lapic_id[0x1a] enabled)
ACPI: LAPIC (acpi_id[0x18] lapic_id[0x1b] enabled)
ACPI: LAPIC (acpi_id[0x19] lapic_id[0x1c] enabled)
ACPI: LAPIC (acpi_id[0x1a] lapic_id[0x1d] enabled)
ACPI: LAPIC (acpi_id[0x1b] lapic_id[0x1e] enabled)
ACPI: LAPIC (acpi_id[0x1c] lapic_id[0x1f] enabled)
ACPI: LAPIC (acpi_id[0x1d] lapic_id[0x20] enabled)
ACPI: LAPIC (acpi_id[0x1e] lapic_id[0x21] enabled)
ACPI: LAPIC (acpi_id[0x1f] lapic_id[0x22] enabled)
ACPI: LAPIC (acpi_id[0x20] lapic_id[0x23] enabled)
ACPI: IOAPIC (id[0x00] address[0xfec00000] gsi_base[0])
IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23
ACPI: IOAPIC (id[0x01] address[0xfdefd000] gsi_base[48])
IOAPIC[1]: apic_id 1, version 17, address 0xfdefd000, GSI 48-54
ACPI: IOAPIC (id[0x02] address[0xfdefc000] gsi_base[56])
IOAPIC[2]: apic_id 2, version 17, address 0xfdefc000, GSI 56-62
ACPI: IOAPIC (id[0x03] address[0xfeaff000] gsi_base[24])
IOAPIC[3]: apic_id 3, version 17, address 0xfeaff000, GSI 24-47
ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
Setting APIC routing to xen
Using ACPI (MADT) for SMP configuration information
Allocating PCI resources starting at f1000000 (gap: f0000000:ec00000)
Built 1 zonelists.  Total pages: 57808305
Kernel command line: ro root=LABEL=/1 earlyprintk=ttyS0,115200 console=ttyS0,115200
Initializing CPU#0
PID hash table entries: 4096 (order: 12, 32768 bytes)
Xen reported: 2493.374 MHz processor.
disabling early console
Console: colour VGA+ 80x25
Dentry cache hash table entries: 33554432 (order: 16, 268435456 bytes)
Inode-cache hash table entries: 16777216 (order: 15, 134217728 bytes)
low bootmem alloc of 67108864 bytes failed!
Kernel panic - not syncing: Out of low memory
 (XEN) Domain 0 crashed: rebooting machine in 5 seconds.


but
mem=200g works.

Comment 1 Russell Doty 2008-08-13 20:19:26 UTC
I have made a test RPM available at http://people.redhat.com/riel/.bz453467/

This RPM contains one upstream patch to automatically size the xenheap right
(our code base does not have that upstream patch yet) and one to automatically
limit the size of dom0 to 32GB - a size big enough that 99% of the users will
never notice, yet it should be small enough for the really large systems.

Please let me know how the test RPM works for you.

Comment 2 Sandy Garza 2008-08-26 19:54:24 UTC
Created attachment 315042 [details]
meminfo after applying patch

HP found that the kernel boots but the OS only sees 32Gb of memory. See attached meminfo.

Comment 3 Sandy Garza 2008-08-26 19:55:04 UTC
We ran this on a system with 256Gb of memory.

(In reply to comment #2)
> Created an attachment (id=315042) [details]
> meminfo after applying patch
> HP found that the kernel boots but the OS only sees 32Gb of memory. See
> attached meminfo.

Comment 4 Chris Lalancette 2008-08-27 07:34:41 UTC
Yes, this was done intentionally.  Having all 256Gb assigned to the dom0 is generally wasteful in a Xen environment; you'll be eating up a lot of the memory available to dom0 with structures tracking all 256G, when in reality you probably want to give almost all of that memory to your guests.  For that reason, we are now restricting the dom0 to 32G.

That being said, the machine *overall* should still see 256G of memory; you can confirm this by running "xm info" and looking at the total memory column.  If you really feel that you need 256G available to the dom0, you can override the default by passing "dom0_mem=256G" on the hypervisor command-line.

Chris Lalancette

Comment 5 Micah Parrish 2008-08-28 17:44:34 UTC
I tested this patch on the Proliant DL785 with 256GB and it worked.  I plan to test it on a DL785 with 512GB.

Comment 6 Rick Hester 2008-08-28 19:30:02 UTC
I've been informed that this is an architecture independent problem and that testing with other archs (like the IA64 systems I have) is desired.

Would it be possible to build an IA64 kernel with the patch as well?  Currently, there appears to be just an x86_64 kernel.

What is the desired configuration that this should be tested on?   We have
a systems with memory configs up to 2TB, but some of those are not 
immediately available.  So, it would help in the resource scheduling to
know what the requirement is. Is the 
test requested to just install and boot with defaults?  Or should I set the
dom0 memory to be the same as the physical memory (less maybe some amount
for the hypervisor)?

thanks

Rick Hester

Comment 7 Chris Lalancette 2008-08-29 07:14:58 UTC
Well, the patch here won't have any effect on ia64; it's only patching x86.  I think we already have similar constraints in place for ia64 (something like 4GB for dom0), but I really don't know any of the details of that arch.

If you have the time, it's always good to get testing on ia64 on the largest box you can; but nothing should be different about the linked kernel here with respect to it.

Chris Lalancette

Comment 8 Bill Burns 2008-09-03 13:32:19 UTC
Marking modified as the private bug that this one is a duplicate of has been integrated into a build:
in kernel-2.6.18-107.el5
You can download this test kernel from http://people.redhat.com/dzickus/el5
Please test on your platform.

Comment 9 Rick Hester 2008-09-19 19:37:11 UTC
We have tried this kernel out on an IA64 system.

We have installed this the 2.6.18-107 kernel on an IA64 system
with 64 cpu threads and 512GB of memory.  That kernel successfully
booted.  But then so did the kernel in RHEL 5.2 GA. 

We then attempted to install and boot on a 128 cpu thread, 1TB IA64
system.  Both the 2.6.18-107 and the RHEL 5.2 GA kernels failed to
boot. Failure was a hand during boot.  This was already known and
reported behavior for the RHEL 5.2 GA kernel.

So these changes appear to have had no impact on the IA64 kernel (as
was expected from previous Red Hat comments).

Comment 10 Micah Parrish 2008-09-22 20:07:11 UTC
I tested the RHEL5.3 Alpha version of this on a 256GB x86_64 system.  Passing dom0_mem=256G on the kernel command line resulted in the same "Xen is relinquishing VGA console" message.  dom0_mem=128G booted fine, as did the same kernel without any extra options.  Works for me.

Comment 11 Bill Burns 2008-10-23 13:40:08 UTC
Keeping in sync with private bz.

Comment 12 Bill Burns 2009-01-28 13:30:12 UTC
The fix for this was shipped with RHEL 5.3.


Note You need to log in before you can comment on or make changes to this bug.