Bug 669307 - xen can't boot with large memory IA64 system (HP superdome sx2000 )
Summary: xen can't boot with large memory IA64 system (HP superdome sx2000 )
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: kernel-xen
Version: 5.6
Hardware: ia64
OS: Linux
low
medium
Target Milestone: rc
: ---
Assignee: Xen Maintainance List
QA Contact: Virtualization Bugs
URL:
Whiteboard:
Depends On:
Blocks: 514490
TreeView+ depends on / blocked
 
Reported: 2011-01-13 09:32 UTC by Jiayin Shao
Modified: 2011-08-01 05:50 UTC (History)
11 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2011-01-27 08:48:00 UTC
Target Upstream Version:


Attachments (Terms of Use)
sx2000 topo (13.51 KB, image/png)
2011-01-26 07:50 UTC, Jiayin Shao
no flags Details
whole sx2000 topo (186.49 KB, image/png)
2011-01-27 05:34 UTC, Jiayin Shao
no flags Details
sx2000 topo (178.48 KB, image/png)
2011-07-26 12:54 UTC, Lihua Xie
no flags Details
sx2000 topo (187.77 KB, image/png)
2011-08-01 05:50 UTC, Lihua Xie
no flags Details

Description Jiayin Shao 2011-01-13 09:32:55 UTC
Description of problem:
xen for rhel5.6rc2 can boot with append="mem=1024G -- root=LABEL=/123", not boot successfully without mem=1024G as physical memory is 1024G on HP superdome sx2000 system. Xen will hang as finishing checking hypervisor's memory amount. I just wonder why, and if memory must be assigned in elilo.conf on a IA64 system with a large memory.

And is there any memory limitation for xen hypervisor on rhel5.6? related document is not found.  

 
Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.
  
Actual results:


Expected results:


Additional info:

Comment 1 Paolo Bonzini 2011-01-13 10:01:14 UTC
From the release notes at bug 521865:

The "xenheap_megabytes" hypervisor option is now supported on ia64 systems as well.  The option can be used to run the Xen hypervisor on ia64 systems with more than 64GB of RAM.  If the installed memory exceeds 64GB, it is suggested to set the option to a value equal to the memory size in gigabytes.  For example, on a system with 128GB of memory the elilo.conf file should include the directive 

      append="xenheap_megabytes=128 --"

Please try again with this option.  Also, if it boots but cannot create guests, please attach the output of lstopo.

Comment 2 Jiayin Shao 2011-01-15 07:35:48 UTC
Can boot on a system with 1.6T memory in elilo.conf include 
   append="mem=1650G -- root=LABEL=/123"
And, if include 
   append="xenheap_megabytes=1650 -- root=LABEL=/123"
Below output will be seen.




ELILO boot: Uncompressing Linux... done
Loading file initrd-2.6.18-238.el5xen.img...done
Loading file vmlinuz-2.6.18-238.el5xen...done
Uncompressing... done
 __  __            _____  _   ____    ____  _____  ___       _ ____
 \ \/ /___ _ __   |___ / / | |___ \  |___ \|___ / ( _ )  ___| | ___|
  \  // _ \ '_ \    |_ \ | |   __) |__ __) | |_ \ / _ \ / _ \ |___ \
  /  \  __/ | | |  ___) || |_ / __/|__/ __/ ___) | (_) |  __/ |___) |
 /_/\_\___|_| |_| |____(_)_(_)_____| |_____|____/ \___(_)___|_|____/

 http://www.cl.cam.ac.uk/netos/xen
 University of Cambridge Computer Laboratory

 Xen version 3.1.2-238.el5 (mockbuild) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-50)) Sun Dec 19 14:20:14 EST 2010
 Latest ChangeSet: unavailable

(XEN) Xen command line: BOOT_IMAGE=scsi4:\EFI\redhat\xen.gz-2.6.18-238.el5  xenheap_megabytes=1650
(XEN) xen image pstart: 0x4000000, xenheap pend: 0x6b200000
(XEN) Xen patching physical address access by offset: 0x0
(XEN) Xen BUG at xensetup.c:329
(XEN) FIXME: implement ia64 dump_execution_state()
 MCA event occurred ....

Comment 3 Laszlo Ersek 2011-01-17 12:11:59 UTC
(CC'ing Matt and Alex.)

Jiayin, please attach the output of the "lstopo" command, which belongs to the hwloc package. Example: "lstopo superdome.png". Please see bug 661989 attachment 470649 [details] for an example.

Please also see bug 661989 comment 28. (This is probably what Paolo had in mind in comment 1.) That's another ia64 machine (with 122G RAM) that xen won't run on, due to the machine's NUMA configuration. The PNG file produced by "lstopo" might allow us to close this bug as a duplicate. Thanks.

Comment 4 Jiayin Shao 2011-01-18 07:16:34 UTC
(In reply to comment #3)
> (CC'ing Matt and Alex.)
> 
> Jiayin, please attach the output of the "lstopo" command, which belongs to the
> hwloc package. Example: "lstopo superdome.png". Please see bug 661989
> attachment 470649 [details] for an example.

hi, I'm not authorized to access this attachment. 
should I download this package?

Comment 5 Laszlo Ersek 2011-01-18 09:40:10 UTC
(In reply to comment #4)
> (In reply to comment #3)
> > 
> > Jiayin, please attach the output of the "lstopo" command, which belongs to 
> > the hwloc package. Example: "lstopo superdome.png".

> should I download this package?

Sorry, I didn't notice that hwloc is in epel, and epel doesn't support ia64. Please install it from source then [0], but make sure that you have the libpng-devel package installed first. Thanks!

[0] http://www.open-mpi.org/software/hwloc/v1.1/

Comment 6 Jiayin Shao 2011-01-26 07:50:12 UTC
Created attachment 475345 [details]
sx2000 topo

sx2000 topo

Comment 7 Jiayin Shao 2011-01-26 07:53:41 UTC
cairo-devel package also has to be installed, then "lstopo" could identify png format.

Jiayin

Comment 8 Jiayin Shao 2011-01-26 07:56:23 UTC
And additional message.

"append="mem=1650G -- root=LABEL=/123" 

[root@worm0 hwloc-1.1.1rc3]# virsh nodeinfo
CPU model:           ia64
CPU(s):              64
CPU frequency:       1598 MHz
CPU socket(s):       32
Core(s) per socket:  2
Thread(s) per core:  1
NUMA cell(s):        1
Memory size:         1730149376 kB

[root@worm0 hwloc-1.1.1rc3]# virsh dominfo 0
Id:             0
Name:           Domain-0
UUID:           00000000-0000-0000-0000-000000000000
OS Type:        linux
State:          running
CPU(s):         4
CPU time:       3747.7s
Max memory:     4194304 kB
Used memory:    4092320 kB
Persistent:     no
Autostart:      disable

Comment 9 Laszlo Ersek 2011-01-26 10:30:15 UTC
Hello Jiayin,

could you please boot a bare-metal kernel and run "lstopo" on that? Thanks! (Sorry for not mentioning bare-metal before!)

(In reply to comment #2)
> 
>  Xen version 3.1.2-238.el5 (mockbuild) (gcc version 4.1.2 20080704
> (Red Hat 4.1.2-50)) Sun Dec 19 14:20:14 EST 2010
>  Latest ChangeSet: unavailable
> 
> (XEN) Xen command line: BOOT_IMAGE=scsi4:\EFI\redhat\xen.gz-2.6.18-238.el5 
> xenheap_megabytes=1650
> (XEN) xen image pstart: 0x4000000, xenheap pend: 0x6b200000
> (XEN) Xen patching physical address access by offset: 0x0
> (XEN) Xen BUG at xensetup.c:329

This is the affected region from "xen/arch/ia64/xen/xensetup.c", function start_kernel():

320      /*
321       * Scan through the memory descriptors after the kernel
322       * image to make sure we have enough room for the xenheap
323       * area, pushing out whatever may already be there.
324       */
325      while (relo_start + relo_size >= md_end) {
326          md = efi_get_md(md_end);
327  
328          BUG_ON(!md);
329          BUG_ON(!is_xenheap_usable_memory(md));

222  static int
223  is_xenheap_usable_memory(efi_memory_desc_t *md)
224  {
225      if (!(md->attribute & EFI_MEMORY_WB))
226          return 0;
227  
228      switch (md->type) {
229          case EFI_LOADER_CODE:
230          case EFI_LOADER_DATA:
231          case EFI_BOOT_SERVICES_CODE:
232          case EFI_BOOT_SERVICES_DATA:
233          case EFI_CONVENTIONAL_MEMORY:
234              return 1;
235      }
236      return 0;
237  }

"EFI_MEMORY_WB"	means "write-back".

So the loop stumbles on a memory descriptor "after the kernel image" that either has an unknown/unhandled type, or doesn't have the write-back attribute.

I think the memory layout should be reconfigured somehow -- running lstopo on a bare-metal kernel might suggest something.

Comment 10 Jiayin Shao 2011-01-27 05:34:01 UTC
Created attachment 475525 [details]
whole sx2000 topo

Comment 11 Paolo Bonzini 2011-01-27 08:48:00 UTC
This is not supported by Xen, see the similarity with attachment 470649 [details].

Comment 12 Jiayin Shao 2011-01-28 07:44:07 UTC
(In reply to comment #11)
> This is not supported by Xen, see the similarity with attachment 470649 [details].

I have no authority to access this attachment.

BTW, I would like to know how much max memory supported for xen Hypervisor? I did not find the memory limitation in rhel5.6 virtualization doc.

Comment 13 Paolo Bonzini 2011-01-28 09:20:08 UTC
The problem is not the amount of memory, it is the NUMA configuration that is used by the machine. With such a NUMA configuration you cannot run xen, because no node can see the bulk of the memory.  Xen will only be able to use 256 MB or less.

Comment 14 Lihua Xie 2011-07-22 04:34:10 UTC
RHEL5.7s5 also encounter this problem on HP superdome sx2000 with 1.6T memory.

Comment 15 Lihua Xie 2011-07-22 06:05:35 UTC
RHEL5.7s5 also encounter this problem on HP superdome sx2000 with 1.6T memory.

Comment 16 Paolo Bonzini 2011-07-25 07:04:01 UTC
The bug was not fixed.  The NUMA configuration of this machine is not supported by Xen, see comment 13.

Comment 17 Lihua Xie 2011-07-26 10:02:25 UTC
I encounter a new issue about xen kernel boot with large memory.
Whether config mem=1700G or not, the machine also cant boot into the OS, will occur MCA.
additionall,where the user can find the official announcement how to deal with this situation,the os release notes or some documents from xen.

(XEN) Init boot pages: 0x70020000000 -> 0x71dfc000000.
(XEN) Init boot pages: 0xe9ffbe7c000 -> 0xe9ffbfdc000.
(XEN) System RAM: 1694878MB (1735555584kB)
(XEN) size of virtual frame_table: 4239312kB
(XEN) virtual machine to physical table: f3fffffe2c008048 size: 848256kB
(XEN) max_page: 0x3a7feff7
(XEN) Xen heap: 17592186044361MB (18014398509425776kB)
(XEN) Cannot handle page request order 1!
(XEN) Xen BUG at page_alloc.c:343
(XEN) FIXME: implement ia64 dump_execution_state()
 MCA event occurred ....

 Unable to handoff: OS_MCA handler not registered.

sequencer: The MCA was NOT corrected. A cold boot is required.
sequencer: Resetting...

Comment 18 Paolo Bonzini 2011-07-26 10:34:20 UTC
You can boot the system under non-Xen in order to run lstopo.

Comment 19 Lihua Xie 2011-07-26 12:54:43 UTC
Created attachment 515273 [details]
sx2000 topo

sx2000 topo

Comment 20 Paolo Bonzini 2011-07-26 14:02:58 UTC
Alex,

https://bugzilla.redhat.com/attachment.cgi?id=515273 is a different NUMA topology than the one that was reported in the past and which Xen cannot work with (https://bugzilla.redhat.com/attachment.cgi?id=475525).

Does it have the same problem though?

Comment 21 Alex Williamson 2011-07-26 14:58:02 UTC
This looks like a completely bogus config, 13 cells with memory *only* installed on cell0?!  I can't imagine a customer actually making use of such a configuration.  I believe we only support memory installed equally among cells and the system configured for no cell local memory.

Comment 22 Lihua Xie 2011-08-01 05:47:07 UTC
I know why only one cell has memory installed ,displaying in the sanddune.png.When booting the os,i use the elilo.conf config with mem=64G option.

Have captured the topo information with both format,textual and png file.


Machine (1655GB)
  NUMANode L#0 (P#1 127GB)
    Socket L#0
      L3 L#0 (12MB) + L2 L#0 (256KB) + L1 L#0 (16KB) + Core L#0 + PU L#0 (P#0)
      L3 L#1 (12MB) + L2 L#1 (256KB) + L1 L#1 (16KB) + Core L#1 + PU L#1 (P#1)
    Socket L#1
      L3 L#2 (12MB) + L2 L#2 (256KB) + L1 L#2 (16KB) + Core L#2 + PU L#2 (P#2)
      L3 L#3 (12MB) + L2 L#3 (256KB) + L1 L#3 (16KB) + Core L#3 + PU L#3 (P#3)
    Socket L#2
      L3 L#4 (12MB) + L2 L#4 (256KB) + L1 L#4 (16KB) + Core L#4 + PU L#4 (P#4)
      L3 L#5 (12MB) + L2 L#5 (256KB) + L1 L#5 (16KB) + Core L#5 + PU L#5 (P#5)
    Socket L#3
      L3 L#6 (12MB) + L2 L#6 (256KB) + L1 L#6 (16KB) + Core L#6 + PU L#6 (P#6)
      L3 L#7 (12MB) + L2 L#7 (256KB) + L1 L#7 (16KB) + Core L#7 + PU L#7 (P#7)
  NUMANode L#1 (P#0 119GB)
    Socket L#4
      L3 L#8 (12MB) + L2 L#8 (256KB) + L1 L#8 (16KB) + Core L#8 + PU L#8 (P#8)
      L3 L#9 (12MB) + L2 L#9 (256KB) + L1 L#9 (16KB) + Core L#9 + PU L#9 (P#9)
    Socket L#5
      L3 L#10 (12MB) + L2 L#10 (256KB) + L1 L#10 (16KB) + Core L#10 + PU L#10 (P#10)
      L3 L#11 (12MB) + L2 L#11 (256KB) + L1 L#11 (16KB) + Core L#11 + PU L#11 (P#11)
    Socket L#6
      L3 L#12 (12MB) + L2 L#12 (256KB) + L1 L#12 (16KB) + Core L#12 + PU L#12 (P#12)
      L3 L#13 (12MB) + L2 L#13 (256KB) + L1 L#13 (16KB) + Core L#13 + PU L#13 (P#13)
    Socket L#7
      L3 L#14 (12MB) + L2 L#14 (256KB) + L1 L#14 (16KB) + Core L#14 + PU L#14 (P#14)
      L3 L#15 (12MB) + L2 L#15 (256KB) + L1 L#15 (16KB) + Core L#15 + PU L#15 (P#15)
  NUMANode L#2 (P#2 128GB)
    Socket L#8
      L3 L#16 (12MB) + L2 L#16 (256KB) + L1 L#16 (16KB) + Core L#16 + PU L#16 (P#16)
      L3 L#17 (12MB) + L2 L#17 (256KB) + L1 L#17 (16KB) + Core L#17 + PU L#17 (P#17)
    Socket L#9
      L3 L#18 (12MB) + L2 L#18 (256KB) + L1 L#18 (16KB) + Core L#18 + PU L#18 (P#18)
      L3 L#19 (12MB) + L2 L#19 (256KB) + L1 L#19 (16KB) + Core L#19 + PU L#19 (P#19)
    Socket L#10
      L3 L#20 (12MB) + L2 L#20 (256KB) + L1 L#20 (16KB) + Core L#20 + PU L#20 (P#20)
      L3 L#21 (12MB) + L2 L#21 (256KB) + L1 L#21 (16KB) + Core L#21 + PU L#21 (P#21)
    Socket L#11
      L3 L#22 (12MB) + L2 L#22 (256KB) + L1 L#22 (16KB) + Core L#22 + PU L#22 (P#22)
      L3 L#23 (12MB) + L2 L#23 (256KB) + L1 L#23 (16KB) + Core L#23 + PU L#23 (P#23)
  NUMANode L#3 (P#3 128GB)
    Socket L#12
      L3 L#24 (12MB) + L2 L#24 (256KB) + L1 L#24 (16KB) + Core L#24 + PU L#24 (P#24)
      L3 L#25 (12MB) + L2 L#25 (256KB) + L1 L#25 (16KB) + Core L#25 + PU L#25 (P#25)
    Socket L#13
      L3 L#26 (12MB) + L2 L#26 (256KB) + L1 L#26 (16KB) + Core L#26 + PU L#26 (P#26)
      L3 L#27 (12MB) + L2 L#27 (256KB) + L1 L#27 (16KB) + Core L#27 + PU L#27 (P#27)
    Socket L#14
      L3 L#28 (12MB) + L2 L#28 (256KB) + L1 L#28 (16KB) + Core L#28 + PU L#28 (P#28)
      L3 L#29 (12MB) + L2 L#29 (256KB) + L1 L#29 (16KB) + Core L#29 + PU L#29 (P#29)
    Socket L#15
      L3 L#30 (12MB) + L2 L#30 (256KB) + L1 L#30 (16KB) + Core L#30 + PU L#30 (P#30)
      L3 L#31 (12MB) + L2 L#31 (256KB) + L1 L#31 (16KB) + Core L#31 + PU L#31 (P#31)
  NUMANode L#4 (P#4 128GB)
    Socket L#16
      L3 L#32 (12MB) + L2 L#32 (256KB) + L1 L#32 (16KB) + Core L#32 + PU L#32 (P#32)
      L3 L#33 (12MB) + L2 L#33 (256KB) + L1 L#33 (16KB) + Core L#33 + PU L#33 (P#33)
    Socket L#17
      L3 L#34 (12MB) + L2 L#34 (256KB) + L1 L#34 (16KB) + Core L#34 + PU L#34 (P#34)
      L3 L#35 (12MB) + L2 L#35 (256KB) + L1 L#35 (16KB) + Core L#35 + PU L#35 (P#35)
    Socket L#18
      L3 L#36 (12MB) + L2 L#36 (256KB) + L1 L#36 (16KB) + Core L#36 + PU L#36 (P#36)
      L3 L#37 (12MB) + L2 L#37 (256KB) + L1 L#37 (16KB) + Core L#37 + PU L#37 (P#37)
    Socket L#19
      L3 L#38 (12MB) + L2 L#38 (256KB) + L1 L#38 (16KB) + Core L#38 + PU L#38 (P#38)
      L3 L#39 (12MB) + L2 L#39 (256KB) + L1 L#39 (16KB) + Core L#39 + PU L#39 (P#39)
  NUMANode L#5 (P#5 128GB)
    Socket L#20
      L3 L#40 (12MB) + L2 L#40 (256KB) + L1 L#40 (16KB) + Core L#40 + PU L#40 (P#40)
      L3 L#41 (12MB) + L2 L#41 (256KB) + L1 L#41 (16KB) + Core L#41 + PU L#41 (P#41)
    Socket L#21
      L3 L#42 (12MB) + L2 L#42 (256KB) + L1 L#42 (16KB) + Core L#42 + PU L#42 (P#42)
      L3 L#43 (12MB) + L2 L#43 (256KB) + L1 L#43 (16KB) + Core L#43 + PU L#43 (P#43)
    Socket L#22
      L3 L#44 (12MB) + L2 L#44 (256KB) + L1 L#44 (16KB) + Core L#44 + PU L#44 (P#44)
      L3 L#45 (12MB) + L2 L#45 (256KB) + L1 L#45 (16KB) + Core L#45 + PU L#45 (P#45)
    Socket L#23
      L3 L#46 (12MB) + L2 L#46 (256KB) + L1 L#46 (16KB) + Core L#46 + PU L#46 (P#46)
      L3 L#47 (12MB) + L2 L#47 (256KB) + L1 L#47 (16KB) + Core L#47 + PU L#47 (P#47)
  NUMANode L#6 (P#6 128GB)
    Socket L#24
      L3 L#48 (12MB) + L2 L#48 (256KB) + L1 L#48 (16KB) + Core L#48 + PU L#48 (P#48)
      L3 L#49 (12MB) + L2 L#49 (256KB) + L1 L#49 (16KB) + Core L#49 + PU L#49 (P#49)
    Socket L#25
      L3 L#50 (12MB) + L2 L#50 (256KB) + L1 L#50 (16KB) + Core L#50 + PU L#50 (P#50)
      L3 L#51 (12MB) + L2 L#51 (256KB) + L1 L#51 (16KB) + Core L#51 + PU L#51 (P#51)
    Socket L#26
      L3 L#52 (12MB) + L2 L#52 (256KB) + L1 L#52 (16KB) + Core L#52 + PU L#52 (P#52)
      L3 L#53 (12MB) + L2 L#53 (256KB) + L1 L#53 (16KB) + Core L#53 + PU L#53 (P#53)
    Socket L#27
      L3 L#54 (12MB) + L2 L#54 (256KB) + L1 L#54 (16KB) + Core L#54 + PU L#54 (P#54)
      L3 L#55 (12MB) + L2 L#55 (256KB) + L1 L#55 (16KB) + Core L#55 + PU L#55 (P#55)
  NUMANode L#7 (P#7 128GB)
    Socket L#28
      L3 L#56 (12MB) + L2 L#56 (256KB) + L1 L#56 (16KB) + Core L#56 + PU L#56 (P#56)
      L3 L#57 (12MB) + L2 L#57 (256KB) + L1 L#57 (16KB) + Core L#57 + PU L#57 (P#57)
    Socket L#29
      L3 L#58 (12MB) + L2 L#58 (256KB) + L1 L#58 (16KB) + Core L#58 + PU L#58 (P#58)
      L3 L#59 (12MB) + L2 L#59 (256KB) + L1 L#59 (16KB) + Core L#59 + PU L#59 (P#59)
    Socket L#30
      L3 L#60 (12MB) + L2 L#60 (256KB) + L1 L#60 (16KB) + Core L#60 + PU L#60 (P#60)
      L3 L#61 (12MB) + L2 L#61 (256KB) + L1 L#61 (16KB) + Core L#61 + PU L#61 (P#61)
    Socket L#31
      L3 L#62 (12MB) + L2 L#62 (256KB) + L1 L#62 (16KB) + Core L#62 + PU L#62 (P#62)
      L3 L#63 (12MB) + L2 L#63 (256KB) + L1 L#63 (16KB) + Core L#63 + PU L#63 (P#63)
  NUMANode L#8 (P#8 128GB)
    Socket L#32
      L3 L#64 (12MB) + L2 L#64 (256KB) + L1 L#64 (16KB) + Core L#64 + PU L#64 (P#64)
      L3 L#65 (12MB) + L2 L#65 (256KB) + L1 L#65 (16KB) + Core L#65 + PU L#65 (P#65)
    Socket L#33
      L3 L#66 (12MB) + L2 L#66 (256KB) + L1 L#66 (16KB) + Core L#66 + PU L#66 (P#66)
      L3 L#67 (12MB) + L2 L#67 (256KB) + L1 L#67 (16KB) + Core L#67 + PU L#67 (P#67)
    Socket L#34
      L3 L#68 (12MB) + L2 L#68 (256KB) + L1 L#68 (16KB) + Core L#68 + PU L#68 (P#68)
      L3 L#69 (12MB) + L2 L#69 (256KB) + L1 L#69 (16KB) + Core L#69 + PU L#69 (P#69)
    Socket L#35
      L3 L#70 (12MB) + L2 L#70 (256KB) + L1 L#70 (16KB) + Core L#70 + PU L#70 (P#70)
      L3 L#71 (12MB) + L2 L#71 (256KB) + L1 L#71 (16KB) + Core L#71 + PU L#71 (P#71)
  NUMANode L#9 (P#9 128GB)
    Socket L#36
      L3 L#72 (12MB) + L2 L#72 (256KB) + L1 L#72 (16KB) + Core L#72 + PU L#72 (P#72)
      L3 L#73 (12MB) + L2 L#73 (256KB) + L1 L#73 (16KB) + Core L#73 + PU L#73 (P#73)
    Socket L#37
      L3 L#74 (12MB) + L2 L#74 (256KB) + L1 L#74 (16KB) + Core L#74 + PU L#74 (P#74)
      L3 L#75 (12MB) + L2 L#75 (256KB) + L1 L#75 (16KB) + Core L#75 + PU L#75 (P#75)
    Socket L#38
      L3 L#76 (12MB) + L2 L#76 (256KB) + L1 L#76 (16KB) + Core L#76 + PU L#76 (P#76)
      L3 L#77 (12MB) + L2 L#77 (256KB) + L1 L#77 (16KB) + Core L#77 + PU L#77 (P#77)
    Socket L#39
      L3 L#78 (12MB) + L2 L#78 (256KB) + L1 L#78 (16KB) + Core L#78 + PU L#78 (P#78)
      L3 L#79 (12MB) + L2 L#79 (256KB) + L1 L#79 (16KB) + Core L#79 + PU L#79 (P#79)
  NUMANode L#10 (P#10 128GB)
    Socket L#40
      L3 L#80 (12MB) + L2 L#80 (256KB) + L1 L#80 (16KB) + Core L#80 + PU L#80 (P#80)
      L3 L#81 (12MB) + L2 L#81 (256KB) + L1 L#81 (16KB) + Core L#81 + PU L#81 (P#81)
    Socket L#41
      L3 L#82 (12MB) + L2 L#82 (256KB) + L1 L#82 (16KB) + Core L#82 + PU L#82 (P#82)
      L3 L#83 (12MB) + L2 L#83 (256KB) + L1 L#83 (16KB) + Core L#83 + PU L#83 (P#83)
    Socket L#42
      L3 L#84 (12MB) + L2 L#84 (256KB) + L1 L#84 (16KB) + Core L#84 + PU L#84 (P#84)
      L3 L#85 (12MB) + L2 L#85 (256KB) + L1 L#85 (16KB) + Core L#85 + PU L#85 (P#85)
    Socket L#43
      L3 L#86 (12MB) + L2 L#86 (256KB) + L1 L#86 (16KB) + Core L#86 + PU L#86 (P#86)
      L3 L#87 (12MB) + L2 L#87 (256KB) + L1 L#87 (16KB) + Core L#87 + PU L#87 (P#87)
  NUMANode L#11 (P#11 128GB)
    Socket L#44
      L3 L#88 (12MB) + L2 L#88 (256KB) + L1 L#88 (16KB) + Core L#88 + PU L#88 (P#88)
      L3 L#89 (12MB) + L2 L#89 (256KB) + L1 L#89 (16KB) + Core L#89 + PU L#89 (P#89)
    Socket L#45
      L3 L#90 (12MB) + L2 L#90 (256KB) + L1 L#90 (16KB) + Core L#90 + PU L#90 (P#90)
      L3 L#91 (12MB) + L2 L#91 (256KB) + L1 L#91 (16KB) + Core L#91 + PU L#91 (P#91)
    Socket L#46
      L3 L#92 (12MB) + L2 L#92 (256KB) + L1 L#92 (16KB) + Core L#92 + PU L#92 (P#92)
      L3 L#93 (12MB) + L2 L#93 (256KB) + L1 L#93 (16KB) + Core L#93 + PU L#93 (P#93)
    Socket L#47
      L3 L#94 (12MB) + L2 L#94 (256KB) + L1 L#94 (16KB) + Core L#94 + PU L#94 (P#94)
      L3 L#95 (12MB) + L2 L#95 (256KB) + L1 L#95 (16KB) + Core L#95 + PU L#95 (P#95)
  NUMANode L#12 (P#12 128GB)
    Socket L#48
      L3 L#96 (12MB) + L2 L#96 (256KB) + L1 L#96 (16KB) + Core L#96 + PU L#96 (P#96)
      L3 L#97 (12MB) + L2 L#97 (256KB) + L1 L#97 (16KB) + Core L#97 + PU L#97 (P#97)
    Socket L#49
      L3 L#98 (12MB) + L2 L#98 (256KB) + L1 L#98 (16KB) + Core L#98 + PU L#98 (P#98)
      L3 L#99 (12MB) + L2 L#99 (256KB) + L1 L#99 (16KB) + Core L#99 + PU L#99 (P#99)
    Socket L#50
      L3 L#100 (12MB) + L2 L#100 (256KB) + L1 L#100 (16KB) + Core L#100 + PU L#100 (P#100)
      L3 L#101 (12MB) + L2 L#101 (256KB) + L1 L#101 (16KB) + Core L#101 + PU L#101 (P#101)
    Socket L#51
      L3 L#102 (12MB) + L2 L#102 (256KB) + L1 L#102 (16KB) + Core L#102 + PU L#102 (P#102)
      L3 L#103 (12MB) + L2 L#103 (256KB) + L1 L#103 (16KB) + Core L#103 + PU L#103 (P#103)
  NUMANode L#13 (P#13 1009MB)


[root@worm0 ~]# free -m
             total       used       free     shared    buffers     cached
Mem:       1688992       8784    1680207          0         50        252
-/+ buffers/cache:       8481    1680510
Swap:            0          0          0
[root@worm0 ~]#

Comment 23 Lihua Xie 2011-08-01 05:50:08 UTC
Created attachment 516076 [details]
sx2000 topo

new sx2000 topo file ,without the mem=64G option in elilo.conf when booting the os.


Note You need to log in before you can comment on or make changes to this bug.