Glen, I think this is a NUMA issue, please add "numa=off" to the command line, reboot and and let me know if the problem goes away. Larry Woodman
----- Additional Comments From ysluk.com 2004-08-23 18:43 ------- The problem goes away after rebooting with "numa=off" Thanks!
----- Additional Comments From salina.com 2004-08-24 11:07 ------- Question for RedHat, Is numa=on by default on RHEL 3 U3 ? If this does not work, it will be better if this is off by default i.e. mayne until RHEL 4 where 2.6 kernel has more stable numa code ?? Thhanks.
For RHEL3-U3 NUMA is ON by default and its too late to change it now. For RHEL3-U4 wa are investigating what changes would be necessary to allow it to remain ON by default and still work correctly under high memory load situations. Larry Woodman
----- Additional Comments From salina.com 2004-08-24 13:29 ------- Ananda, I have added you to cc list. Please see if you are hitting this on p-series with NUMA=on. Thanks.
----- Additional Comments From AVenkat.com 2004-08-24 13:32 ------- Thanks for letting me know. Will check and let you know. --Ananda
I think this problem is restricted to Opteron. Larry
----- Additional Comments From salina.com 2004-08-24 13:25 ------- Based on response from Larry, I am going to put target milesstone in as RHEL 3 QU4. We will leave problem open until numa=on works. Thanks.
*** This bug has been marked as a duplicate of 131295 ***
----- Additional Comments From AVenkat.com 2004-12-01 22:19 EDT ------- I haven't used numa feature so far on pSeries. Probably, I might not be using the large pages since the squadron boxes that I use have only 2GB (SF2) or 4GB (L4). However, I will try to borrow one box with a larger memory.
It has been decided that x86_64 RHEL3 kernels should continue to enable NUMA by default. However, if an OOM kill occurs on a NUMA system, an extra message will be printed by the kernel suggesting that using the "numa=off" boot option might be a good way to work around the issue. The exact message is: OOM kill occurred on an x86_64 NUMA system! The numa=off boot option might help avoid this. This change was committed to the RHEL3 U5 patch pool on 9-Feb-2005 (in kernel version 2.4.21-27.12.EL).
A fix for this problem has just been committed to the RHEL3 U7 patch pool this evening (in kernel version 2.4.21-37.12.EL). To enable an improved NUMA-friendly page allocation policy, please set /proc/sys/vm/numa_memory_allocator via the "sysctl" command (or put "vm.numa_memory_allocator = 1" in /etc/sysctl.conf).
An advisory has been issued which should help the problem described in this bug report. This report is therefore being closed with a resolution of ERRATA. For more information on the solution and/or where to find the updated files, please follow the link below. You may reopen this bug report if the solution does not work for you. http://rhn.redhat.com/errata/RHSA-2006-0144.html