From Bugzilla Helper: User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.5) Gecko/20041107 Firefox/1.0 Description of problem: Kernel killing off processes including oracle procs when lowmem is exhausted: Sar data indicates that there is still 1.2 gigs of RAM unused and all of 4 Gigs of swap unused when the event occurs. System is HP DL Dec 12 04:02:05 fadvdb02 kernel: 197896 pages of slabcache Dec 12 04:02:05 fadvdb02 kernel: 516 pages of kernel stacks Dec 12 04:02:05 fadvdb02 kernel: 0 lowmem pagetables, 740 highmem pagetables Dec 12 04:02:05 fadvdb02 kernel: Free swap: 4092064kB Dec 12 04:02:05 fadvdb02 kernel: 1310719 pages of RAM Dec 12 04:02:05 fadvdb02 kernel: 1015801 pages of HIGHMEM Dec 12 04:02:05 fadvdb02 kernel: 85979 reserved pages Dec 12 04:02:05 fadvdb02 kernel: 51499 pages shared Dec 12 04:02:05 fadvdb02 kernel: 0 pages swap cached Dec 12 04:02:05 fadvdb02 kernel: Out of Memory: Killed process 28221 (oracle). Dec 12 04:02:11 fadvdb02 kernel: Mem-info: Dec 12 04:02:11 fadvdb02 kernel: Zone:DMA freepages: 2648 min: 0 low: 0 high: 0 Dec 12 04:02:11 fadvdb02 kernel: Zone:Normal freepages: 1103 min: 1279 low: 4544 high: 6304 Dec 12 04:02:11 fadvdb02 kernel: Zone:HighMem freepages:874206 min: 255 low: 16894 high: 25341 Version-Release number of selected component (if applicable): 2.4.21-15.ELsmp How reproducible: Didn't try Additional info: Event has occured multiples times on this hosts and others.
A number of fixes related to this went into the RHEL3 U4 kernel, which was released yesterday. Could you please confirm whether or not this problem is still present in kernel-2.4.21-27EL ? I believe that the latest kernel should fix your problem.
The RHEL3 U4 kernel advisory is RHBA-2004:550.
Joe, we are still waiting for confirmation that this problem has been fixed in then latest RHEL3 kernel. Can you verify this so we can close this bug? Thaank You, Larry Woodman
Joe, we are still waiting for your verification that the RHEL3-U4 update fixed this problem. Your system exhausted lowmem in the slabcache: >>>197896 pages of slabcache The previous version had a problem reclaiming inodes from the inode cache which are allocated form the slabcache. I fixed that problem in U4, can you verify this so I can close this bug? Thanks for your help on this Joe, Larry Woodman
I have a server with 16gig of memeory running Oracle and Informatica. The kernel is 2.4.21-27smp and I am experiencing the same memory issues. The applications appear to be exhausting the lowmem. I have over 10 gig of highmem available.
Michael, please attach the show_mem() output that is written to /var/log/messages when the OOM kill occurs so I can see where the lowmem on your system is going. While you are at it please get a "/proc/slabinfo" output just incase your slabcache is consuming all of lowmem as well. Thanks, Larry Woodman
Created attachment 114775 [details] Out of memory error
The attched showmem is from 16GB system running the SMP kernel. Lowmem is totally exhausted with the mem_map and slabcache!!! Please use the Hugemem kernel and this problem will go away. Larry Woodman
Closing due to lack of reporter's response to comment #1.