Bug 159330

*** This bug has been marked as a duplicate of 106503 ***

Kevin, the title suggests that this is an x86_64 bug, but the "Steps to
Reproduce" section indidates an ia64-specific compilation.  Which arch
is this for?

The duplicate bug above (106503) was opened as an ia64 bug. The same methods to
reproduce work on x86_64. And the problem still exists for x86_64 (as stated in
the details for BZ 106503).

There are numerous ways to reproduce the problem, but basically do anything that
allocates and releases memory often enough over some span of time (sometimes a
very short span) and you'll see leakage in the Committed_AS value in /proc/meminfo.

Also, this was reported originally on ia64 and fixed there... then another bug
was fixed (probably for all platforms, but we saw it on ia32). It was never
fixed for x86_64 (even though that's what's said above in the initial description).

Created attachment 115060 [details]
Walk /pro/<pid>/maps to determine mem usage (supplied by Redhat)

Yep, Kevin just clipped the very beginning of a long thread where we saw this
on ia64, then ia32, and finally on x86_64 (all but the last are fixed).

With the latest U5 eratta kernel (2.4.21-32.0.1EL) we even see this w/ a test
script used to try to determine the *real* memory usage by walking
/proc/<pid>/maps, don't even need the reproducer app anymore.

 root@saias78:walkert# uname -a 					       
									       
						  
Linux saias78 2.4.21-32.0.1.EL #1 SMP Tue May 17 17:52:26 EDT 2005 x86_64      
									       
						 
root@saias78:walkert# uptime						       
									       
						 
 11:20:16  up 2 min,  1 user,  load average: 0.29, 0.20, 0.08		       
									       
						 
root@saias78:walkert# sysctl -a | grep overcommit			       
									       
						 
vm.overcommit_ratio = 90						       
									       
						 
vm.overcommit_memory = 2						       
									       
						 
root@saias78:walkert#  while true					       
									       
						 
> do									       
									       
						 
>  grep Committed_AS /proc/meminfo					       
									       
						 
> ~walkert/bin/procmem.sh > /dev/null					       
									       
						 
> done									       
									       
						 
Committed_AS:	258476 kB						       
									       
						 
Committed_AS:	353024 kB						       
									       
						 
Committed_AS:	447380 kB						       
									       
						 
Committed_AS:	541196 kB						       
									       
						 
Committed_AS:	635052 kB						       
									       
						 
Committed_AS:	728952 kB						       
									       
						 
Committed_AS:	823324 kB						       
									       
						 
Committed_AS:	916936 kB						       
									       
						 
Committed_AS:  1010488 kB						       
									       
						 
Committed_AS:  1104648 kB						       
									       
						 
Committed_AS:  1199292 kB						       
									       
						 
Committed_AS:  1293068 kB						       
									       
						 
Committed_AS:  1386588 kB						       
									       
						 
Committed_AS:  1480632 kB						       
									       
						 
Committed_AS:  1574084 kB						       
									       
						 
									       
									       
						 
<...>									       
									       
						 
									       
									       
						 
Until we implode...							       
									       
						 
									       
									       
						 
Committed_AS: 11151536 kB						       
									       
						 
Committed_AS: 11248696 kB						       
									       
						 
awk: fatal: more_nodes: nextfree: can't allocate 6400 bytes of memory	       
									       
						 
(Cannot allocate memory)						       
									       
						 
/ms/user/w/walkert/bin/procmem.sh: fork: Cannot allocate memory 	       
									       
						 
root@saias78:walkert# sysctl -w vm.overcommit_memory=1			       
									       
						 
vm.overcommit_memory = 1						       
									       
						 
root@saias78:walkert# uptime						       
									       
						 
 11:34:58  up 17 min,  1 user,	load average: 1.19, 1.58, 1.05		       
									       
						 
root@saias78:walkert# grep Commit /proc/meminfo 			       
									       
						 
CommitLimit:  11276400 kB						       
									       
						 
Committed_AS: 11271168 kB						       
									       
						 
root@saias78:walkert# ~walkert/bin/procmem.sh				       
									       
						 
<...>									       
									       
						 
Total process shared mem (x):	      31560 K				       
									       
						 
Total process private mem (w):	     212364 K				       
									       
						 
Total allocated (calc): 	     203844 K	 

Script attached (this is also in IT 28410)

Okay, thanks for clarifying things.  I've reclosed bug 106503 (against ia64,
which was due to a different problem) and have removed the dependency.

I ran the following test: With RHEL3 U5 x86_64 arch ia32e kernel

 [root@testsystem tst3]# sysctl -w vm.overcommit_memory=2
 vm.overcommit_memory = 2
 [root@testsystem tst3]# ./pig 
 fork failed:: Cannot allocate memory
 fork failed:: Cannot allocate memory
 fork failed:: Cannot allocate memory
 fork failed:: Cannot allocate memory
 fork failed:: Cannot allocate memory
 [root@testsystem tst3]# child is done
 child is done
 child is done
 child is done
 child is done
 child is done
 child is done
 At this point the test application is virtual hung. 
  It will never complete. It duplicates the result in
   Comment #28.

I then ran the following test: 

 [root@testsystem tst3]# sysctl -w vm.overcommit_memory=1
 vm.overcommit_memory = 1
 [root@testsystem tst3]# sysctl -a  | grep comm
 vm.overcommit_ratio = 50
 vm.overcommit_memory = 1
 [root@testsystem tst3]# ./pig 
 child is done
 child is done
 child is done
 child is done
 child is done
 child is done
 child is done
 child is done
 child is done
 (Above message many more times........)
 parent is done

 The test runs to completion. 

 I believe that the summary on the BZ is in fact still true. 
  "x86_63 kernel stops allocating memory too early when 
  overcommit_memory set to strict"

When /proc/sys/vm/overcommit_memory is set to 2 memory allocations will fail
pre-maturely.  The reason for this is that we are leaking vm_committed_space! 
Durring exec(), setup_arg_pages() calls vm_enough_memory() for a vma without the
VM_ACCOUNT flag set(every other call to vm_enough_memory either sets this flag
or insures that it is already set before making this call). When the process
exits, exit_mmap() only calls vm_unacct_memory() if the vma has the VM_ACCOUNT
flag set, therefore we leak vm_committed_space.  This is benign by default
because its ignored but it causes incorrect memory allocation failures when
overcommit_memory is set to strict(2).

This patch fixes this problem:

--------------------------------------------------------------------------------------------------------------------------------------------------
--- linux-2.4.21/include/asm-x86_64/page.h.orig
+++ linux-2.4.21/include/asm-x86_64/page.h
@@ -152,7 +152,8 @@ extern __inline__ int get_order(unsigned
#define __VM_DATA_DEFAULT_FLAGS        (VM_READ | VM_WRITE | VM_EXEC | \
                               VM_MAYREAD | VM_MAYWRITE | VM_MAYEXEC)
#define __VM_STACK_FLAGS       (VM_GROWSDOWN | VM_READ | VM_WRITE | VM_EXEC | \
-                                VM_MAYREAD | VM_MAYWRITE | VM_MAYEXEC)
+                                VM_MAYREAD | VM_MAYWRITE | VM_MAYEXEC | \
+                                VM_ACCOUNT)
                                                                               
                                 
#define VM_DATA_DEFAULT_FLAGS \
      ((current->thread.flags & THREAD_IA32) ? \
--------------------------------------------------------------------------------------------------------------------------------------------------------


The test kernel binary rpms for x86_64 and EM64T are located here:

>>>http://people.redhat.com/~lwoodman/RHEL3

A fix for this problem has just been committed to the RHEL3 U6
patch pool this evening (in kernel version 2.4.21-32.7.EL).

An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on the solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHSA-2005-663.html