Bug 173470

Summary: ia32el service takes too much of memory
Product: Red Hat Enterprise Linux 3 Reporter: Vaibhav Khanduja <vaibhav.khanduja>
Component: ia32elAssignee: Petr Machata <pmachata>
Status: CLOSED WONTFIX QA Contact:
Severity: medium Docs Contact:
Priority: medium    
Version: 3.0CC: eric.lin, mnewsome, yoav.zach
Target Milestone: ---   
Target Release: ---   
Hardware: ia64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2007-10-19 18:50:51 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Test Prgram
none
Sample program executes the script
none
Resident Stack Size Checking Program
none
Logs when service is on
none
Logs when service is down none

Description Vaibhav Khanduja 2005-11-17 12:28:21 UTC
From Bugzilla Helper:
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.7) Gecko/20050414 Firefox/1.0.3

Description of problem:
Applications running with ia32el setvice takes up too much of memory.

Version-Release number of selected component (if applicable):
ia32el-1.1-9.EL3

How reproducible:
Always

Steps to Reproduce:
1.Download the test program attached.
2.Compile it on a 32 bit box
3.Execute the test program with service and down.
4.Get the memory mapping using pmap <pid>, where pid is the process id of the program in execution.

The sample program creates a thread, in the thread task it registers handler for SIGCHLD. The parent process creates infinite number of processes. These processes send SIGCHLD, which are caught by the handler.
  

Actual Results:  When the sample program is executed with service  up the pmap command shows 

2952: signal_test
Start         Size Perm Mapping
00000000       16K r--p [ anon ]
08048000      144K rwxp [ anon ]
40000000       80K r-xp /emul/ia32-linux/lib/ld-2.3.2.so
40014000       32K rwxp [ anon ]
4003c000       48K r-xp /emul/ia32-linux/lib/i686/libpthread-0.10.so
40048000      304K rwxp [ anon ]
40094000      208K r-xp /emul/ia32-linux/usr/lib/libstdc++-3-libc6.2-2-2.10.0.so
400c8000       16K rwxp [ anon ]
400cc000       32K rwxp /emul/ia32-linux/usr/lib/libstdc++-3-libc6.2-2-2.10.0.so
400d4000       16K rwxp [ anon ]
400d8000      128K r-xp /emul/ia32-linux/lib/i686/libm-2.3.2.so
400f8000       16K rwxp [ anon ]
400fc000     1232K r-xp /emul/ia32-linux/lib/i686/libc-2.3.2.so
40230000       32K rwxp [ anon ]
40238000       32K r-xp /emul/ia32-linux/lib/libgcc_s-3.2.3-20040701.so.1
40240000    10272K rwxp [ anon ]
bffe0000      128K rwxp [ anon ]
c0000000       16K rwxs /dev/zero
2000000000000000      224K rwxp [ anon ]
2000000100000000       64K r-xp /usr/lib/ia32el/ia32exec.bin
2000000100010000     3456K r-xs /usr/lib/ia32el/ia32exec.bin
2000000100370000     2496K r-xp /usr/lib/ia32el/ia32exec.bin
20000001005e0000       64K r-xs /usr/lib/ia32el/ia32exec.bin
20000001005f0000      896K rwxp /usr/lib/ia32el/ia32exec.bin
20000001006d0000     1472K rwxp [ anon ]
2000000100840000       64K rwxp /usr/lib/ia32el/ia32exec.bin
2000000100850000       64K rwxp [ anon ]
2000000100860000       64K r-xs /usr/lib/ia32el/ia32exec.bin
2000000110000000    10240K rwxp [ anon ]
2000000120000000    10240K rwxp [ anon ]
2000000130000000     1664K rwxp [ anon ]
2000000150000000     1120K rwxp [ anon ]
4000000000000000      352K r-xp /usr/lib/ia32el/libia32x.so
6000000000004000       32K rwxp /usr/lib/ia32el/libia32x.so
600000000000c000       96K rw-p [ anon ]
6000000000024000       32K rwxp [ anon ]
60000fff80000000       16K rw-p [ anon ]
60000fffffff4000       32K rwxp [ stack ]
mapped: 45440K    writeable/private: 37184K    shared: 3600K

The memory mapped is shown as 45440K bytes. 

Expected Results:  31837: signal_test
Start         Size Perm Mapping
08048000       16K rwxp [ anon ]
0804c000      144K rwxp [ anon ]
40000000       80K r-xp /emul/ia32-linux/lib/ld-2.3.2.so
40014000       16K rwxp [ anon ]
40018000       16K rwxp [ anon ]
40038000       16K r-xp /etc/ld.so.cache
4003c000       48K r-xp /emul/ia32-linux/lib/i686/libpthread-0.10.so
40048000      304K rwxp [ anon ]
40094000      208K r-xp /emul/ia32-linux/usr/lib/libstdc++-3-libc6.2-2-2.10.0.so
400c8000       16K rwxp [ anon ]
400cc000       32K rwxp /emul/ia32-linux/usr/lib/libstdc++-3-libc6.2-2-2.10.0.so
400d4000       16K rwxp [ anon ]
400d8000      128K r-xp /emul/ia32-linux/lib/i686/libm-2.3.2.so
400f8000       16K rwxp [ anon ]
400fc000     1232K r-xp /emul/ia32-linux/lib/i686/libc-2.3.2.so
40230000       32K rwxp [ anon ]
40238000       32K r-xp /emul/ia32-linux/lib/libgcc_s-3.2.3-20040701.so.1
40240000       32K rwxp [ anon ]
bfffc000       16K rwxp [ stack ]
c0000000       16K r--p [ anon ]
c0008000       64K rw-p [ anon ]
mapped: 2480K    writeable/private: 720K    shared: 0K

The same program when executed with service down the memory mapped comes to 2480K bytes.


Additional info:

Comment 1 Vaibhav Khanduja 2005-11-17 12:29:38 UTC
Created attachment 121180 [details]
Test Prgram

Comment 2 Vaibhav Khanduja 2005-11-17 12:30:20 UTC
Created attachment 121181 [details]
Sample program executes  the script

Comment 3 Yoav Zach 2005-11-17 13:04:08 UTC
if i understand correctly, the problem that you see is with the memory 
allocated about 4G. is this right ?
if so, then please keep in mind that ia32el is a piece of software. it is 
loaded to memory and uses memory for its own purposes. i'm not sure there is a 
way to avoid these need for extra memory.

thanks,
yoav.

Comment 4 Vaibhav Khanduja 2005-11-17 13:32:30 UTC
Hi Yoav,

I can understand that, if there is software it would take memory. But when it
comes to actually deploying software in production environment it becomes
difficult to have applications taking so much of memory. 
I would request you to please look the piece of code sent by me and if nothing
else suggest a workaround;  4GB with such a small piece of code is something
which worries me.


Thanks,
Vaibhav Khanduja

Comment 5 Yoav Zach 2005-11-17 13:47:19 UTC
i made a mistake - i meant to say 'above 4G', not 'about 4G', meaning that the 
extra memory is mapped at addresses higher than 4G. sorry for the confusion.
i see the test case with ia32el consumes about 45MB. from our experience this 
is expected. i would expect to see the same memory-footprint even for 
running 'hello world'. please remember - the program that really executes is 
ia32el, which is quite a large application, not the ia32 binary.

thanks,
yoav.

Comment 6 Vaibhav Khanduja 2005-11-21 09:42:33 UTC
Would it be possible for you to somehow instrument the service and see if the 
memory consumption can be tunned a bit?
Apart from this, with what confidence level can you say that the memory 
consumption would remain same if the service is kept running over a period of 
time, with more than one program making use of the service?

thanks,
Vaibhav Khanduja


Comment 7 Yoav Zach 2005-11-22 07:56:58 UTC
we are continuously working on improving ia32el. memory footprint is one of the 
factors we try to improve. given your input, we will put more focus on this 
specific factor now.
there is however one point that skipped my eyes before, this is the allocation 
of 10M at address 40240000. we will investigate this issue.
as for your second question - it has two parts really : about many programs 
running with ia32el - there are parts in libia32x.so and ia32exec.bin that are 
not writable. these parts (~7M) will be shared among all the ia32el processes 
in the system. all the other parts will not be shared. about running for a long 
period of time - there is a known problem of memory leak in libia32x.so, which 
is fixed in the version that will be shipped with EL3U7/EL4U3. we don't know of 
any other such issue.

Comment 8 Yoav Zach 2005-11-27 21:39:21 UTC
We recheked the issue with the large allocation below the ia32 stack - running 
the attached test case on native ia32 EL3 machine gives the same amount of 
mapping. as for running the test on ipf machine with ia32el disabled - we 
usaully compare ia32el with native ia32, and not with the hw emulation on ipf. 
nevertheless, we repeated this test with the hw emulation and got same results 
in some of the runs. the memory consumption was as read from /proc/*/maps file 
was not consistent in all runs, but again - in some of the runs it showed the 
same amount of allocated memory.
will it be possible for you to verify our findings ?

thanks,
yoav.

Comment 9 Vaibhav Khanduja 2005-12-06 09:35:05 UTC
Would it be possible for you send the version of ia32el service or in patch 
form, which has fixed the memory leak?

Regards,
Vaibhav Khanduja

Comment 10 Vaibhav Khanduja 2006-02-23 06:14:05 UTC
Hi Yoav,

I have some more data for you. I am attaching another sample program which 
periodically logs its resident stack size in a file. The program when executed 
with the service enabled the resident stack size tends to increase more 
frequently as compared to when the program is executed without service being 
enabled i.e. using the kernel emulation. 

I compiled this program on a Debian 3.0 x86 box with kernel 2.4.9-7. The 
program was executed on Redhat 3.0 kernel 2.4.21-38.EL IA64 with ia32el-1.3-
1.EL3 as the service version.

Logs are also attached along with the program.

Thanks,
Vaibhav Khanduja


Comment 11 Vaibhav Khanduja 2006-02-23 06:18:22 UTC
Created attachment 125081 [details]
Resident Stack Size Checking Program

Comment 12 Vaibhav Khanduja 2006-02-23 06:20:15 UTC
Created attachment 125082 [details]
Logs when service is on

Comment 13 Vaibhav Khanduja 2006-02-23 06:21:21 UTC
Created attachment 125083 [details]
Logs when service is down

Comment 14 Eric Lin 2006-02-27 03:27:20 UTC
Hi, Vaibhav, 

Thanks for your information. I will take over Yoav for this work. I will start 
to investigate and keep you updated

Eric.

Comment 15 Petr Machata 2007-05-23 14:23:08 UTC
This bug has been opened for more than a year now, with no clear progress.  Is
this even still an issue?  There has been one new release since then.

Comment 16 RHEL Program Management 2007-10-19 18:50:51 UTC
This bug is filed against RHEL 3, which is in maintenance phase.
During the maintenance phase, only security errata and select mission
critical bug fixes will be released for enterprise products. Since
this bug does not meet that criteria, it is now being closed.
 
For more information of the RHEL errata support policy, please visit:
http://www.redhat.com/security/updates/errata/
 
If you feel this bug is indeed mission critical, please contact your
support representative. You may be asked to provide detailed
information on how this bug is affecting you.