From Bugzilla Helper: User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.7) Gecko/20050414 Firefox/1.0.3 Description of problem: Applications running with ia32el setvice takes up too much of memory. Version-Release number of selected component (if applicable): ia32el-1.1-9.EL3 How reproducible: Always Steps to Reproduce: 1.Download the test program attached. 2.Compile it on a 32 bit box 3.Execute the test program with service and down. 4.Get the memory mapping using pmap <pid>, where pid is the process id of the program in execution. The sample program creates a thread, in the thread task it registers handler for SIGCHLD. The parent process creates infinite number of processes. These processes send SIGCHLD, which are caught by the handler. Actual Results: When the sample program is executed with service up the pmap command shows 2952: signal_test Start Size Perm Mapping 00000000 16K r--p [ anon ] 08048000 144K rwxp [ anon ] 40000000 80K r-xp /emul/ia32-linux/lib/ld-2.3.2.so 40014000 32K rwxp [ anon ] 4003c000 48K r-xp /emul/ia32-linux/lib/i686/libpthread-0.10.so 40048000 304K rwxp [ anon ] 40094000 208K r-xp /emul/ia32-linux/usr/lib/libstdc++-3-libc6.2-2-2.10.0.so 400c8000 16K rwxp [ anon ] 400cc000 32K rwxp /emul/ia32-linux/usr/lib/libstdc++-3-libc6.2-2-2.10.0.so 400d4000 16K rwxp [ anon ] 400d8000 128K r-xp /emul/ia32-linux/lib/i686/libm-2.3.2.so 400f8000 16K rwxp [ anon ] 400fc000 1232K r-xp /emul/ia32-linux/lib/i686/libc-2.3.2.so 40230000 32K rwxp [ anon ] 40238000 32K r-xp /emul/ia32-linux/lib/libgcc_s-3.2.3-20040701.so.1 40240000 10272K rwxp [ anon ] bffe0000 128K rwxp [ anon ] c0000000 16K rwxs /dev/zero 2000000000000000 224K rwxp [ anon ] 2000000100000000 64K r-xp /usr/lib/ia32el/ia32exec.bin 2000000100010000 3456K r-xs /usr/lib/ia32el/ia32exec.bin 2000000100370000 2496K r-xp /usr/lib/ia32el/ia32exec.bin 20000001005e0000 64K r-xs /usr/lib/ia32el/ia32exec.bin 20000001005f0000 896K rwxp /usr/lib/ia32el/ia32exec.bin 20000001006d0000 1472K rwxp [ anon ] 2000000100840000 64K rwxp /usr/lib/ia32el/ia32exec.bin 2000000100850000 64K rwxp [ anon ] 2000000100860000 64K r-xs /usr/lib/ia32el/ia32exec.bin 2000000110000000 10240K rwxp [ anon ] 2000000120000000 10240K rwxp [ anon ] 2000000130000000 1664K rwxp [ anon ] 2000000150000000 1120K rwxp [ anon ] 4000000000000000 352K r-xp /usr/lib/ia32el/libia32x.so 6000000000004000 32K rwxp /usr/lib/ia32el/libia32x.so 600000000000c000 96K rw-p [ anon ] 6000000000024000 32K rwxp [ anon ] 60000fff80000000 16K rw-p [ anon ] 60000fffffff4000 32K rwxp [ stack ] mapped: 45440K writeable/private: 37184K shared: 3600K The memory mapped is shown as 45440K bytes. Expected Results: 31837: signal_test Start Size Perm Mapping 08048000 16K rwxp [ anon ] 0804c000 144K rwxp [ anon ] 40000000 80K r-xp /emul/ia32-linux/lib/ld-2.3.2.so 40014000 16K rwxp [ anon ] 40018000 16K rwxp [ anon ] 40038000 16K r-xp /etc/ld.so.cache 4003c000 48K r-xp /emul/ia32-linux/lib/i686/libpthread-0.10.so 40048000 304K rwxp [ anon ] 40094000 208K r-xp /emul/ia32-linux/usr/lib/libstdc++-3-libc6.2-2-2.10.0.so 400c8000 16K rwxp [ anon ] 400cc000 32K rwxp /emul/ia32-linux/usr/lib/libstdc++-3-libc6.2-2-2.10.0.so 400d4000 16K rwxp [ anon ] 400d8000 128K r-xp /emul/ia32-linux/lib/i686/libm-2.3.2.so 400f8000 16K rwxp [ anon ] 400fc000 1232K r-xp /emul/ia32-linux/lib/i686/libc-2.3.2.so 40230000 32K rwxp [ anon ] 40238000 32K r-xp /emul/ia32-linux/lib/libgcc_s-3.2.3-20040701.so.1 40240000 32K rwxp [ anon ] bfffc000 16K rwxp [ stack ] c0000000 16K r--p [ anon ] c0008000 64K rw-p [ anon ] mapped: 2480K writeable/private: 720K shared: 0K The same program when executed with service down the memory mapped comes to 2480K bytes. Additional info:
Created attachment 121180 [details] Test Prgram
Created attachment 121181 [details] Sample program executes the script
if i understand correctly, the problem that you see is with the memory allocated about 4G. is this right ? if so, then please keep in mind that ia32el is a piece of software. it is loaded to memory and uses memory for its own purposes. i'm not sure there is a way to avoid these need for extra memory. thanks, yoav.
Hi Yoav, I can understand that, if there is software it would take memory. But when it comes to actually deploying software in production environment it becomes difficult to have applications taking so much of memory. I would request you to please look the piece of code sent by me and if nothing else suggest a workaround; 4GB with such a small piece of code is something which worries me. Thanks, Vaibhav Khanduja
i made a mistake - i meant to say 'above 4G', not 'about 4G', meaning that the extra memory is mapped at addresses higher than 4G. sorry for the confusion. i see the test case with ia32el consumes about 45MB. from our experience this is expected. i would expect to see the same memory-footprint even for running 'hello world'. please remember - the program that really executes is ia32el, which is quite a large application, not the ia32 binary. thanks, yoav.
Would it be possible for you to somehow instrument the service and see if the memory consumption can be tunned a bit? Apart from this, with what confidence level can you say that the memory consumption would remain same if the service is kept running over a period of time, with more than one program making use of the service? thanks, Vaibhav Khanduja
we are continuously working on improving ia32el. memory footprint is one of the factors we try to improve. given your input, we will put more focus on this specific factor now. there is however one point that skipped my eyes before, this is the allocation of 10M at address 40240000. we will investigate this issue. as for your second question - it has two parts really : about many programs running with ia32el - there are parts in libia32x.so and ia32exec.bin that are not writable. these parts (~7M) will be shared among all the ia32el processes in the system. all the other parts will not be shared. about running for a long period of time - there is a known problem of memory leak in libia32x.so, which is fixed in the version that will be shipped with EL3U7/EL4U3. we don't know of any other such issue.
We recheked the issue with the large allocation below the ia32 stack - running the attached test case on native ia32 EL3 machine gives the same amount of mapping. as for running the test on ipf machine with ia32el disabled - we usaully compare ia32el with native ia32, and not with the hw emulation on ipf. nevertheless, we repeated this test with the hw emulation and got same results in some of the runs. the memory consumption was as read from /proc/*/maps file was not consistent in all runs, but again - in some of the runs it showed the same amount of allocated memory. will it be possible for you to verify our findings ? thanks, yoav.
Would it be possible for you send the version of ia32el service or in patch form, which has fixed the memory leak? Regards, Vaibhav Khanduja
Hi Yoav, I have some more data for you. I am attaching another sample program which periodically logs its resident stack size in a file. The program when executed with the service enabled the resident stack size tends to increase more frequently as compared to when the program is executed without service being enabled i.e. using the kernel emulation. I compiled this program on a Debian 3.0 x86 box with kernel 2.4.9-7. The program was executed on Redhat 3.0 kernel 2.4.21-38.EL IA64 with ia32el-1.3- 1.EL3 as the service version. Logs are also attached along with the program. Thanks, Vaibhav Khanduja
Created attachment 125081 [details] Resident Stack Size Checking Program
Created attachment 125082 [details] Logs when service is on
Created attachment 125083 [details] Logs when service is down
Hi, Vaibhav, Thanks for your information. I will take over Yoav for this work. I will start to investigate and keep you updated Eric.
This bug has been opened for more than a year now, with no clear progress. Is this even still an issue? There has been one new release since then.
This bug is filed against RHEL 3, which is in maintenance phase. During the maintenance phase, only security errata and select mission critical bug fixes will be released for enterprise products. Since this bug does not meet that criteria, it is now being closed. For more information of the RHEL errata support policy, please visit: http://www.redhat.com/security/updates/errata/ If you feel this bug is indeed mission critical, please contact your support representative. You may be asked to provide detailed information on how this bug is affecting you.