Bug 612204 - Significant memory consumption by QMF python console applications.
Significant memory consumption by QMF python console applications.
Status: NEW
Product: Red Hat Enterprise MRG
Classification: Red Hat
Component: qpid-qmf (Show other bugs)
beta
All Linux
low Severity high
: ---
: ---
Assigned To: Ken Giusti
MRG Quality Engineering
: Reopened
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2010-07-07 10:39 EDT by Ken Giusti
Modified: 2013-02-26 09:21 EST (History)
2 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2011-03-10 13:29:23 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Ken Giusti 2010-07-07 10:39:33 EDT
Description of problem:

Under some circumstances, QMF console applications using the python console implementation consume huge amounts of memory.


Version-Release number of selected component (if applicable):
beta 1.3

How reproducible:
Not deterministice.

Steps to Reproduce:
1. Unknown
2.
3.
  
Actual results:

#
top - 09:45:40 up 184 days, 49 min,  1 user,  load average: 3.51, 3.87, 4.22
#
Tasks: 245 total,   1 running, 243 sleeping,   1 stopped,   0 zombie
#
Cpu(s):  0.3%us,  0.1%sy,  0.0%ni, 87.1%id, 12.4%wa,  0.0%hi,  0.1%si,  0.0%st
#
Mem:  12299640k total, 12234804k used,    64836k free,    39184k buffers
#
Swap: 18481144k total,  9270860k used,  9210284k free,   140668k cached
#
 
#
  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND           
#
 6700 root      15   0 11.8g  10g 1940 S  0.7 93.2   1590:43 condor_configd     
#
 2007 qpidd     15   0 2048m 126m 2592 S  2.7  1.1  38:55.52 qpidd             
#
20860 postgres  15   0  118m 7572 7376 S  0.0  0.1   1:13.72 postmaster         


Expected results:
Memory usage is expected to be much lower.

Additional info:
Broker appears to be experiencing queue backups at the same time - potentially related?
Comment 1 Ken Giusti 2010-07-07 10:41:16 EDT
pmap output of application exhibiting the problem:


   1.
      [root@mrg31 ~]# pmap -d 6700
   2.
      6700:   /usr/bin/python /usr/sbin/condor_configd
   3.
      Address           Kbytes Mode  Offset           Device    Mapping
   4.
      0000000000400000       4 r-x-- 0000000000000000 0fd:00000 python
   5.
      0000000000600000       4 rw--- 0000000000000000 0fd:00000 python
   6.
      000000000da17000  833348 rw--- 000000000da17000 000:00000   [ anon ]
   7.
      00000000407f0000       4 ----- 00000000407f0000 000:00000   [ anon ]
   8.
      00000000407f1000   10240 rw--- 00000000407f1000 000:00000   [ anon ]
   9.
      00000000411f1000       4 ----- 00000000411f1000 000:00000   [ anon ]
  10.
      00000000411f2000   10240 rw--- 00000000411f2000 000:00000   [ anon ]
  11.
      0000000041bf2000       4 ----- 0000000041bf2000 000:00000   [ anon ]
  12.
      0000000041bf3000   10240 rw--- 0000000041bf3000 000:00000   [ anon ]
  13.
      00000000425f3000       4 ----- 00000000425f3000 000:00000   [ anon ]
  14.
      00000000425f4000   10240 rw--- 00000000425f4000 000:00000   [ anon ]
  15.
      0000000042ff4000       4 ----- 0000000042ff4000 000:00000   [ anon ]
  16.
      0000000042ff5000   10240 rw--- 0000000042ff5000 000:00000   [ anon ]
  17.
      00000000439f5000       4 ----- 00000000439f5000 000:00000   [ anon ]
  18.
      00000000439f6000   10240 rw--- 00000000439f6000 000:00000   [ anon ]
  19.
      00000000443f6000       4 ----- 00000000443f6000 000:00000   [ anon ]
  20.
      00000000443f7000   10240 rw--- 00000000443f7000 000:00000   [ anon ]
  21.
      0000000044df7000       4 ----- 0000000044df7000 000:00000   [ anon ]
  22.
      0000000044df8000   10240 rw--- 0000000044df8000 000:00000   [ anon ]
  23.
      00000000457f8000       4 ----- 00000000457f8000 000:00000   [ anon ]
  24.
      00000000457f9000   10240 rw--- 00000000457f9000 000:00000   [ anon ]
  25.
      00000000461f9000       4 ----- 00000000461f9000 000:00000   [ anon ]
  26.
      00000000461fa000   10240 rw--- 00000000461fa000 000:00000   [ anon ]
  27.
      0000000046bfa000       4 ----- 0000000046bfa000 000:00000   [ anon ]
  28.
      0000000046bfb000   10240 rw--- 0000000046bfb000 000:00000   [ anon ]
  29.
      00000000475fb000       4 ----- 00000000475fb000 000:00000   [ anon ]
  30.
      00000000475fc000   10240 rw--- 00000000475fc000 000:00000   [ anon ]
  31.
      0000000047ffc000       4 ----- 0000000047ffc000 000:00000   [ anon ]
  32.
      0000000047ffd000   10240 rw--- 0000000047ffd000 000:00000   [ anon ]
  33.
      00000038aec00000     112 r-x-- 0000000000000000 0fd:00000 ld-2.5.so
  34.
      00000038aee1b000       4 r---- 000000000001b000 0fd:00000 ld-2.5.so
  35.
      00000038aee1c000       4 rw--- 000000000001c000 0fd:00000 ld-2.5.so
  36.
      00000038af000000    1332 r-x-- 0000000000000000 0fd:00000 libc-2.5.so
  37.
      00000038af14d000    2048 ----- 000000000014d000 0fd:00000 libc-2.5.so
  38.
      00000038af34d000      16 r---- 000000000014d000 0fd:00000 libc-2.5.so
  39.
      00000038af351000       4 rw--- 0000000000151000 0fd:00000 libc-2.5.so
  40.
      00000038af352000      20 rw--- 00000038af352000 000:00000   [ anon ]
  41.
      00000038af400000     520 r-x-- 0000000000000000 0fd:00000 libm-2.5.so
  42.
      00000038af482000    2044 ----- 0000000000082000 0fd:00000 libm-2.5.so
  43.
      00000038af681000       4 r---- 0000000000081000 0fd:00000 libm-2.5.so
  44.
      00000038af682000       4 rw--- 0000000000082000 0fd:00000 libm-2.5.so
  45.
      00000038af800000       8 r-x-- 0000000000000000 0fd:00000 libdl-2.5.so
  46.
      00000038af802000    2048 ----- 0000000000002000 0fd:00000 libdl-2.5.so
  47.
      00000038afa02000       4 r---- 0000000000002000 0fd:00000 libdl-2.5.so
  48.
      00000038afa03000       4 rw--- 0000000000003000 0fd:00000 libdl-2.5.so
  49.
      00000038afc00000      88 r-x-- 0000000000000000 0fd:00000 libpthread-2.5.so
  50.
      00000038afc16000    2044 ----- 0000000000016000 0fd:00000 libpthread-2.5.so
  51.
      00000038afe15000       4 r---- 0000000000015000 0fd:00000 libpthread-2.5.so
  52.
      00000038afe16000       4 rw--- 0000000000016000 0fd:00000 libpthread-2.5.so
  53.
      00000038afe17000      16 rw--- 00000038afe17000 000:00000   [ anon ]
  54.
      00000038b0000000      80 r-x-- 0000000000000000 0fd:00000 libz.so.1.2.3
  55.
      00000038b0014000    2044 ----- 0000000000014000 0fd:00000 libz.so.1.2.3
  56.
      00000038b0213000       4 rw--- 0000000000013000 0fd:00000 libz.so.1.2.3
  57.
      00000038b0800000     236 r-x-- 0000000000000000 0fd:00000 libsepol.so.1
  58.
      00000038b083b000    2048 ----- 000000000003b000 0fd:00000 libsepol.so.1
  59.
      00000038b0a3b000       4 rw--- 000000000003b000 0fd:00000 libsepol.so.1
  60.
      00000038b0a3c000      40 rw--- 00000038b0a3c000 000:00000   [ anon ]
  61.
      00000038b0c00000      84 r-x-- 0000000000000000 0fd:00000 libselinux.so.1
  62.
      00000038b0c15000    2048 ----- 0000000000015000 0fd:00000 libselinux.so.1
  63.
      00000038b0e15000       8 rw--- 0000000000015000 0fd:00000 libselinux.so.1
  64.
      00000038b0e17000       4 rw--- 00000038b0e17000 000:00000   [ anon ]
  65.
      00000038b3400000    1204 r-x-- 0000000000000000 0fd:00000 libcrypto.so.0.9.8e
  66.
      00000038b352d000    2044 ----- 000000000012d000 0fd:00000 libcrypto.so.0.9.8e
  67.
      00000038b372c000     132 rw--- 000000000012c000 0fd:00000 libcrypto.so.0.9.8e
  68.
      00000038b374d000      16 rw--- 00000038b374d000 000:00000   [ anon ]
  69.
      00000038b3c00000       8 r-x-- 0000000000000000 0fd:00000 libcom_err.so.2.1
  70.
      00000038b3c02000    2044 ----- 0000000000002000 0fd:00000 libcom_err.so.2.1
  71.
      00000038b3e01000       4 rw--- 0000000000001000 0fd:00000 libcom_err.so.2.1
  72.
      00000038b4000000      68 r-x-- 0000000000000000 0fd:00000 libresolv-2.5.so
  73.
      00000038b4011000    2048 ----- 0000000000011000 0fd:00000 libresolv-2.5.so
  74.
      00000038b4211000       4 r---- 0000000000011000 0fd:00000 libresolv-2.5.so
  75.
      00000038b4212000       4 rw--- 0000000000012000 0fd:00000 libresolv-2.5.so
  76.
      00000038b4213000       8 rw--- 00000038b4213000 000:00000   [ anon ]
  77.
      00000038b6400000       8 r-x-- 0000000000000000 0fd:00000 libkeyutils-1.2.so
  78.
      00000038b6402000    2044 ----- 0000000000002000 0fd:00000 libkeyutils-1.2.so
  79.
      00000038b6601000       4 rw--- 0000000000001000 0fd:00000 libkeyutils-1.2.so
  80.
      00000038b6800000      32 r-x-- 0000000000000000 0fd:00000 libkrb5support.so.0.1
  81.
      00000038b6808000    2044 ----- 0000000000008000 0fd:00000 libkrb5support.so.0.1
  82.
      00000038b6a07000       4 rw--- 0000000000007000 0fd:00000 libkrb5support.so.0.1
  83.
      00000038b6c00000     580 r-x-- 0000000000000000 0fd:00000 libkrb5.so.3.3
  84.
      00000038b6c91000    2048 ----- 0000000000091000 0fd:00000 libkrb5.so.3.3
  85.
      00000038b6e91000      16 rw--- 0000000000091000 0fd:00000 libkrb5.so.3.3
  86.
      00000038b7400000     276 r-x-- 0000000000000000 0fd:00000 libssl.so.0.9.8e
  87.
      00000038b7445000    2044 ----- 0000000000045000 0fd:00000 libssl.so.0.9.8e
  88.
      00000038b7644000      24 rw--- 0000000000044000 0fd:00000 libssl.so.0.9.8e
  89.
      00000038b7800000     176 r-x-- 0000000000000000 0fd:00000 libgssapi_krb5.so.2.2
  90.
      00000038b782c000    2048 ----- 000000000002c000 0fd:00000 libgssapi_krb5.so.2.2
  91.
      00000038b7a2c000       8 rw--- 000000000002c000 0fd:00000 libgssapi_krb5.so.2.2
  92.
      00000038b8e00000     144 r-x-- 0000000000000000 0fd:00000 libk5crypto.so.3.1
  93.
      00000038b8e24000    2044 ----- 0000000000024000 0fd:00000 libk5crypto.so.3.1
  94.
      00000038b9023000       8 rw--- 0000000000023000 0fd:00000 libk5crypto.so.3.1
  95.
      00000038bce00000       8 r-x-- 0000000000000000 0fd:00000 libutil-2.5.so
  96.
      00000038bce02000    2044 ----- 0000000000002000 0fd:00000 libutil-2.5.so
  97.
      00000038bd001000       4 r---- 0000000000001000 0fd:00000 libutil-2.5.so
  98.
      00000038bd002000       4 rw--- 0000000000002000 0fd:00000 libutil-2.5.so
  99.
      00000038be200000    1024 r-x-- 0000000000000000 0fd:00000 libpython2.4.so.1.0
 100.
      00000038be300000    2048 ----- 0000000000100000 0fd:00000 libpython2.4.so.1.0
 101.
      00000038be500000     180 rw--- 0000000000100000 0fd:00000 libpython2.4.so.1.0
 102.
      00000038be52d000      20 rw--- 00000038be52d000 000:00000   [ anon ]
 103.
      00002aaaaaaab000   21840 rw--- 00002aaaaaaab000 000:00000   [ anon ]
 104.
      00002aaaac000000     132 rw--- 00002aaaac000000 000:00000   [ anon ]
 105.
      00002aaaac021000   65404 ----- 00002aaaac021000 000:00000   [ anon ]
 106.
      00002aaab0000000 2880960 rw--- 00002aaab0000000 000:00000   [ anon ]
 107.
      00002aab5fd71000 2103728 rw--- 00002aab5fd71000 000:00000   [ anon ]
 108.
      00002aabe03de000  916436 rw--- 00002aabe03de000 000:00000   [ anon ]
 109.
      00002aac18311000 3342620 rw--- 00002aac18311000 000:00000   [ anon ]
 110.
      00002aace4396000  971244 rw--- 00002aace4396000 000:00000   [ anon ]
 111.
      00002aad1f812000  262856 rw--- 00002aad1f812000 000:00000   [ anon ]
 112.
      00002aad2f943000  676568 rw--- 00002aad2f943000 000:00000   [ anon ]
 113.
      00002aad58dfa000   65584 rw--- 00002aad58dfa000 000:00000   [ anon ]
 114.
      00002af42ab49000       4 rw--- 00002af42ab49000 000:00000   [ anon ]
 115.
      00002af42ab5d000     276 rw--- 00002af42ab5d000 000:00000   [ anon ]
 116.
      00002af42aba3000     260 rw--- 00002af42aba3000 000:00000   [ anon ]
 117.
      00002af42ac16000   55144 r---- 0000000000000000 0fd:00000 locale-archive
 118.
      00002af42e1f1000     520 rw--- 00002af42e1f1000 000:00000   [ anon ]
 119.
      00002af42e273000      12 r-x-- 0000000000000000 0fd:00000 timemodule.so
 120.
      00002af42e276000    2048 ----- 0000000000003000 0fd:00000 timemodule.so
 121.
      00002af42e476000       8 rw--- 0000000000003000 0fd:00000 timemodule.so
 122.
      00002af42e478000      20 r-x-- 0000000000000000 0fd:00000 stropmodule.so
 123.
      00002af42e47d000    2044 ----- 0000000000005000 0fd:00000 stropmodule.so
 124.
      00002af42e67c000       8 rw--- 0000000000004000 0fd:00000 stropmodule.so
 125.
      00002af42e67e000      12 r-x-- 0000000000000000 0fd:00000 cStringIO.so
 126.
      00002af42e681000    2048 ----- 0000000000003000 0fd:00000 cStringIO.so
 127.
      00002af42e881000       8 rw--- 0000000000003000 0fd:00000 cStringIO.so
 128.
      00002af42e883000      16 r-x-- 0000000000000000 0fd:00000 collectionsmodule.so
 129.
      00002af42e887000    2048 ----- 0000000000004000 0fd:00000 collectionsmodule.so
 130.
      00002af42ea87000       4 rw--- 0000000000004000 0fd:00000 collectionsmodule.so
 131.
      00002af42ea88000      40 r-x-- 0000000000000000 0fd:00000 _socketmodule.so
 132.
      00002af42ea92000    2044 ----- 000000000000a000 0fd:00000 _socketmodule.so
 133.
      00002af42ec91000      16 rw--- 0000000000009000 0fd:00000 _socketmodule.so
 134.
      00002af42ec95000      16 r-x-- 0000000000000000 0fd:00000 _ssl.so
 135.
      00002af42ec99000    2044 ----- 0000000000004000 0fd:00000 _ssl.so
 136.
      00002af42ee98000       4 rw--- 0000000000003000 0fd:00000 _ssl.so
 137.
      00002af42ee99000      12 r-x-- 0000000000000000 0fd:00000 mathmodule.so
 138.
      00002af42ee9c000    2048 ----- 0000000000003000 0fd:00000 mathmodule.so
 139.
      00002af42f09c000       4 rw--- 0000000000003000 0fd:00000 mathmodule.so
 140.
      00002af42f09d000      20 r-x-- 0000000000000000 0fd:00000 binascii.so
 141.
      00002af42f0a2000    2044 ----- 0000000000005000 0fd:00000 binascii.so
 142.
      00002af42f2a1000       4 rw--- 0000000000004000 0fd:00000 binascii.so
 143.
      00002af42f2a2000      12 r-x-- 0000000000000000 0fd:00000 _randommodule.so
 144.
      00002af42f2a5000    2044 ----- 0000000000003000 0fd:00000 _randommodule.so
 145.
      00002af42f4a4000       4 rw--- 0000000000002000 0fd:00000 _randommodule.so
 146.
      00002af42f4a5000      12 r-x-- 0000000000000000 0fd:00000 fcntlmodule.so
 147.
      00002af42f4a8000    2044 ----- 0000000000003000 0fd:00000 fcntlmodule.so
 148.
      00002af42f6a7000       8 rw--- 0000000000002000 0fd:00000 fcntlmodule.so
 149.
      00002af42f6aa000     260 rw--- 00002af42f6aa000 000:00000   [ anon ]
 150.
      00002af42f6eb000      20 r-x-- 0000000000000000 0fd:00000 structmodule.so
 151.
      00002af42f6f0000    2044 ----- 0000000000005000 0fd:00000 structmodule.so
 152.
      00002af42f8ef000       8 rw--- 0000000000004000 0fd:00000 structmodule.so
 153.
      00002af42f8f1000      60 r-x-- 0000000000000000 0fd:00000 datetime.so
 154.
      00002af42f900000    2044 ----- 000000000000f000 0fd:00000 datetime.so
 155.
      00002af42faff000      16 rw--- 000000000000e000 0fd:00000 datetime.so
 156.
      00002af42fb03000     260 rw--- 00002af42fb03000 000:00000   [ anon ]
 157.
      00002af42fb44000      68 r-x-- 0000000000000000 0fd:00000 cPickle.so
 158.
      00002af42fb55000    2048 ----- 0000000000011000 0fd:00000 cPickle.so
 159.
      00002af42fd55000       4 rw--- 0000000000011000 0fd:00000 cPickle.so
 160.
      00002af42fd57000     520 rw--- 00002af42fd57000 000:00000   [ anon ]
 161.
      00002af42fdda000    1032 rw--- 00002af42fdda000 000:00000   [ anon ]
 162.
      00002af42fedc000       8 r-x-- 0000000000000000 0fd:00000 grpmodule.so
 163.
      00002af42fede000    2044 ----- 0000000000002000 0fd:00000 grpmodule.so
 164.
      00002af4300dd000       4 rw--- 0000000000001000 0fd:00000 grpmodule.so
 165.
      00002af4300de000      16 r-x-- 0000000000000000 0fd:00000 zlibmodule.so
 166.
      00002af4300e2000    2044 ----- 0000000000004000 0fd:00000 zlibmodule.so
 167.
      00002af4302e1000       8 rw--- 0000000000003000 0fd:00000 zlibmodule.so
 168.
      00002af4302e3000      12 r-x-- 0000000000000000 0fd:00000 selectmodule.so
 169.
      00002af4302e6000    2048 ----- 0000000000003000 0fd:00000 selectmodule.so
 170.
      00002af4304e6000       4 rw--- 0000000000003000 0fd:00000 selectmodule.so
 171.
      00002af4304e8000     264 rw--- 00002af4304e8000 000:00000   [ anon ]
 172.
      00002af43053d000      40 r-x-- 0000000000000000 0fd:00000 libnss_files-2.5.so
 173.
      00002af430547000    2044 ----- 000000000000a000 0fd:00000 libnss_files-2.5.so
 174.
      00002af430746000       4 r---- 0000000000009000 0fd:00000 libnss_files-2.5.so
 175.
      00002af430747000       4 rw--- 000000000000a000 0fd:00000 libnss_files-2.5.so
 176.
      00002af430748000      16 r-x-- 0000000000000000 0fd:00000 libnss_dns-2.5.so
 177.
      00002af43074c000    2044 ----- 0000000000004000 0fd:00000 libnss_dns-2.5.so
 178.
      00002af43094b000       4 r---- 0000000000003000 0fd:00000 libnss_dns-2.5.so
 179.
      00002af43094c000       4 rw--- 0000000000004000 0fd:00000 libnss_dns-2.5.so
 180.
      00007fff2f2a0000      84 rw--- 00007ffffffea000 000:00000   [ stack ]
 181.
      ffffffffff600000    8192 ----- 0000000000000000 000:00000   [ anon ]
 182.
      mapped: 12421472K    writeable/private: 12212588K    shared: 0K
Comment 2 Matthew Farrellee 2010-07-07 14:23:07 EDT
  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND           
 6700 root      15   0 11.8g  10g 1940 S  0.7 93.2   1590:43 condor_configd     

I'd note that the condor_configd typically uses a vanishingly small amount of CPU. Some abnormal event must have occurred for it to accumulate so much TIME.

Note You need to log in before you can comment on or make changes to this bug.