Bug 1333157

Created attachment 1153998 [details]
Reproducer

Suggest running:

# python reproducer.py | grep total

 I believe this is the "fix":

    diff --git a/yum/plugins.py b/yum/plugins.py
    index 6857626..fcdfac8 100644
    --- a/yum/plugins.py
    +++ b/yum/plugins.py
    @@ -160,7 +160,7 @@ class YumPlugins:
             self.cmdlines = {}
     
             # Call close handlers when yum exit's
    -        atexit.register(self.run, 'close')
    +        # atexit.register(self.run, 'close')
     
             # Let plugins register custom config file options
             self.run('config')


...which took way too long to debug :-o.

 In theory it's part of the plugin API, but we can probably just break it and any users can call atexit themselves (tmprepo is a notable one).

This is a cleaner patch which doesn't break the API, but won't leak if nobody uses the feature:

diff --git a/yum/plugins.py b/yum/plugins.py
index 6857626..e1e5270 100644
--- a/yum/plugins.py
+++ b/yum/plugins.py
@@ -160,7 +160,10 @@ class YumPlugins:
         self.cmdlines = {}
 
         # Call close handlers when yum exit's
-        atexit.register(self.run, 'close')
+        if self._pluginfuncs['close']:
+            self.verbose_logger.error(
+                _('One or more plugins uses "close" handling but should use atexit directly.'))
+            atexit.register(self.run, 'close')
 
         # Let plugins register custom config file options
         self.run('config')

James, Thanks for your attention on this!

I tested the patch using the reproducer and it is an improvement.  However still seeing RSS growth of 139004 KB / 100 (operations) = 139 KB per erase/install.

100 erase/install operations
__________________________________________
total kB          537052  194472  175804
total kB          545412  202836  184168
total kB          546616  204040  185372
total kB          552992  210416  191748
total kB          556720  214144  195476
total kB          562640  220064  201396
total kB          565328  222752  204084
total kB          569836  227260  208592
total kB          572352  229776  211108
total kB          575928  233352  214684
total kB          578768  236192  217524
total kB          582084  239508  220840
total kB          584816  242240  223572
total kB          586664  244088  225420
total kB          588200  245624  226956
total kB          591916  249340  230672
total kB          576644  234196  215528
total kB          578180  235732  217064
total kB          579204  236756  218088
total kB          580740  238292  219624
total kB          581508  239060  220392
total kB          583044  240596  221928
total kB          583812  241364  222696
total kB          584580  242132  223464
total kB          585604  243156  224488
total kB          586628  244180  225512
total kB          587908  245460  226792
total kB          589188  246740  228072
total kB          590212  247764  229096
total kB          591492  249044  230376
total kB          606548  264100  245432
total kB          609964  267516  248848
total kB          607828  265380  246712
total kB          609108  266660  247992
total kB          610644  268196  249528
total kB          611668  269220  250552
total kB          612436  269988  251320
total kB          613716  271268  252600
total kB          639768  297192  278524
total kB          642536  299960  281292
total kB          643740  301164  282496
total kB          645296  302720  284052
total kB          630032  287584  268916
total kB          630032  287584  268916
total kB          630544  288096  269428
total kB          633080  290632  271964
total kB          651608  309032  290364
total kB          651864  309404  290736
total kB          654252  311676  293008
total kB          653996  311424  292756
total kB          656436  313860  295192
total kB          657364  314788  296120
total kB          659700  317124  298456
total kB          660592  318016  299348
total kB          661520  318944  300276
total kB          661936  319360  300692
total kB          646404  303956  285288
total kB          646660  304212  285544
total kB          646404  303956  285288
total kB          666732  324156  305488
total kB          667996  325420  306752
total kB          669180  326604  307936
total kB          670476  327900  309232
total kB          671560  328984  310316
total kB          672484  329908  311240
total kB          672740  330164  311496
total kB          657928  315480  296812
total kB          657928  315480  296812
total kB          658440  315992  297324
total kB          658696  316128  297460
total kB          658952  316504  297836
total kB          659464  316888  298220
total kB          659464  317016  298348
total kB          659464  317016  298348
total kB          659464  317016  298348
total kB          659464  317016  298348
total kB          659976  317528  298860
total kB          659720  317272  298604
total kB          659976  317528  298860
total kB          659720  317272  298604
total kB          660232  317784  299116
total kB          660232  317784  299116
total kB          660744  318248  299580
total kB          660488  318040  299372
total kB          660744  318244  299576
total kB          661000  318552  299884
total kB          661512  319016  300348
total kB          661256  318808  300140
total kB          662024  319516  300848
total kB          661512  319064  300396
total kB          662024  319576  300908
total kB          661768  319320  300652
total kB          662024  319576  300908
total kB          670008  327560  308892
total kB          679964  337516  318848
total kB          680220  337772  319104
total kB          680220  337772  319104
total kB          679964  337516  318848
total kB          684288  341840  323172
total kB          684288  341840  323172

Ok, that was weird as I had it doing nothing for hundreds of runs.


Hmmm, setting init_plugins=False makes it not leak, is that true there?


Or *&^* this is ugly, I believe I know what it is, try this patch:

diff --git a/yum/plugins.py b/yum/plugins.py
index e1e5270..e00a7b4 100644
--- a/yum/plugins.py
+++ b/yum/plugins.py
@@ -15,6 +15,7 @@
 # Copyright 2005 Duke University
 
 import os
+import sys
 import glob
 import imp
 import warnings
@@ -276,13 +277,18 @@ class YumPlugins:
             self.verbose_logger.debug(_('Not loading "%s" plugin, as it is disabled'), modname)
             return
 
-        try:
+        #  If we are running a second YumBase() with the same plugins, then
+        # reloading the plugins leaks a bit or memory. So just take them.
+        if modname in sys.modules:
+            module = sys.modules[modname]
+        else:
+         try:
             fp, pathname, description = imp.find_module(modname, [dir])
             try:
                 module = imp.load_module(modname, fp, pathname, description)
             finally:
                 fp.close()
-        except:
+         except:
             if self.verbose_logger.isEnabledFor(logginglevels.DEBUG_4):
                 raise # Give full backtrace:
             self.verbose_logger.error(_('Plugin "%s" can\'t be imported') %



...I get hundreds of runs with no change again.

Created attachment 1154387 [details]
Patch to fix small memory leaks in yum with plugins

All in a single patch, I think this could pretty much go into el7 as is and will solve the problems without any other changes.

My initial report and numbers were based on observations running the reproducer on f23 which is really dnf.  Sorry for the confusion.

Running the reproducer on RHEL 7.2 with patch applied, I'm also not seeing any growth in RSS.  Although, even with the patch not applied, I'm not amble to reproduce on EL7.

yum-3.4.3-132.el7.noarch
____________________________
[root@el7 yum]# python /tmp/reproducer.py | grep total
total kB          339096   20244   13236
total kB          517468  102068   91868
total kB          513268   98136   87920
total kB          513268   98136   87920
total kB          513268   98136   87920
total kB          513268   98136   87920
total kB          513268   98136   87920
total kB          513268   98136   87920
total kB          513268   98136   87920
total kB          513268   98136   87920
total kB          513268   98136   87920
total kB          513268   98136   87920
total kB          513268   98136   87920
total kB          513268   98136   87920
total kB          513268   98136   87920

Running the reproducer on RHEL 6.5 with patch applied I'm seeing growth.
yum-3.2.29-43.el6_5.noarch
____________________________
[root@jortel-el6 yum]# patch < /tmp/yum.patch 
patching file plugins.py
Hunk #1 succeeded at 161 (offset 1 line).
patching file plugins.py
Hunk #2 succeeded at 162 (offset 1 line).
Hunk #3 succeeded at 200 (offset -4 lines).
Hunk #4 succeeded at 264 (offset -14 lines).
[root@jortel-el6 yum]# python /tmp/reproducer.py | grep total
total kB          302996   17568   11380
total kB          412964   92056   82016
total kB          447112  121448  110492
total kB          479340  150972  139232
total kB          511304  180288  167132
total kB          543488  209912  196204
total kB          575696  239580  225068
total kB          607660  269048  253216
total kB          639888  298824  282116

Updated the reproducer as:

  yb = YumBase()
+ yb.preconf.init_plugins = False

with seemingly not affect.  Did I do this right?

James, what versions are you testing / working with?

I was just using upstream on Fedora, as it's easiest to test here. I assumed there would be no difference between that and the el7 version on el7, as there hadn't been when we'd previously had leaks due to circular references etc..

Also for el6 VSZ going up by 22Mb per. run almost certainly implies one of the older leaks, of the kind I'd expected to find. It shouldn't be hard to find the patch to backport if you need the fixes there (someone just needs to go looking for the missing weakref).