Hide Forgot
Description of problem: While unit-testing the python-ethtool using TC#81755 I've observed an increased memory usage when functions get_active_devices() and get_interfaces_info() are called repeatedly (see bug 680559). The memory usage can be checked by the following script: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ #!/usr/bin/python import ethtool import subprocess, os max = 1000000 def memory(): """ Get current memory usage in MB """ command = "ps --no-header -o rss -p %s" % os.getpid() p = subprocess.Popen(command.split(), stdout=subprocess.PIPE) stdout, stderr = p.communicate() return int(stdout) / 1024 def measure(fun, arg): """ Run given function on give arg max times """ start = memory() print "\n%s on %s" % (fun, arg) for i in range(max + 1): result = fun(arg) if i % (max / 10) == 0: print "%s %% ... %s MB" % (str(100 * i / max).rjust(3), str(memory()).rjust(3)) print "Memory increase: %s MB" % (memory() - start) measure(ethtool.get_active_devices, None) measure(ethtool.get_interfaces_info, 'eth0') measure(ethtool.get_interfaces_info, 'bad interface name') measure(os.listdir, '/') ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Which on python-ethtool-0.3-5.1.el6.x86_64 gives: <built-in function get_active_devices> on None 0 % ... 36 MB 10 % ... 46 MB 20 % ... 56 MB 30 % ... 65 MB 40 % ... 75 MB 50 % ... 84 MB 60 % ... 94 MB 70 % ... 104 MB 80 % ... 113 MB 90 % ... 123 MB 100 % ... 132 MB Memory increase: 120 MB ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ And on the latest python-ethtool-0.6-1.el6.x86_64 gives: <built-in function get_active_devices> on None 0 % ... 36 MB 10 % ... 46 MB 20 % ... 55 MB 30 % ... 65 MB 40 % ... 74 MB 50 % ... 84 MB 60 % ... 93 MB 70 % ... 103 MB 80 % ... 112 MB 90 % ... 121 MB 100 % ... 131 MB Memory increase: 118 MB <built-in function get_interfaces_info> on eth0 0 % ... 131 MB 10 % ... 156 MB 20 % ... 181 MB 30 % ... 205 MB 40 % ... 230 MB 50 % ... 255 MB 60 % ... 280 MB 70 % ... 305 MB 80 % ... 329 MB 90 % ... 354 MB 100 % ... 379 MB Memory increase: 256 MB <built-in function get_interfaces_info> on bad interface name 0 % ... 379 MB 10 % ... 404 MB 20 % ... 429 MB 30 % ... 453 MB 40 % ... 478 MB 50 % ... 503 MB 60 % ... 528 MB 70 % ... 553 MB 80 % ... 577 MB 90 % ... 602 MB 100 % ... 627 MB Memory increase: 248 MB ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ So this is not a regression. However it seems there is some memory issue which causes memory to constantly grow and which should be investigated.
Memory check script once more, this time without syntax error ;-) #!/usr/bin/python import ethtool import subprocess, os max = 1000000 def memory(): """ Get current memory usage in MB """ command = "ps --no-header -o rss -p %s" % os.getpid() p = subprocess.Popen(command.split(), stdout=subprocess.PIPE) stdout, stderr = p.communicate() return int(stdout) / 1024 def measure(fun, arg): """ Run given function on give arg max times """ start = memory() print "\n%s on %s" % (fun, arg) for i in range(max + 1): result = fun(arg) if i % (max / 10) == 0: print "%s %% ... %s MB" % \ (str(100 * i / max).rjust(3), str(memory()).rjust(3)) print "Memory increase: %s MB" % (memory() - start) measure(ethtool.get_active_devices, None) measure(ethtool.get_interfaces_info, 'eth0') measure(ethtool.get_interfaces_info, 'bad interface name') measure(os.listdir, '/')
This should be fixed in upstream commit abc7f912f66d41dd734a10900429d4cad9377da5. http://fedorapeople.org/gitweb?p=dsommers/public_git/python-ethtool.git;a=commitdiff;h=abc7f912f66d41dd734a10900429d4cad9377da5 This is fix is also included in the python-ethtool-0.7 release. $ rpm -q python-ethtool python-ethtool-0.7-2.fc14.x86_64 $ python -c "import ethtool ; print ethtool.version" python-ethtool v0.7 $ python python-ethtool-memleak-test.py <built-in function get_active_devices> on None 0 % ... 37 MB 10 % ... 37 MB 20 % ... 37 MB 30 % ... 37 MB 40 % ... 37 MB 50 % ... 37 MB 60 % ... 37 MB 70 % ... 37 MB 80 % ... 37 MB 90 % ... 37 MB 100 % ... 37 MB Memory increase: 23 MB <built-in function get_interfaces_info> on eth0 0 % ... 37 MB 10 % ... 37 MB 20 % ... 37 MB 30 % ... 37 MB 40 % ... 37 MB 50 % ... 37 MB 60 % ... 37 MB 70 % ... 37 MB 80 % ... 37 MB 90 % ... 37 MB 100 % ... 37 MB Memory increase: 8 MB <built-in function get_interfaces_info> on bad interface name 0 % ... 37 MB 10 % ... 37 MB 20 % ... 37 MB 30 % ... 37 MB 40 % ... 37 MB 50 % ... 37 MB 60 % ... 37 MB 70 % ... 37 MB 80 % ... 37 MB 90 % ... 37 MB 100 % ... 37 MB Memory increase: 0 MB <built-in function listdir> on / 0 % ... 37 MB 10 % ... 37 MB 20 % ... 37 MB 30 % ... 37 MB 40 % ... 37 MB 50 % ... 37 MB 60 % ... 37 MB 70 % ... 37 MB 80 % ... 37 MB 90 % ... 37 MB 100 % ... 37 MB Memory increase: 0 MB
Results from running the memleak check script via valgrind: $ valgrind --leak-check=full python python-ethtool-memleak-test.py [...snip...] ==8434== LEAK SUMMARY: ==8434== definitely lost: 0 bytes in 0 blocks ==8434== indirectly lost: 0 bytes in 0 blocks ==8434== possibly lost: 971,380 bytes in 6,009 blocks ==8434== still reachable: 131,706 bytes in 3,055 blocks ==8434== suppressed: 0 bytes in 0 blocks
From my reading of comment #0, the rate of the leak is approximately: 118MB over a million calls to ethtool.get_active_devices(None) 256MB over a million calls to ethtool.get_interfaces_info('eth0') (and these "MB" are the 1024*1024 version of the unit) i.e. leaking about 124 bytes per call to ethtool.get_active_devices(None) leaking about 268 bytes per call to ethtool.get_interfaces_info('eth0') The precise amount leaked is dependent on the precise arguments to the function.
This request was evaluated by Red Hat Product Management for inclusion in the current release of Red Hat Enterprise Linux. Because the affected component is not scheduled to be updated in the current release, Red Hat is unfortunately unable to address this request at this time. Red Hat invites you to ask your support representative to propose this request, if appropriate and relevant, in the next release of Red Hat Enterprise Linux. If you would like it considered as an exception in the current release, please ask your support representative.
This request was not resolved in time for the current release. Red Hat invites you to ask your support representative to propose this request, if still desired, for consideration in the next release of Red Hat Enterprise Linux.
This request was erroneously removed from consideration in Red Hat Enterprise Linux 6.4, which is currently under development. This request will be evaluated for inclusion in Red Hat Enterprise Linux 6.4.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. http://rhn.redhat.com/errata/RHBA-2013-0454.html