Bug 1461423
| Summary: | Memory leak in python-dnf | ||
|---|---|---|---|
| Product: | [Fedora] Fedora | Reporter: | Florian Ludwig <vierzigundzwei> |
| Component: | dnf | Assignee: | Jaroslav Mracek <jmracek> |
| Status: | CLOSED ERRATA | QA Contact: | Fedora Extras Quality Assurance <extras-qa> |
| Severity: | unspecified | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 26 | CC: | jmracek, mhatina, packaging-team-maint, rpm-software-management, vmukhame |
| Target Milestone: | --- | Keywords: | Triaged |
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | dnf-2.7.4-1.fc26 dnf-2.7.4-1.fc27 | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2017-10-25 23:11:22 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
I tried to reproduce the problem and got following:
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
File "<stdin>", line 5, in get_base
File "/usr/lib/python2.7/site-packages/dnf/base.py", line 314, in fill_sack
self._add_repo_to_sack(r)
File "/usr/lib/python2.7/site-packages/dnf/base.py", line 128, in _add_repo_to_sack
load_updateinfo=True)
IOError: can not read file /var/tmp/dnf-mhatina-q9t71bzx/mhatina-modularity-demo-c3d7dc7fad9d9682/repodata/repomd.xml: Too many open files
The memory was oscillating on the same level, it wasn't rising.
that means that either librepo doesn't close files properly.... I can still reproduce on fedora 26. I created a docker image to reproduce it on a clean system: docker run ludwigf/dnf-1461423 It will linearly increase memory usage for me: starting endless loop 85 mb memory, delta: 85 mb 142 mb memory, delta: 57 mb 191 mb memory, delta: 49 mb 252 mb memory, delta: 61 mb 298 mb memory, delta: 46 mb 353 mb memory, delta: 55 mb 407 mb memory, delta: 54 mb 461 mb memory, delta: 54 mb 512 mb memory, delta: 51 mb 562 mb memory, delta: 49 mb 618 mb memory, delta: 56 mb 671 mb memory, delta: 54 mb 723 mb memory, delta: 52 mb 778 mb memory, delta: 55 mb 827 mb memory, delta: 49 mb 882 mb memory, delta: 55 mb [...] The code used is still the same, just added some output to display memory usage after each base.close() Link to full code used: https://github.com/FlorianLudwig/dnf-1461423/blob/master/bug.py I have found first memory leak in base.read_all_repos(). Here is the patch (https://github.com/rpm-software-management/librepo/pull/118). But I believe that this is not the last problem. I create two additional patches that solve the issue: https://github.com/rpm-software-management/dnf/pull/943 https://github.com/rpm-software-management/libdnf/pull/337 Hope that it will solve the problem completly. dnf-2.7.4-1.fc26 libdnf-0.11.1-1.fc26 has been submitted as an update to Fedora 26. https://bodhi.fedoraproject.org/updates/FEDORA-2017-9f04c2c90f dnf-2.7.4-1.fc27 libdnf-0.11.1-1.fc27 has been submitted as an update to Fedora 27. https://bodhi.fedoraproject.org/updates/FEDORA-2017-113a221a3d dnf-2.7.4-1.fc26, libdnf-0.11.1-1.fc26 has been pushed to the Fedora 26 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-9f04c2c90f dnf-2.7.4-1.fc27, libdnf-0.11.1-1.fc27 has been pushed to the Fedora 27 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-113a221a3d Tested it on fc26 updates-testing. Looks good! dnf-2.7.4-1.fc26, libdnf-0.11.1-1.fc26 has been pushed to the Fedora 26 stable repository. If problems still persist, please make note of it in this bug report. dnf-2.7.4-1.fc27, libdnf-0.11.1-1.fc27 has been pushed to the Fedora 27 stable repository. If problems still persist, please make note of it in this bug report. |
Description of problem: using the API results in memory leak Version-Release number of selected component (if applicable): python2-dnf.noarch 1.1.10-6.fc25 How reproducible: ``` import dnf def get_base(): base = dnf.Base() base.conf.assumeyes = True base.read_all_repos() base.fill_sack(load_system_repo='auto') base.close() while True: get_base() ``` Actual results: Out of memory Expected results: Additional info: