Description of problem: A Python interpreter on Fedora imports lots of additional modules and packages. On my Fedora system a fresh interpreter has 104 entries in sys.modules. For comparision an upstream Python 3.9 process as well as Python 3.9 in podman container image only has 33 entries. Any additional imports slows down startup and has a negative impact on general performance. Especially command line tools are affected. The largest culprict is python3-abrt-addon. It increases the amount of loaded modules by 57 (!) to 90 total. Version-Release number of selected component (if applicable): python3-3.9.7-1.fc34.x86_64 python3-abrt-addon-2.14.6-3.fc34.noarch How reproducible: always Steps to Reproduce: 1. python3 -c "import sys; print(len(sys.modules))" 2. python3 -c "import sys; print(sorted(sys.modules))" Actual results: 104 ['__future__', '__main__', '_abc', '_bootlocale', '_codecs', '_collections', '_collections_abc', '_datetime', '_frozen_importlib', '_frozen_importlib_external', '_functools', '_heapq', '_imp', '_io', '_locale', '_operator', '_posixsubprocess', '_signal', '_sitebuiltins', '_sre', '_stat', '_string', '_thread', '_uuid', '_warnings', '_weakref', '_weakrefset', 'abc', 'abrt_exception_handler3', 'atexit', 'bkr', 'builtins', 'codecs', 'collections', 'collections.abc', 'contextlib', 'copyreg', 'datetime', 'encodings', 'encodings.aliases', 'encodings.latin_1', 'encodings.utf_8', 'enum', 'errno', 'functools', 'genericpath', 'google', 'grp', 'heapq', 'importlib', 'importlib._bootstrap', 'importlib._bootstrap_external', 'importlib.abc', 'importlib.machinery', 'importlib.util', 'io', 'itertools', 'keyword', 'linecache', 'logging', 'marshal', 'math', 'operator', 'os', 'os.path', 'paste', 'platform', 'posix', 'posixpath', 'pwd', 're', 'reprlib', 'ruamel', 'select', 'selectors', 'signal', 'site', 'sphinxcontrib', 'sre_compile', 'sre_constants', 'sre_parse', 'stat', 'string', 'subprocess', 'sys', 'syslog', 'systemd', 'systemd._journal', 'systemd._reader', 'systemd.id128', 'systemd.journal', 'threading', 'time', 'token', 'tokenize', 'traceback', 'types', 'typing', 'typing.io', 'typing.re', 'uuid', 'warnings', 'weakref', 'zipimport'] Expected results: I expect a smaller number of modules that gets import by default Additional info: $ podman run -ti fedora:34 # python3 -c "import sys; print(len(sys.modules))" 33 # python3 -c "import sys; print(sorted(sys.modules))" ['__main__', '_abc', '_codecs', '_collections_abc', '_frozen_importlib', '_frozen_importlib_external', '_imp', '_io', '_signal', '_sitebuiltins', '_stat', '_thread', '_warnings', '_weakref', 'abc', 'builtins', 'codecs', 'encodings', 'encodings.aliases', 'encodings.latin_1', 'encodings.utf_8', 'genericpath', 'io', 'marshal', 'os', 'os.path', 'posix', 'posixpath', 'site', 'stat', 'sys', 'time', 'zipimport'] # dnf install -y python3-abrt-addon # python3 -c "import sys; print(len(sys.modules))" 90 # python3 -c "import sys; print(sorted(sys.modules))" ['__future__', '__main__', '_abc', '_bootlocale', '_codecs', '_collections', '_collections_abc', '_datetime', '_frozen_importlib', '_frozen_importlib_external', '_functools', '_heapq', '_imp', '_io', '_locale', '_operator', '_posixsubprocess', '_signal', '_sitebuiltins', '_sre', '_stat', '_string', '_thread', '_uuid', '_warnings', '_weakref', '_weakrefset', 'abc', 'abrt_exception_handler3', 'atexit', 'builtins', 'codecs', 'collections', 'collections.abc', 'contextlib', 'copyreg', 'datetime', 'encodings', 'encodings.aliases', 'encodings.latin_1', 'encodings.utf_8', 'enum', 'errno', 'functools', 'genericpath', 'grp', 'heapq', 'io', 'itertools', 'keyword', 'linecache', 'logging', 'marshal', 'math', 'operator', 'os', 'os.path', 'platform', 'posix', 'posixpath', 'pwd', 're', 'reprlib', 'select', 'selectors', 'signal', 'site', 'sre_compile', 'sre_constants', 'sre_parse', 'stat', 'string', 'subprocess', 'sys', 'syslog', 'systemd', 'systemd._journal', 'systemd._reader', 'systemd.id128', 'systemd.journal', 'threading', 'time', 'token', 'tokenize', 'traceback', 'types', 'uuid', 'warnings', 'weakref', 'zipimport']
Changing the component to abrt to get feedback from the maintainers.
/usr/lib/python3.9/site-packages/abrt3.pth has: import abrt_exception_handler3 /usr/lib/python3.9/site-packages/abrt_exception_handler3.py has: import sys import os from systemd import journal I think ideally, abrt_exception_handler3 could lazy import: Even this helps a lot: def syslog(msg): """Log message to system logger (journal)""" from systemd import journal journal.send(msg) On rawhide: - no python3-abrt-addon installed: 32 modules - python3-abrt-addon installed: 85 modules - python3-abrt-addon with the modification from above: 33 modules The plugin could even do the right thing and module-level importing os, but that currently does not make any difference. My diff: @@ -21,13 +21,11 @@ """ import sys -import os -from systemd import journal def syslog(msg): """Log message to system logger (journal)""" - + from systemd import journal journal.send(msg) @@ -68,6 +66,8 @@ def write_dump(tb_text, tb): + import os + if sys.argv[0][0] == "/": executable = os.path.abspath(sys.argv[0]) else: @@ -117,7 +117,7 @@ # Restore original exception handler sys.excepthook = sys.__excepthook__ # pylint: disable-msg=E1101 - import errno + import errno, os # Ignore Ctrl-C # SystemExit rhbz#636913 -> this exception is not an error
Christian, as for the other 14 modules imported on your workstation: 'bkr', 'google', 'importlib', 'importlib._bootstrap', 'importlib._bootstrap_external', 'importlib.abc', 'importlib.machinery', 'importlib.util', 'paste', 'ruamel', 'sphinxcontrib', 'typing', 'typing.io', 'typing.re' I believe those are caused by some old-school namespace packages with files like /usr/lib/python3.9/site-packages/ruamel.yaml-0.16.6-py3.9-nspkg.pth: import sys, types, os;has_mfs = sys.version_info > (3, 5);p = os.path.join(sys._getframe(1).f_locals['sitedir'], *('ruamel',));importlib = has_mfs and __import__('importlib.util');has_mfs and __import__('importlib.machinery');m = has_mfs and sys.modules.setdefault('ruamel', importlib.util.module_from_spec(importlib.machinery.PathFinder.find_spec('ruamel', [os.path.dirname(p)])));m = m or sys.modules.setdefault('ruamel', types.ModuleType('ruamel'));mp = (m or []) and m.__dict__.setdefault('__path__',[]);(p not in mp) and mp.append(p) Not much we can do about those except talking to upstream not to do this anymore.
> The plugin could even do the right thing and module-level importing os... Correction: And **stop** module-level importing os...
Upstream PR: https://github.com/abrt/abrt/pull/1580
Pull request merged. The change will appear in the next release of abrt.
(In reply to Matej Grabovsky from comment #6) > The change will appear in the next release of abrt. Will that happen in time for Fedora 35 final freeze? If not, I'd rather see that backported.
OK, I'll push a backport for active releases.
Thanks!
FEDORA-2021-db079c289a has been submitted as an update to Fedora 34. https://bodhi.fedoraproject.org/updates/FEDORA-2021-db079c289a
FEDORA-2021-db079c289a has been pushed to the Fedora 34 testing repository. Soon you'll be able to install the update with the following command: `sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2021-db079c289a` You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2021-db079c289a See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.
FEDORA-2021-db079c289a has been pushed to the Fedora 34 stable repository. If problem still persists, please make note of it in this bug report.
This has still not reached F35. I'd really like to see it on the media.
FEDORA-2021-418a00501f has been submitted as an update to Fedora 35. https://bodhi.fedoraproject.org/updates/FEDORA-2021-418a00501f
+3 in https://pagure.io/fedora-qa/blocker-review/issue/519 , marking accepted.
FEDORA-2021-418a00501f has been pushed to the Fedora 35 stable repository. If problem still persists, please make note of it in this bug report.