Bug 1896438

Summary: pylint became incredibly slow in F33
Product: [Fedora] Fedora Reporter: Bastien Nocera <bnocera>
Component: pylintAssignee: Gwyn Ciesla <gwync>
Status: CLOSED EOL QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 33CC: gwync, orion, philip.wyett
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-11-30 18:17:08 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Bastien Nocera 2020-11-10 14:40:27 UTC
This build ran with Fedora 32, and python3-pylint-2.4.4-3.fc32.noarch:
https://gitlab.gnome.org/GNOME/totem/-/jobs/953740

The pylint part of the tests took:
5/8 pylint-dbusservice   OK             2.24s
6/8 pylint-opensubtitles OK             24.51s
7/8 pylint-pythonconsole OK             22.69s
8/8 pylint-samplepython  OK             1.49s

On my F33 desktop system now (and on the F33 CI runners):
 5/10 pylint-dbusservice                   OK             4.45s
 6/10 pylint-opensubtitlesopensubtitles.py TIMEOUT        120.13s
 7/10 pylint-opensubtitleshash.py          OK             6.10s
 8/10 pylint-pythonconsolepythonconsole.py TIMEOUT        120.13s
 9/10 pylint-pythonconsoleconsole.py       OK             56.24s
10/10 pylint-samplepython                  OK             2.92s

You can fetch the opensubtitles.py file here for comparison:
https://gitlab.gnome.org/GNOME/totem/-/tree/master/src/plugins/opensubtitles

F32:
$ rpm -q pylint python3-pylint
pylint-2.4.4-3.fc32.noarch
python3-pylint-2.4.4-3.fc32.noarch
$ time /usr/bin/pylint-3 -d C0111 -d W0511 -d F0401 -d C0326 -d bad-continuation opensubtitles.py

--------------------------------------------------------------------
Your code has been rated at 10.00/10 (previous run: 10.00/10, +0.00)


real	0m1.427s
user	0m1.327s
sys	0m0.059s

F33:
$ rpm -q pylint python3-pylint
pylint-2.6.0-1.fc33.noarch
python3-pylint-2.6.0-1.fc33.noarch
$ time /usr/bin/pylint-3 -d C0111 -d W0511 -d F0401 -d C0326 -d bad-continuation opensubtitles.py

--------------------------------------------------------------------
Your code has been rated at 10.00/10 (previous run: 10.00/10, +0.00)


real	0m12.633s
user	0m12.258s
sys	0m0.288s

Comment 1 Bastien Nocera 2020-11-10 15:09:00 UTC
On a Fedora 32 machine, upgrading pylint to the current rawhide version (after recompilation), only slows the above single check by half.
Upgrading python3-isort to the current rawhide version (after recompilation), doesn't make any changes[1] to the speed.

[1]: There were some known performance regressions in isort in the past:
https://github.com/PyCQA/isort/issues/848

Comment 2 Gwyn Ciesla 2020-11-10 15:28:41 UTC
Looks like upgrading isort is a good thing to do, I'll get that out ASAP.

Comment 3 Fedora Update System 2020-11-10 15:36:22 UTC
FEDORA-2020-d124f335ca has been submitted as an update to Fedora 33. https://bodhi.fedoraproject.org/updates/FEDORA-2020-d124f335ca

Comment 4 Bastien Nocera 2020-11-10 17:44:15 UTC
(In reply to Gwyn Ciesla from comment #2)
> Looks like upgrading isort is a good thing to do, I'll get that out ASAP.

It doesn't make any difference to the problems though.

Comment 5 Gwyn Ciesla 2020-11-10 17:47:23 UTC
Then I misread your comments; I'll keep an eye on the upstream bug.

Comment 6 Bastien Nocera 2020-11-10 18:14:45 UTC
(In reply to Gwyn Ciesla from comment #5)
> Then I misread your comments; I'll keep an eye on the upstream bug.

To clarify, I said that I updated isort because it had had some performance problems in the past.

The problem is easy enough to reproduce (you can use toolbox to create an F32 container to compare),
but doesn't look like it's in pylint itself, but rather one of its many dependencies, maybe even
in the Python core itself. I ran out of time to reproduce to investigate the problem further.

Comment 7 Fedora Update System 2020-11-11 02:20:10 UTC
FEDORA-2020-d124f335ca has been pushed to the Fedora 33 testing repository.
In short time you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2020-d124f335ca`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-d124f335ca

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 8 Fedora Update System 2020-11-19 01:28:46 UTC
FEDORA-2020-d124f335ca has been pushed to the Fedora 33 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 9 Bastien Nocera 2021-03-17 13:19:03 UTC
(In reply to Fedora Update System from comment #8)
> FEDORA-2020-d124f335ca has been pushed to the Fedora 33 stable repository.
> If problem still persists, please make note of it in this bug report.

The problem does persist, for what it's worth.

python3-isort-5.6.4-1.fc33.noarch
python3-3.9.2-1.fc33.x86_64

$ meson test
ninja: Entering directory `/home/hadess/.cache/jhbuild/build/totem'
ninja: no work to do.
1/8 validate-desktop            OK               0.01s
2/8 validate-gschema            OK               0.01s
3/8 validate-appdata            OK               0.04s
4/8 test-totem                  OK               0.09s
5/8 pylint-samplepython         OK               0.95s
6/8 pylint-dbusservice          OK               1.53s
7/8 pylint-opensubtitles        TIMEOUT        120.04s   killed by signal 15 SIGTERM
>>> /usr/bin/pylint-3 -d C0111 -d W0511 -d F0401 -d C0326 -d bad-continuation /home/hadess/.cache/jhbuild/build/totem/../../../../Projects/jhbuild/totem/src/plugins/opensubtitles/opensubtitles.py /home/hadess/.cache/jhbuild/build/totem/../../../../Projects/jhbuild/totem/src/plugins/opensubtitles/hash.py

8/8 pylint-pythonconsole        TIMEOUT        120.04s   killed by signal 15 SIGTERM
>>> /usr/bin/pylint-3 -d C0111 -d W0511 -d F0401 -d C0326 /home/hadess/.cache/jhbuild/build/totem/../../../../Projects/jhbuild/totem/src/plugins/pythonconsole/pythonconsole.py /home/hadess/.cache/jhbuild/build/totem/../../../../Projects/jhbuild/totem/src/plugins/pythonconsole/console.py



Ok:                 6   
Expected Fail:      0   
Fail:               0   
Unexpected Pass:    0   
Skipped:            0   
Timeout:            2   

Full log written to /home/hadess/.cache/jhbuild/build/totem/meson-logs/testlog.txt

Comment 10 Bastien Nocera 2021-03-17 16:20:51 UTC
Filed upstream: https://github.com/PyCQA/pylint/issues/4243

Comment 11 Ben Cotton 2021-11-04 14:54:24 UTC
This message is a reminder that Fedora 33 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora 33 on 2021-11-30.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
Fedora 'version' of '33'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 33 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 12 Ben Cotton 2021-11-04 15:52:58 UTC
This message is a reminder that Fedora 33 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora 33 on 2021-11-30.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
Fedora 'version' of '33'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 33 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 13 Ben Cotton 2021-11-30 18:17:08 UTC
Fedora 33 changed to end-of-life (EOL) status on 2021-11-30. Fedora 33 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.