Bug 1871992 - Possible regression between Python 3.9.0b5 and rc1: SystemError: <built-in function compile> returned NULL without setting an error
Summary: Possible regression between Python 3.9.0b5 and rc1: SystemError: <built-in fu...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: python3.9
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Victor Stinner
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-08-24 19:38 UTC by Miro Hrončok
Modified: 2020-09-26 00:14 UTC (History)
8 users (show)

Fixed In Version: python3.9-3.9.0~rc2-1.fc33
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-09-26 00:14:44 UTC
Type: Bug


Attachments (Terms of Use)
Workaround for FTBFS on Python 3.9 (479 bytes, patch)
2020-09-02 09:11 UTC, Petr Viktorin
no flags Details | Diff


Links
System ID Priority Status Summary Last Updated
Python 41631 None None None 2020-08-25 10:17:05 UTC

Description Miro Hrončok 2020-08-24 19:38:58 UTC
There is a weird failure when building mercurial:

$ mock -r fedora-rawhide-x86_64 init

[mercurial (master)]$ fedpkg --release master mockbuild -N
...
... installs python3.9-3.9.0~rc1-2.fc34 ...
...
make -C doc
make[1]: Entering directory '/builddir/build/BUILD/mercurial-5.4/doc'
/usr/bin/python3 gendoc.py "hg-ssh.8" > hg-ssh.8.txt.tmp
Traceback (most recent call last):
  File "/builddir/build/BUILD/mercurial-5.4/doc/gendoc.py", line 39, in <module>
    from mercurial.i18n import (
  File "/usr/lib64/python3.9/importlib/util.py", line 245, in __getattribute__
    self.__spec__.loader.exec_module(self)
SystemError: <built-in function compile> returned NULL without setting an error
make[1]: *** [Makefile:22: hg-ssh.8.txt] Error 1
make[1]: Leaving directory '/builddir/build/BUILD/mercurial-5.4/doc'
make: *** [Makefile:64: doc] Error 2
error: Bad exit status from /var/tmp/rpm-tmp.j1o0Ut (%build)


This did not happen with beta 5:
$ koji download-build --rpm python3-3.9.0~b5-2.fc33.x86_64
$ koji download-build --rpm python3-libs-3.9.0~b5-2.fc33.x86_64
$ koji download-build --rpm python3-devel-3.9.0~b5-2.fc33.x86_64
$ mock -r fedora-rawhide-x86_64 install ./python3*-3.9.0~b5-2.fc33.x86_64.rpm

[mercurial (master)]$ fedpkg --release master mockbuild -N --disablerepo=\*
(builds fine or dies on unrelated emacs error later in the build)

Comment 1 Miro Hrončok 2020-08-24 19:53:24 UTC
I can reproduce on Fedora 32 outside of mock:

$ fedpkg clone mercurial
$ cd mercurial
$ fedpkg prep
$ cd mercurial-5.4
$ cp -a hg hg3
$ PYTHON=/usr/bin/python3.9 make all
/usr/bin/python3.9 setup.py  build 
running build
running build_mo
running build_py
running build_ext
running build_scripts
make -C doc
make[1]: Entering directory '/home/churchyard/rpmbuild/fedora-scm/mercurial/mercurial-5.4/doc'
/usr/bin/python3.9 gendoc.py "hgrc.5" > hgrc.5.txt.tmp
Traceback (most recent call last):
  File "/home/churchyard/rpmbuild/fedora-scm/mercurial/mercurial-5.4/doc/gendoc.py", line 39, in <module>
    from mercurial.i18n import (
  File "/usr/lib64/python3.9/importlib/util.py", line 245, in __getattribute__
    self.__spec__.loader.exec_module(self)
SystemError: <built-in function compile> returned NULL without setting an error


This will make ti much easier to bisect, but I won't get to it today.

Comment 2 Miro Hrončok 2020-08-24 21:18:05 UTC
> but I won't get to it today.

Couldn't resist.

Git bisect says it is: https://github.com/python/cpython/commit/d2bea2636d5f0c2b196966315790af8e79c7bf82


Victor, please make a look.

Comment 3 Petr Stodulka 2020-08-25 09:30:48 UTC
Just leeting know that the same issue happens on F33 as well.

Comment 4 Victor Stinner 2020-08-25 10:17:05 UTC
I reported the issue upstream: https://bugs.python.org/issue41631

It's a Python 3.9 regression. I understand that only projects which monkey-patch the __import__() function, like Mercurial, are impacted.

Comment 5 Petr Viktorin 2020-08-25 11:48:23 UTC
Note that __import__'s docs do mention that it can be replaced: https://docs.python.org/3/library/functions.html#__import__
Since Python 3.3, doing so is strongly discouraged, but nevertheless it should be possible.

Comment 6 Petr Viktorin 2020-09-02 09:11:26 UTC
Created attachment 1713421 [details]
Workaround for FTBFS on Python 3.9

I was able to build with this one-line patch.
I'm not sure whether to call this a workaround or a fix -- that depends on how the CPython issue is solved.

Comment 7 Petr Stodulka 2020-09-02 09:17:44 UTC
Thanks Petr! We will try it.

Comment 8 Petr Viktorin 2020-09-02 09:19:17 UTC
Sorry; I meant to post that comment on the Mercurial bug.

Here's a PR for mercurial dist-git: https://src.fedoraproject.org/rpms/mercurial/pull-request/12

Comment 9 Fedora Update System 2020-09-04 22:17:13 UTC
FEDORA-2020-bc1902e34e has been submitted as an update to Fedora 33. https://bodhi.fedoraproject.org/updates/FEDORA-2020-bc1902e34e

Comment 10 Fedora Update System 2020-09-05 21:39:39 UTC
FEDORA-2020-bc1902e34e has been pushed to the Fedora 33 testing repository.
In short time you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2020-bc1902e34e`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-bc1902e34e

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 11 Petr Viktorin 2020-09-08 11:31:09 UTC
Hello Petr,
Could you offer this patch to Mercurial upstream? Possibly with a link to https://bugs.python.org/issue41631 instead of Bugzilla.
I'm growing more and more convinced that the patch is the right way to go for Mercurial, and I'd like to hear their opinion. But I'm not very familiar with their bug-reporting process.

Comment 12 Petr Stodulka 2020-09-08 13:09:53 UTC
Hi Petr,
thanks for reminder. Reported: https://bz.mercurial-scm.org/show_bug.cgi?id=6407

Comment 13 Miro Hrončok 2020-09-16 12:20:44 UTC
This is fixed in Python 3.9.0rc2 (to be released this week).

Comment 14 Fedora Update System 2020-09-25 13:20:39 UTC
FEDORA-2020-78bb031321 has been submitted as an update to Fedora 33. https://bodhi.fedoraproject.org/updates/FEDORA-2020-78bb031321

Comment 15 Fedora Update System 2020-09-25 16:47:54 UTC
FEDORA-2020-bc1902e34e has been pushed to the Fedora 33 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 16 Fedora Update System 2020-09-26 00:14:44 UTC
FEDORA-2020-78bb031321 has been pushed to the Fedora 33 stable repository.
If problem still persists, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.