Bug 1817710 - python-gmpy2 fails to build with Python 3.9 (doctest/asyncio REPL regression in Python)
Summary: python-gmpy2 fails to build with Python 3.9 (doctest/asyncio REPL regression ...
Alias: None
Product: Fedora
Classification: Fedora
Component: python-gmpy2
Version: rawhide
Hardware: Unspecified
OS: Unspecified
Target Milestone: ---
Assignee: Jerry James
QA Contact: Fedora Extras Quality Assurance
Depends On:
Blocks: PYTHON39
TreeView+ depends on / blocked
Reported: 2020-03-26 19:21 UTC by Miro Hrončok
Modified: 2020-05-26 16:58 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Last Closed: 2020-05-26 16:58:19 UTC
Type: Bug

Attachments (Terms of Use)

System ID Private Priority Status Summary Last Updated
Python 39562 0 None None None 2020-03-30 13:19:51 UTC

Description Miro Hrončok 2020-03-26 19:21:05 UTC
python-gmpy2 fails to build with Python 3.9.0a5.

There are failures like this one:

File "/builddir/build/BUILD/gmpy-gmpy2-2.1.0b4/test/test_gmpy2_xmpz_misc.txt", line 91, in test_gmpy2_xmpz_misc.txt
Failed example:
    [b for b in iter]
Differences (ndiff with -expected +actual):
    - [False, False, False, False, True]

For the build logs, see:

For all our attempts to build python-gmpy2 with Python 3.9, see:

Testing and mass rebuild of packages is happening in copr. You can follow these instructions to test locally in mock if your package builds with Python 3.9:

Let us know here if you have any questions.

Python 3.9 will be included in Fedora 33. To make that update smoother, we're building Fedora packages with early pre-releases of Python 3.9.
A build failure prevents us from testing all dependent packages (transitive [Build]Requires), so if this package is required a lot, it's important for us to get it fixed soon.
We'd appreciate help from the people who know this package best, but if you don't want to work on this now, let us know so we can try to work around it on our side.

Comment 1 Jerry James 2020-03-27 16:24:15 UTC
If I start up python and enter in the lines of the tests by hand, I get correct answers.  I note that the build log shows that the tests don't generate any output at all after a certain point.  I also see this at the top of the %check log:

+ /usr/bin/python3 test/runtests.py
/usr/lib64/python3.9/doctest.py:1329: RuntimeWarning: coroutine '<module>' was never awaited
  exec(compile(example.source, filename, "single",

I think the tests are computing the correct values, and the problem is that the output of the tests is not being collected properly.  I admit that I do not understand the meaning of "coroutine '<module>' was never awaited", but I suspect we are seeing badly managed asynchrony here.

Has the running of doctests become asynchronous in the latest 3.9 alpha?

Comment 2 Miro Hrončok 2020-03-27 18:14:40 UTC
> Has the running of doctests become asynchronous in the latest 3.9 alpha?

I am not aware of anything like that.

Comment 3 Jerry James 2020-03-29 20:25:41 UTC
The warning about the coroutine that was never awaited points to the line in doctest.py where the test code is invoked.  Just a few lines later, the test output is gathered and it is incomplete, as though the test is still running.

Try this.  Run python3 under gdb:

(gdb) break compile.c:5842 if c->u->u_ste->ste_generator != 0 || c->u->u_ste->ste_coroutine != 0
Breakpoint 1 at 0x7ffff7c0aae3: file /usr/src/debug/python3-3.9.0~a5-1.fc33.x86_64/Python/compile.c, line 5842.
(gdb) run test/runtests.py

I then see the same block of passing test output we see in the copr build (i.e., up through test_gmpy2_xmpz_inplace), and then the breakpoint triggers.

Breakpoint 1, compute_code_flags (c=0x7fffffffd590, c=0x7fffffffd590)
    at /usr/src/debug/python3-3.9.0~a5-1.fc33.x86_64/Python/compile.c:5842
5842	    flags = compute_code_flags(c);
(gdb) print *c->u->u_ste
$48 = {ob_base = {ob_refcnt = 2, ob_type = 0x7ffff7dd4100 <PySTEntry_Type>}, 
  ste_id = 93824994774712, ste_symbols = {'iter': 6160}, ste_name = 'top', 
  ste_varnames = [], 
  ste_children = [<symtable entry at remote 0x7ffff6b7e880>], 
  ste_directives = 0x0, ste_type = ModuleBlock, ste_nested = 0, ste_free = 0, 
  ste_child_free = 0, ste_generator = 0, ste_coroutine = 1, 
  ste_comprehension = 0, ste_varargs = 0, ste_varkeywords = 0, 
  ste_returns_value = 0, ste_needs_class_closure = 0, 
  ste_comp_iter_target = 0, ste_comp_iter_expr = 0, ste_lineno = 0, 
  ste_col_offset = 0, ste_opt_lineno = 0, ste_opt_col_offset = 0, 
  ste_table = 0x7ffff6b80530}
(gdb) print c->c_flags->cf_flags
$49 = 76032

76032 == 0x12900, where 0x2000 == PyCF_ALLOW_TOP_LEVEL_AWAIT, so IS_TOP_LEVEL_AWAIT(c) is true.  Therefore, the conditional on line 5762 is true, and the CO_COROUTINE flag is set.  The test runs as a coroutine, but nothing actually calls await, so it is still producing output when doctest.py gathers its output, resulting in the observed test failures.

So what set the ste_coroutine and PyCF_ALLOW_TOP_LEVEL_AWAIT flags?

Comment 4 Victor Stinner 2020-03-30 13:19:51 UTC
It's a regression introduced between alpha 3 and alpha 4 of Python 3.9: https://bugs.python.org/issue39562#msg365311

Comment 5 Victor Stinner 2020-03-30 13:24:22 UTC
I found the bug using git bisect in CPython source code:

$ wget https://files.pythonhosted.org/packages/90/f4/9a2e384b325b69bc5827b9a6510a8fb4a51698c915c06a3f25a86458892a/gmpy2-2.0.8.zip
$ unzip gmpy2-2.0.8.zip -d .

$ cat > run.sh << EOF
set -e -x
cd ../gmpy2-2.0.8/
rm -rf env
../master/python -m venv --without-pip env
env/bin/python setup.py install
rm -rf build
env/bin/python test/runtests.py

$ cd master # my checkout of the Python master branch
$ git bisect reset
$ git bisect start
$ git checkout master
$ git clean -fdx && ./configure --with-pydebug CFLAGS="-O0" --cache-file=../python-config.cache && make && ../run.sh

# tests fail

$ git bisect bad

$ git log --before=2020-03-01
# pick a commit before March
$ git checkout 9052f7a41b90f2d34011c8da68f9a4facebc8a97
$ git clean -fdx && ./configure --with-pydebug CFLAGS="-O0" --cache-file=../python-config.cache && make && ../run.sh

# tests pass

$ git bisect good

... and repeat until git bisect completes.

Comment 6 Jerry James 2020-03-31 20:14:59 UTC
Good find, Victor!  Thanks for tracking that down.

Comment 7 Miro Hrončok 2020-05-25 13:01:46 UTC
This comment is mass posted to all bugs blocking the Python 3.9 tracker, sorry if it is not 100 % relevant. When in doubt, please ask.

The Python 3.9 rebuild is in progress in a Koji side tag.

If you fix this bug, please don't rebuild the package in regular rawhide, but do it in the side tag with:

    $ fedpkg build --target=f33-python

The rebuild is progressing slowly and it is possible this package won't have all the required build dependencies yet. If that's the case, please just leave the fix committed and pushed and we will eventually rebuild it for you.

You are not asked to go and try rebuild all the missing dependencies yourself. If you know there is a bootstrap loop in the dependencies, let me know and we can untangle it together.

If you want to test your fix or reproduce the failure, you can still use the Copr repo mentioned in the initial comment of this bug: https://copr.fedorainfracloud.org/coprs/g/python/python3.9/

Comment 8 Jerry James 2020-05-26 16:58:19 UTC
The python-gmpy2 package has been rebuilt successfully into the side tag, so I presume this bug has been fixed.  Closing.

Note You need to log in before you can comment on or make changes to this bug.