Bug 1817677

Summary: python-cartopy fails to build with Python 3.9 (doctest/asyncio REPL regression in Python)
Product: [Fedora] Fedora Reporter: Miro Hrončok <mhroncok>
Component: python-cartopyAssignee: Elliott Sales de Andrade <quantum.analyst>
Status: CLOSED NOTABUG QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: rawhideCC: cstratak, josdekloe, mhroncok, mplch, python-sig, quantum.analyst, vstinner
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-05-18 15:32:37 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1785415    

Description Miro Hrončok 2020-03-26 18:32:55 UTC
python-cartopy fails to build with Python 3.9.0a5.

=================================== FAILURES ===================================
_______________________ [doctest] cartopy.io.shapereader _______________________
022 
023     >>> import cartopy.io.shapereader as shapereader
024     >>> filename = shapereader.natural_earth(resolution='110m',
025     ...                                      category='physical',
026     ...                                      name='geography_regions_points')
027     >>> reader = shapereader.Reader(filename)
028     >>> len(reader)
029     3
030     >>> records = list(reader.records())
031     >>> print(', '.join(str(r) for r in sorted(records[0].attributes.keys())))
Expected:
    comment, ... name, name_alt, ... region, ...
Got nothing

/builddir/build/BUILDROOT/python-cartopy-0.18.0~b1-1.fc33.x86_64/usr/lib64/python3.9/site-packages/cartopy/io/shapereader.py:31: DocTestFailure
_____________________________ test_transform_point _____________________________

    def test_transform_point():
        """
        Mostly tests the workaround for a specific problem.
        Problem report in: https://github.com/SciTools/cartopy/issues/23
        Fix covered in: https://github.com/SciTools/cartopy/pull/277
        """
    
        # this way has always worked
        result = _CRS_ROB.transform_point(35.0, 70.0, _CRS_PC)
>       assert_array_almost_equal(result, (2376187.27182751, 7275317.81573085),
                                  _TOL)
E       AssertionError: 
E       Arrays are not almost equal to 7 decimals
E       
E       Mismatched elements: 2 / 2 (100%)
E       Max absolute difference: 0.30056718178093433
E       Max relative difference: 4.131327172141421e-08
E        x: array([ 2376187.2182271,  7275318.116298 ])
E        y: array([ 2376187.2718275,  7275317.8157309])

../../BUILDROOT/python-cartopy-0.18.0~b1-1.fc33.x86_64/usr/lib64/python3.9/site-packages/cartopy/tests/crs/test_robinson.py:126: AssertionError
____________________________ test_transform_points _____________________________

    def test_transform_points():
        """
        Mostly tests the workaround for a specific problem.
        Problem report in: https://github.com/SciTools/cartopy/issues/23
        Fix covered in: https://github.com/SciTools/cartopy/pull/277
        """
    
        # these always worked
        result = _CRS_ROB.transform_points(_CRS_PC,
                                           np.array([35.0]),
                                           np.array([70.0]))
>       assert_array_almost_equal(result,
                                  [[2376187.27182751, 7275317.81573085, 0]], _TOL)
E       AssertionError: 
E       Arrays are not almost equal to 7 decimals
E       
E       Mismatched elements: 2 / 3 (66.7%)
E       Max absolute difference: 0.30056718178093433
E       Max relative difference: 4.131327172141421e-08
E        x: array([[ 2376187.2182271,  7275318.116298 ,        0.       ]])
E        y: array([[ 2376187.2718275,  7275317.8157309,        0.       ]])

../../BUILDROOT/python-cartopy-0.18.0~b1-1.fc33.x86_64/usr/lib64/python3.9/site-packages/cartopy/tests/crs/test_robinson.py:149: AssertionError

For the build logs, see:
https://copr-be.cloud.fedoraproject.org/results/@python/python3.9/fedora-rawhide-x86_64/01321079-python-cartopy/

For all our attempts to build python-cartopy with Python 3.9, see:
https://copr.fedorainfracloud.org/coprs/g/python/python3.9/package/python-cartopy/

Testing and mass rebuild of packages is happening in copr. You can follow these instructions to test locally in mock if your package builds with Python 3.9:
https://copr.fedorainfracloud.org/coprs/g/python/python3.9/

Let us know here if you have any questions.

Python 3.9 will be included in Fedora 33. To make that update smoother, we're building Fedora packages with early pre-releases of Python 3.9.
A build failure prevents us from testing all dependent packages (transitive [Build]Requires), so if this package is required a lot, it's important for us to get it fixed soon.
We'd appreciate help from the people who know this package best, but if you don't want to work on this now, let us know so we can try to work around it on our side.

Comment 1 Elliott Sales de Andrade 2020-04-13 10:36:06 UTC
With 0.18.0b2, there is just one failure:

_______________________ [doctest] cartopy.io.shapereader _______________________
022 
023     >>> import cartopy.io.shapereader as shapereader
024     >>> filename = shapereader.natural_earth(resolution='110m',
025     ...                                      category='physical',
026     ...                                      name='geography_regions_points')
027     >>> reader = shapereader.Reader(filename)
028     >>> len(reader)
029     3
030     >>> records = list(reader.records())
031     >>> print(', '.join(str(r) for r in sorted(records[0].attributes.keys())))
Expected:
    comment, ... name, name_alt, ... region, ...
Got nothing
/builddir/build/BUILDROOT/python-cartopy-0.18.0~b2-1.fc33.x86_64/usr/lib64/python3.9/site-packages/cartopy/io/shapereader.py:31: DocTestFailure

I also see this warning:
io/shapereader.py::cartopy.io.shapereader
  /usr/lib64/python3.9/doctest.py:1329: RuntimeWarning: coroutine '<module>' was never awaited
    exec(compile(example.source, filename, "single",

Is this a bug in Python 3.9, or pytest?

Comment 2 Miro Hrončok 2020-04-18 10:38:02 UTC
> Is this a bug in Python 3.9, or pytest?

It is a Python 3.9 regression. Thanks for finding the warning.

See https://bugs.python.org/issue39562#msg365311 and bz1817710.

Comment 3 Elliott Sales de Andrade 2020-05-01 06:47:43 UTC
Did the change get backported? Now it's failing in Rawhide with 3.8 https://koji.fedoraproject.org/koji/taskinfo?taskID=43968331

Comment 4 Miro Hrončok 2020-05-01 21:43:08 UTC
Will check. It looks like it was.

Comment 5 Miro Hrončok 2020-05-01 21:47:01 UTC
Looks like:

 - the problematic commit was backported to 3.8 branch when it was introduced and the fix only appeared in the master branch (3.9)
 - in 3.8.3rc1, the problem was released. the fix was now backported as well and should be released in 3.8.3 (I'll try to make sure)
 - on 3.9 front, we also have 3.9.0a6 now that might have the fix

Comment 6 Miro Hrončok 2020-05-18 15:32:37 UTC
This is fixed in Python 3.9 and 3.8. Thanks for your patience.