Bug 1936839 - python-astropy fails to build with Python 3.10: ValueError: solarx_unit is not a valid unit.
Summary: python-astropy fails to build with Python 3.10: ValueError: solarx_unit is no...
Keywords:
Status: CLOSED RAWHIDE
Alias: None
Product: Fedora
Classification: Fedora
Component: python-astropy
Version: rawhide
Hardware: Unspecified
OS: Unspecified
high
unspecified
Target Milestone: ---
Assignee: Sergio Pascual
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
: 1968898 (view as bug list)
Depends On: 1968818 1969126
Blocks: PYTHON3.10 F35FTBFS F35FailsToInstall 1968765 1968778 1968807 1968896 1968897 1968899 1968901 1968999 1969080 1969106 1969114 1969118 1969198
TreeView+ depends on / blocked
 
Reported: 2021-03-09 10:38 UTC by Tomáš Hrnčiar
Modified: 2021-06-17 19:46 UTC (History)
6 users (show)

Fixed In Version: python-astropy-4.2.1-3.fc35
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-06-17 19:46:42 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github astropy astropy issues 11821 0 None open Test faillures with python 3.10.0a6 2021-06-08 14:24:18 UTC

Description Tomáš Hrnčiar 2021-03-09 10:38:17 UTC
python-astropy fails to build with Python 3.10.0a6.

cls = <class 'astropy.units.format.generic.Generic'>, s = 'solarx_unit'
detailed_exception = True

    @classmethod
    def _parse_unit(cls, s, detailed_exception=True):
        registry = core.get_current_unit_registry().registry
        if s == '%':
            return registry['percent']
    
        if not _is_ascii(s):
            if s[0] == '\N{MICRO SIGN}':
                s = 'u' + s[1:]
            if s[-1] == '\N{GREEK CAPITAL LETTER OMEGA}':
                s = s[:-1] + 'Ohm'
            elif s[-1] == '\N{LATIN CAPITAL LETTER A WITH RING ABOVE}':
                s = s[:-1] + 'Angstrom'
    
        if s in registry:
            return registry[s]
    
        if detailed_exception:
>           raise ValueError(
                '{} is not a valid unit. {}'.format(
                    s, did_you_mean(s, registry)))
E           ValueError: solarx_unit is not a valid unit.

For the build logs, see:
https://copr-be.cloud.fedoraproject.org/results/@python/python3.10/fedora-rawhide-x86_64/02054143-python-astropy/

For all our attempts to build python-astropy with Python 3.10, see:
https://copr.fedorainfracloud.org/coprs/g/python/python3.10/package/python-astropy/

Testing and mass rebuild of packages is happening in copr. You can follow these instructions to test locally in mock if your package builds with Python 3.10:
https://copr.fedorainfracloud.org/coprs/g/python/python3.10/

Let us know here if you have any questions.

Python 3.10 will be included in Fedora 35. To make that update smoother, we're building Fedora packages with early pre-releases of Python 3.10.
A build failure prevents us from testing all dependent packages (transitive [Build]Requires), so if this package is required a lot, it's important for us to get it fixed soon.
We'd appreciate help from the people who know this package best, but if you don't want to work on this now, let us know so we can try to work around it on our side.

Comment 1 Tomáš Hrnčiar 2021-03-30 09:15:28 UTC
Hello, is there any update on this? Python 3.10 will be the main Python on Fedora 35 and we are already testing packages with it. I am setting priority to high because python-astropy is blocking plenty of packages we cannot test without it.

├── python-astropy
│   ├── astrometry
│   ├── python-astroML
│   ├── python-astroplan
│   ├── python-astroscrappy
│   │   └── python-ccdproc
│   ├── python-healpy
│   │   ├── python-astropy-healpix
│   │   │   └── python-reproject
│   │   │       └── APLpy
│   │   └── python-pymoc
│   ├── python-photutils
│   ├── python-pyshtools
│   ├── python-pyvo
│   │   └── python-astroquery
│   └── veusz

Comment 2 Sergio Pascual 2021-04-04 16:45:24 UTC
The errors area in the unit parser:

astropy/units/tests/test_quantity_annotations.py FFFFFFFFFFFFFFFFFFFFFFF [ 82%]
FFFFF                                                                    [ 82%]
astropy/units/tests/test_quantity_array_methods.py ..................... [ 82%]
......................................                                   [ 82%]
astropy/units/tests/test_quantity_decorator.py ..F.F........F.F........F [ 82%]
.F........F.F........F.F........F.F........F.F........F.F........F.F.... [ 83%]
....F.F........F.F........F.F........F.F........F.F........F.F........F. [ 83%]
F........F.F........F.F........F.F........F.F........F.F........F.F..... [ 84%]
...F.F........F.F........F.F........F.F........F.F........F.F........F.F [ 84%]
........F.F........F.F........F.F........F.F........F.F........F.F...... [ 85%]
..F.F..........F.F........F.F........F.F........F.F........F.F........F. [ 85%]
F..........F.F........F.F........F.F........F.F........F.F........F.F... [ 86%]
.......................F..........

I'm going to start building 4.2.1 to see if it helps

Comment 3 Tomáš Hrnčiar 2021-05-13 06:34:07 UTC
I've tried to build 4.2.1, most of the errors disappeared, but there are still some remaining ones. You can find full logs here: https://copr.fedorainfracloud.org/coprs/g/python/python3.10/build/2183681/

=================================== FAILURES ===================================
___________________ TestFileFunctions.test_mmap_unwriteable ____________________

self = <astropy.io.fits.tests.test_core.TestFileFunctions object at 0x7fcdda8732e0>

    def test_mmap_unwriteable(self):
        """Regression test for https://github.com/astropy/astropy/issues/968
    
        Temporarily patches mmap.mmap to exhibit platform-specific bad
        behavior.
        """
    
        class MockMmap(mmap.mmap):
            def flush(self):
                raise OSError('flush is broken on this platform')
    
        old_mmap = mmap.mmap
        mmap.mmap = MockMmap
    
        # Force the mmap test to be rerun
        _File.__dict__['_mmap_available']._cache.clear()
    
        try:
            self.copy_file('test0.fits')
            with pytest.warns(AstropyUserWarning, match=r'mmap\.flush is unavailable') as w:
                with fits.open(self.temp('test0.fits'), mode='update',
                               memmap=True) as h:
                    h[1].data[0, 0] = 999
    
>           assert len(w) == 1
E           assert 4 == 1
E            +  where 4 = len(WarningsChecker(record=True))

astropy/io/fits/tests/test_core.py:1007: AssertionError
___________________ TestHDUListFunctions.test_flush_readonly ___________________

self = <astropy.io.fits.tests.test_hdulist.TestHDUListFunctions object at 0x7fcdf8169300>

    def test_flush_readonly(self):
        """Test flushing changes to a file opened in a read only mode."""
    
        oldmtime = os.stat(self.data('test0.fits')).st_mtime
        with fits.open(self.data('test0.fits')) as hdul:
            hdul[0].header['FOO'] = 'BAR'
            with pytest.warns(AstropyUserWarning, match='mode is not supported') as w:
                hdul.flush()
>           assert len(w) == 1
E           assert 4 == 1
E            +  where 4 = len(WarningsChecker(record=True))

astropy/io/fits/tests/test_hdulist.py:462: AssertionError
___________________ TestHDUListFunctions.test_output_verify ____________________

self = <astropy.io.fits.tests.test_hdulist.TestHDUListFunctions object at 0x7fcdfdbe36d0>

    def test_output_verify(self):
        hdul = fits.HDUList([fits.PrimaryHDU()])
        hdul[0].header['FOOBAR'] = 42
        hdul.writeto(self.temp('test.fits'))
    
        with open(self.temp('test.fits'), 'rb') as f:
            data = f.read()
        # create invalid card
        data = data.replace(b'FOOBAR  =', b'FOOBAR = ')
        with open(self.temp('test2.fits'), 'wb') as f:
            f.write(data)
    
        with pytest.raises(VerifyError):
            with fits.open(self.temp('test2.fits'), mode='update') as hdul:
                hdul[0].header['MORE'] = 'here'
    
        with pytest.warns(VerifyWarning) as ww:
            with fits.open(self.temp('test2.fits'), mode='update',
                           output_verify='fix+warn') as hdul:
                hdul[0].header['MORE'] = 'here'
>       assert len(ww) == 6
E       assert 9 == 6
E        +  where 9 = len(WarningsChecker(record=True))

astropy/io/fits/tests/test_hdulist.py:1083: AssertionError
_________________________ test_read_write_format[json] _________________________

fmt = 'json'

    @pytest.mark.parametrize('fmt', WRITE_FMTS)
    def test_read_write_format(fmt):
        """
        Test round-trip through pandas write/read for supported formats.
    
        :param fmt: format name, e.g. csv, html, json
        :return:
        """
        # Skip the reading tests
        if fmt == 'html' and not HAS_HTML_DEPS:
            pytest.skip('Missing lxml or bs4 + html5lib for HTML read/write test')
    
        pandas_fmt = 'pandas.' + fmt
        # Explicitly provide dtype to avoid casting 'a' to int32.
        # See https://github.com/astropy/astropy/issues/8682
        t = Table([[1, 2, 3], [1.0, 2.5, 5.0], ['a', 'b', 'c']],
                  dtype=(np.int64, np.float64, str))
        buf = StringIO()
>       t.write(buf, format=pandas_fmt)

astropy/io/misc/tests/test_pandas.py:43: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
astropy/table/connect.py:127: in __call__
    registry.write(instance, *args, **kwargs)
astropy/io/registry.py:563: in write
    writer(data, *args, **kwargs)
astropy/io/misc/pandas/connect.py:113: in _pandas_write
    return write_method(filespec, **write_kwargs)
/usr/lib64/python3.10/site-packages/pandas/core/generic.py:2465: in to_json
    return json.to_json(
/usr/lib64/python3.10/site-packages/pandas/io/json/_json.py:95: in to_json
    ).write()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <pandas.io.json._json.FrameWriter object at 0x7fcddac6b670>

    def write(self):
        iso_dates = self.date_format == "iso"
>       return dumps(
            self.obj_to_write,
            orient=self.orient,
            double_precision=self.double_precision,
            ensure_ascii=self.ensure_ascii,
            date_unit=self.date_unit,
            iso_dates=iso_dates,
            default_handler=self.default_handler,
            indent=self.indent,
        )
E       ValueError: "dtype('O')" or "<class 'pandas.core.indexes.base.Index'>" are not JSON serializable yet

/usr/lib64/python3.10/site-packages/pandas/io/json/_json.py:147: ValueError
____________________________ test_warnings_logging _____________________________

    def test_warnings_logging():
    
        # Without warnings logging
        with pytest.warns(AstropyUserWarning, match="This is a warning") as warn_list:
            with log.log_to_list() as log_list:
                warnings.warn("This is a warning", AstropyUserWarning)
        assert len(log_list) == 0
        assert len(warn_list) == 1
    
        # With warnings logging
        with pytest.warns(None) as warn_list:
            log.enable_warnings_logging()
            with log.log_to_list() as log_list:
                warnings.warn("This is a warning", AstropyUserWarning)
            log.disable_warnings_logging()
        assert len(log_list) == 1
>       assert len(warn_list) == 0
E       assert 2 == 0
E        +  where 2 = len(WarningsChecker(record=True))

astropy/tests/test_logger.py:89: AssertionError
----------------------------- Captured stderr call -----------------------------
WARNING: This is a warning [astropy.tests.test_logger]
------------------------------ Captured log call -------------------------------
WARNING  astropy:logger.py:204 This is a warning
___________________ test_warnings_logging_with_custom_class ____________________

    def test_warnings_logging_with_custom_class():
        class CustomAstropyWarningClass(AstropyWarning):
            pass
    
        # With warnings logging
        with pytest.warns(None) as warn_list:
            log.enable_warnings_logging()
            with log.log_to_list() as log_list:
                warnings.warn("This is a warning", CustomAstropyWarningClass)
            log.disable_warnings_logging()
        assert len(log_list) == 1
>       assert len(warn_list) == 0
E       assert 2 == 0
E        +  where 2 = len(WarningsChecker(record=True))

astropy/tests/test_logger.py:127: AssertionError
----------------------------- Captured stderr call -----------------------------
WARNING: CustomAstropyWarningClass: This is a warning [astropy.tests.test_logger]
------------------------------ Captured log call -------------------------------
WARNING  astropy:logger.py:204 CustomAstropyWarningClass: This is a warning
_________________ test_warning_logging_with_io_votable_warning _________________

    def test_warning_logging_with_io_votable_warning():
        from astropy.io.votable.exceptions import W02, vo_warn
    
        with pytest.warns(None) as warn_list:
            log.enable_warnings_logging()
            with log.log_to_list() as log_list:
                vo_warn(W02, ('a', 'b'))
            log.disable_warnings_logging()
        assert len(log_list) == 1
>       assert len(warn_list) == 0
E       assert 2 == 0
E        +  where 2 = len(WarningsChecker(record=True))

astropy/tests/test_logger.py:142: AssertionError

Comment 4 Miro Hrončok 2021-06-04 20:13:40 UTC
This is a mass-posted update. Sorry if it is not 100% accurate to this bugzilla.


The Python 3.10 rebuild is in progress in a Koji side tag. If you manage to fix the problem, please commit the fix in the rawhide branch, but don't build the package in regular rawhide.

You can either build the package in the side tag, with:

    $ fedpkg build --target=f35-python

Or you can the build and we will eventually build it for you.

Note that the rebuild is still in progress, so not all (build) dependencies of this package might be available right away.

Thanks.

See also https://fedoraproject.org/wiki/Changes/Python3.10

If you have general questions about the rebuild, please use this mailing list thread: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/G47SGOYIQLRDTWGOSLSWERZSSHXDEDH5/

Comment 5 Miro Hrončok 2021-06-07 22:58:26 UTC
The f35-python side tag has been merged to Rawhide. From now on, build as you would normally build.

Comment 6 Miro Hrončok 2021-06-08 11:22:11 UTC
*** Bug 1968898 has been marked as a duplicate of this bug. ***

Comment 7 Victor Stinner 2021-06-08 15:23:39 UTC
I commented the issue upstream:
https://github.com/astropy/astropy/issues/11821#issuecomment-856861575

Currently, the package can only be built with Python 3.10 in the Python 3.10 COPR. In Rawhide using "mock --enablerepo=local", I get two dependency issues to build astropy (networkx, python3-h5py).

Comment 8 Sergio Pascual 2021-06-09 08:57:33 UTC
(In reply to Victor Stinner from comment #7)
> I commented the issue upstream:
> https://github.com/astropy/astropy/issues/11821#issuecomment-856861575
> 
> Currently, the package can only be built with Python 3.10 in the Python 3.10
> COPR. In Rawhide using "mock --enablerepo=local", I get two dependency
> issues to build astropy (networkx, python3-h5py).

Thank you Victor. networkx is a dependency of scikit-image, which astropy requires for testing

Comment 9 Sergio Pascual 2021-06-17 11:58:41 UTC
I'm going to deselect the broken tests. Given that the problems are reported upstream, I expect they will get fixed in time

Comment 10 Zbigniew Jędrzejewski-Szmek 2021-06-17 19:46:42 UTC
Rebuilt.


Note You need to log in before you can comment on or make changes to this bug.