Bug 2096956

Summary:	h5py fails to build on power: Insufficient precision in available types to represent (127, 112, 15, 0, 112)
Product:	[Fedora] Fedora	Reporter:	Miro Hrončok <mhroncok>
Component:	h5py	Assignee:	Terje Røsten <terje.rosten>
Status:	CLOSED ERRATA	QA Contact:	Fedora Extras Quality Assurance <extras-qa>
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	rawhide	CC:	dan, orion, steve.traylen, terje.rosten, thrnciar
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	h5py-3.7.0-2.fc37 h5py-3.6.0-1.1.fc36	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2022-06-14 17:22:55 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1071880, 1992484, 2016048, 2045102

Description Miro Hrončok 2022-06-14 15:43:52 UTC

Description of problem:
h5py fails to build on power, see https://koschei.fedoraproject.org/package/h5py

=================================== FAILURES ===================================
_______________________ TestCreateShape.test_complex256 ________________________
self = <h5py.tests.test_dataset.TestCreateShape testMethod=test_complex256>
    @ut.skipIf(not hasattr(np, "complex256"), "No support for complex256")
    def test_complex256(self):
        """ Confirm that the default dtype is float """
        dset = self.f.create_dataset('foo', (63,),
                                     dtype=np.dtype('complex256'))
>       self.assertEqual(dset.dtype, np.dtype('complex256'))
serial/build/lib.linux-ppc64le-3.10/h5py/tests/test_dataset.py:119: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
h5py/_objects.pyx:54: in h5py._objects.with_phil.wrapper
    ???
h5py/_objects.pyx:55: in h5py._objects.with_phil.wrapper
    ???
serial/build/lib.linux-ppc64le-3.10/h5py/_hl/dataset.py:487: in dtype
    return self.id.dtype
h5py/h5d.pyx:131: in h5py.h5d.DatasetID.dtype.__get__
    ???
h5py/h5d.pyx:134: in h5py.h5d.DatasetID.dtype.__get__
    ???
h5py/h5t.pyx:434: in h5py.h5t.TypeID.dtype.__get__
    ???
h5py/h5t.pyx:435: in h5py.h5t.TypeID.dtype.__get__
    ???
h5py/h5t.pyx:1218: in h5py.h5t.TypeCompoundID.py_dtype
    ???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
>   ???
E   ValueError: Insufficient precision in available types to represent (127, 112, 15, 0, 112)
h5py/h5t.pyx:1079: ValueError
_______________________ TestCreateShape.test_long_double _______________________
>   ???
E   KeyError: <class 'numpy.float128'>
h5py/h5t.pyx:1431: KeyError
During handling of the above exception, another exception occurred:
self = <h5py.tests.test_dataset.TestCreateShape testMethod=test_long_double>
    def test_long_double(self):
        """ Confirm that the default dtype is float """
>       dset = self.f.create_dataset('foo', (63,), dtype=np.longdouble)
serial/build/lib.linux-ppc64le-3.10/h5py/tests/test_dataset.py:109: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
serial/build/lib.linux-ppc64le-3.10/h5py/_hl/group.py:149: in create_dataset
    dsid = dataset.make_new_dset(group, shape, dtype, data, name, **kwds)
serial/build/lib.linux-ppc64le-3.10/h5py/_hl/dataset.py:91: in make_new_dset
    tid = h5t.py_create(dtype, logical=1)
h5py/h5t.pyx:1663: in h5py.h5t.py_create
    ???
h5py/h5t.pyx:1687: in h5py.h5t.py_create
    ???
h5py/h5t.pyx:1694: in h5py.h5t.py_create
    ???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
>   ???
E   TypeError: Unsupported float type (float128)
h5py/h5t.pyx:1433: TypeError
_________________ TestTypeFloatID.test_custom_float_promotion __________________
self = <h5py.tests.test_h5t.TestTypeFloatID testMethod=test_custom_float_promotion>
    def test_custom_float_promotion(self):
        """Custom floats are correctly promoted to standard floats on read."""
    
        # This test uses the low-level API, so we need names as byte strings
        test_filename = self.mktemp().encode()
        dataset = b'DS1'
        dataset2 = b'DS2'
        dataset3 = b'DS3'
        dataset4 = b'DS4'
        dataset5 = b'DS5'
    
        dims = (4, 7)
    
        wdata = np.array([[-1.50066626e-09,   1.40062184e-09,   1.81216819e-10,
                           4.01087163e-10,   4.27917257e-10,  -7.04858394e-11,
                           5.74800652e-10],
                          [-1.50066626e-09,   4.86579665e-10,   3.42879503e-10,
                           5.12045517e-10,   5.10226528e-10,   2.24190444e-10,
                           3.93356459e-10],
                          [-1.50066626e-09,   5.24778443e-10,   8.19454726e-10,
                           1.28966349e-09,   1.68483894e-10,   5.71276360e-11,
                           -1.08684617e-10],
                          [-1.50066626e-09,  -1.08343556e-10,  -1.58934199e-10,
                           8.52196536e-10,   6.18456397e-10,   6.16637408e-10,
                           1.31694833e-09]], dtype=np.float32)
    
        wdata2 = np.array([[-1.50066626e-09,   5.63886715e-10,  -8.74251782e-11,
                            1.32558853e-10,   1.59161573e-10,   2.29420039e-10,
                            -7.24185156e-11],
                           [-1.50066626e-09,   1.87810656e-10,   7.74889486e-10,
                            3.95630195e-10,   9.42236511e-10,   8.38554115e-10,
                            -8.71978045e-11],
                           [-1.50066626e-09,   6.20275387e-10,   7.34871719e-10,
                            6.64840627e-10,   2.64662958e-10,   1.05319486e-09,
                            1.68256520e-10],
                           [-1.50066626e-09,   1.67347025e-10,   5.12045517e-10,
                            3.36513040e-10,   1.02545528e-10,   1.28784450e-09,
                            4.06089384e-10]], dtype=np.float32)
    
        # Create a new file using the default properties.
        fid = h5py.h5f.create(test_filename)
        # Create the dataspace.  No maximum size parameter needed.
        space = h5py.h5s.create_simple(dims)
    
        # create a custom type with larger bias
        mytype = h5t.IEEE_F16LE.copy()
        mytype.set_fields(14, 9, 5, 0, 9)
        mytype.set_size(2)
        mytype.set_ebias(53)
        mytype.lock()
    
        dset = h5py.h5d.create(fid, dataset, mytype, space)
        dset.write(h5py.h5s.ALL, h5py.h5s.ALL, wdata)
    
        del dset
    
        # create a custom type with larger exponent
        mytype2 = h5t.IEEE_F16LE.copy()
        mytype2.set_fields(15, 9, 6, 0, 9)
        mytype2.set_size(2)
        mytype2.set_ebias(53)
        mytype2.lock()
    
        dset = h5py.h5d.create(fid, dataset2, mytype2, space)
        dset.write(h5py.h5s.ALL, h5py.h5s.ALL, wdata2)
    
        del dset
    
        # create a custom type which reimplements 16-bit floats
        mytype3 = h5t.IEEE_F16LE.copy()
        mytype3.set_fields(15, 10, 5, 0, 10)
        mytype3.set_size(2)
        mytype3.set_ebias(15)
        mytype3.lock()
    
        dset = h5py.h5d.create(fid, dataset3, mytype3, space)
        dset.write(h5py.h5s.ALL, h5py.h5s.ALL, wdata2)
    
        del dset
    
        # create a custom type with larger bias
        mytype4 = h5t.IEEE_F16LE.copy()
        mytype4.set_fields(15, 10, 5, 0, 10)
        mytype4.set_size(2)
        mytype4.set_ebias(258)
        mytype4.lock()
    
        dset = h5py.h5d.create(fid, dataset4, mytype4, space)
        dset.write(h5py.h5s.ALL, h5py.h5s.ALL, wdata2)
    
        del dset
    
        # create a dataset with long doubles
        dset = h5py.h5d.create(fid, dataset5, h5t.NATIVE_LDOUBLE, space)
        dset.write(h5py.h5s.ALL, h5py.h5s.ALL, wdata2)
    
        # Explicitly close and release resources.
        del space
        del dset
        del fid
    
        f = h5py.File(test_filename, 'r')
    
        # ebias promotion to float32
        values = f[dataset][:]
        np.testing.assert_array_equal(values, wdata)
        self.assertEqual(values.dtype, np.dtype('<f4'))
    
        # esize promotion to float32
        values = f[dataset2][:]
        np.testing.assert_array_equal(values, wdata2)
        self.assertEqual(values.dtype, np.dtype('<f4'))
    
        # regular half floats
        dset = f[dataset3]
        try:
            self.assertEqual(dset.dtype, np.dtype('<f2'))
        except AttributeError:
            self.assertEqual(dset.dtype, np.dtype('<f4'))
    
        # ebias promotion to float64
        dset = f[dataset4]
        self.assertEqual(dset.dtype, np.dtype('<f8'))
    
        # long double floats
        dset = f[dataset5]
>       self.assertEqual(dset.dtype, np.longdouble)
serial/build/lib.linux-ppc64le-3.10/h5py/tests/test_h5t.py:188: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
h5py/_objects.pyx:54: in h5py._objects.with_phil.wrapper
    ???
h5py/_objects.pyx:55: in h5py._objects.with_phil.wrapper
    ???
serial/build/lib.linux-ppc64le-3.10/h5py/_hl/dataset.py:487: in dtype
    return self.id.dtype
h5py/h5d.pyx:131: in h5py.h5d.DatasetID.dtype.__get__
    ???
h5py/h5d.pyx:134: in h5py.h5d.DatasetID.dtype.__get__
    ???
h5py/h5t.pyx:434: in h5py.h5t.TypeID.dtype.__get__
    ???
h5py/h5t.pyx:435: in h5py.h5t.TypeID.dtype.__get__
    ???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
>   ???
E   ValueError: Insufficient precision in available types to represent (127, 112, 15, 0, 112)
h5py/h5t.pyx:1079: ValueError

Version-Release number of selected component (if applicable):
h5py-3.6.0-1


We need to rebuild it for Python 3.11. The mass rebuild is happening now.

Quite a lot of packages requires this, hence setting the severity to high.


$ repoquery -q --repo=rawhide{,-source} --whatrequires python3-h5py --recursive
PyMca-0:5.6.7-6.fc37.src
PyMca-0:5.6.7-6.fc37.x86_64
diffoscope-0:213-1.fc37.src
espresso-0:4.1.4-12.fc37.src
libsonata-0:0.1.11-6.fc37.src
morphio-0:3.3.3-1.fc37.src
python-astropy-0:5.0.4-1.fc37.src
python-dask-0:2022.5.0-4.fc37~bootstrap.src
python-dipy-0:1.5.0-1.fc37.src
python-emcee-0:3.1.2-1.fc37.src
python-exdir-0:0.4.2-2.fc36.src
python-fsleyes-0:1.4.1-1.fc37.src
python-fsleyes-props-0:1.7.3-2.fc36.src
python-fslpy-0:3.9.0-1.fc37.src
python-h5io-0:0.1.2-8.fc36.src
python-hdf5storage-0:0.1.18-5.fc36.src
python-hdmf-0:3.1.1-6.fc36.src
python-klusta-0:3.0.16-19.fc36.src
python-lazy-ops-0:0.2.0-2.fc36.src
python-lfpy-0:2.2.4-1.fc37.src
python-mne-0:0.24.1-4.fc36.src
python-mne-bids-0:0.8-2.fc36.src
python-neo-0:0.10.2-1.fc37.src
python-nibabel-0:3.2.2-1.fc37.src
python-nixio-0:1.5.3-1.fc37.src
python-pybv-0:0.6.0-2.fc36.src
python-pymatreader-0:0.0.24-6.fc36.src
python-pynn-0:0.10.0-2.fc36.src
python-pynwb-0:2.0.0-7.fc36.src
python-pyqtgraph-0:0.12.4-2.fc37.src
python-pyscf-0:2.0.1-4.fc36.src
python-simframe-0:1.0.2-1.fc37.src
python-spyking-circus-0:1.1.0-2.fc37.src
python-zarr-0:2.11.3-1.fc37.src
python3-dipy-0:1.5.0-1.fc37.x86_64
python3-espresso-mpich-0:4.1.4-12.fc37.x86_64
python3-espresso-openmpi-0:4.1.4-12.fc37.x86_64
python3-fsleyes-0:1.4.1-1.fc37.noarch
python3-fsleyes-props-0:1.7.3-2.fc36.noarch
python3-fslpy-0:3.9.0-1.fc37.noarch
python3-h5io-0:0.1.2-8.fc36.noarch
python3-hdf5storage-0:0.1.18-5.fc36.noarch
python3-hdmf-0:3.1.1-6.fc36.noarch
python3-klusta-0:3.0.16-19.fc36.noarch
python3-lazy-ops-0:0.2.0-2.fc36.noarch
python3-lfpy-0:2.2.4-1.fc37.x86_64
python3-mne-0:0.24.1-4.fc36.noarch
python3-mne-bids-0:0.8-2.fc36.noarch
python3-nixio-0:1.5.3-1.fc37.noarch
python3-pymatreader-0:0.0.24-6.fc36.noarch
python3-pynwb-0:2.0.0-7.fc36.noarch
python3-pyscf-0:2.0.1-4.fc36.x86_64
python3-simframe-0:1.0.2-1.fc37.noarch
python3-spyking-circus-0:1.1.0-2.fc37.noarch
python3-tvb-data-0:1.5.9-8.fc36.noarch
veusz-0:3.4-2.fc36.src

Comment 1 Dan Horák 2022-06-14 16:13:37 UTC

The last successful build of h5py has been made with the ieee128 type as default, but the new type likely still plays a role in this failure as the log points to Float128 and long double.

Comment 2 Dan Horák 2022-06-14 16:28:59 UTC

and they have a special case for ppc64 and float128, see https://github.com/h5py/h5py/blob/master/h5py/h5t.pyx#L285
perhaps it just need to be removed ...

Comment 3 Miro Hrončok 2022-06-14 17:22:55 UTC

https://src.fedoraproject.org/rpms/h5py/c/4578df64e4c923d1c3edf1213a209500e7caf453?branch=rawhide

Comment 4 Dan Horák 2022-06-14 17:25:57 UTC

Terje, I think the fix from rawhide should go into f36 as well. The original successful build in f36 used the old nympy (not rebuilt with the ieee128 long double) thus it required the workaround from the h5py code I guess.

Comment 5 Fedora Update System 2022-06-14 19:15:49 UTC

FEDORA-2022-94490f62c1 has been submitted as an update to Fedora 36. https://bodhi.fedoraproject.org/updates/FEDORA-2022-94490f62c1

Comment 6 Fedora Update System 2022-06-15 01:11:31 UTC

FEDORA-2022-94490f62c1 has been pushed to the Fedora 36 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2022-94490f62c1`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2022-94490f62c1

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 7 Fedora Update System 2022-06-23 00:41:28 UTC

FEDORA-2022-94490f62c1 has been pushed to the Fedora 36 stable repository.
If problem still persists, please make note of it in this bug report.