Bug 2096956 - h5py fails to build on power: Insufficient precision in available types to represent (127, 112, 15, 0, 112)
Summary: h5py fails to build on power: Insufficient precision in available types to re...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: h5py
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Terje Røsten
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: PPCTracker F36FTBFS PYTHON3.11 F37FTBFS
TreeView+ depends on / blocked
 
Reported: 2022-06-14 15:43 UTC by Miro Hrončok
Modified: 2022-06-23 00:41 UTC (History)
5 users (show)

Fixed In Version: h5py-3.7.0-2.fc37 h5py-3.6.0-1.1.fc36
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-06-14 17:22:55 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Miro Hrončok 2022-06-14 15:43:52 UTC
Description of problem:
h5py fails to build on power, see https://koschei.fedoraproject.org/package/h5py

=================================== FAILURES ===================================
_______________________ TestCreateShape.test_complex256 ________________________
self = <h5py.tests.test_dataset.TestCreateShape testMethod=test_complex256>
    @ut.skipIf(not hasattr(np, "complex256"), "No support for complex256")
    def test_complex256(self):
        """ Confirm that the default dtype is float """
        dset = self.f.create_dataset('foo', (63,),
                                     dtype=np.dtype('complex256'))
>       self.assertEqual(dset.dtype, np.dtype('complex256'))
serial/build/lib.linux-ppc64le-3.10/h5py/tests/test_dataset.py:119: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
h5py/_objects.pyx:54: in h5py._objects.with_phil.wrapper
    ???
h5py/_objects.pyx:55: in h5py._objects.with_phil.wrapper
    ???
serial/build/lib.linux-ppc64le-3.10/h5py/_hl/dataset.py:487: in dtype
    return self.id.dtype
h5py/h5d.pyx:131: in h5py.h5d.DatasetID.dtype.__get__
    ???
h5py/h5d.pyx:134: in h5py.h5d.DatasetID.dtype.__get__
    ???
h5py/h5t.pyx:434: in h5py.h5t.TypeID.dtype.__get__
    ???
h5py/h5t.pyx:435: in h5py.h5t.TypeID.dtype.__get__
    ???
h5py/h5t.pyx:1218: in h5py.h5t.TypeCompoundID.py_dtype
    ???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
>   ???
E   ValueError: Insufficient precision in available types to represent (127, 112, 15, 0, 112)
h5py/h5t.pyx:1079: ValueError
_______________________ TestCreateShape.test_long_double _______________________
>   ???
E   KeyError: <class 'numpy.float128'>
h5py/h5t.pyx:1431: KeyError
During handling of the above exception, another exception occurred:
self = <h5py.tests.test_dataset.TestCreateShape testMethod=test_long_double>
    def test_long_double(self):
        """ Confirm that the default dtype is float """
>       dset = self.f.create_dataset('foo', (63,), dtype=np.longdouble)
serial/build/lib.linux-ppc64le-3.10/h5py/tests/test_dataset.py:109: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
serial/build/lib.linux-ppc64le-3.10/h5py/_hl/group.py:149: in create_dataset
    dsid = dataset.make_new_dset(group, shape, dtype, data, name, **kwds)
serial/build/lib.linux-ppc64le-3.10/h5py/_hl/dataset.py:91: in make_new_dset
    tid = h5t.py_create(dtype, logical=1)
h5py/h5t.pyx:1663: in h5py.h5t.py_create
    ???
h5py/h5t.pyx:1687: in h5py.h5t.py_create
    ???
h5py/h5t.pyx:1694: in h5py.h5t.py_create
    ???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
>   ???
E   TypeError: Unsupported float type (float128)
h5py/h5t.pyx:1433: TypeError
_________________ TestTypeFloatID.test_custom_float_promotion __________________
self = <h5py.tests.test_h5t.TestTypeFloatID testMethod=test_custom_float_promotion>
    def test_custom_float_promotion(self):
        """Custom floats are correctly promoted to standard floats on read."""
    
        # This test uses the low-level API, so we need names as byte strings
        test_filename = self.mktemp().encode()
        dataset = b'DS1'
        dataset2 = b'DS2'
        dataset3 = b'DS3'
        dataset4 = b'DS4'
        dataset5 = b'DS5'
    
        dims = (4, 7)
    
        wdata = np.array([[-1.50066626e-09,   1.40062184e-09,   1.81216819e-10,
                           4.01087163e-10,   4.27917257e-10,  -7.04858394e-11,
                           5.74800652e-10],
                          [-1.50066626e-09,   4.86579665e-10,   3.42879503e-10,
                           5.12045517e-10,   5.10226528e-10,   2.24190444e-10,
                           3.93356459e-10],
                          [-1.50066626e-09,   5.24778443e-10,   8.19454726e-10,
                           1.28966349e-09,   1.68483894e-10,   5.71276360e-11,
                           -1.08684617e-10],
                          [-1.50066626e-09,  -1.08343556e-10,  -1.58934199e-10,
                           8.52196536e-10,   6.18456397e-10,   6.16637408e-10,
                           1.31694833e-09]], dtype=np.float32)
    
        wdata2 = np.array([[-1.50066626e-09,   5.63886715e-10,  -8.74251782e-11,
                            1.32558853e-10,   1.59161573e-10,   2.29420039e-10,
                            -7.24185156e-11],
                           [-1.50066626e-09,   1.87810656e-10,   7.74889486e-10,
                            3.95630195e-10,   9.42236511e-10,   8.38554115e-10,
                            -8.71978045e-11],
                           [-1.50066626e-09,   6.20275387e-10,   7.34871719e-10,
                            6.64840627e-10,   2.64662958e-10,   1.05319486e-09,
                            1.68256520e-10],
                           [-1.50066626e-09,   1.67347025e-10,   5.12045517e-10,
                            3.36513040e-10,   1.02545528e-10,   1.28784450e-09,
                            4.06089384e-10]], dtype=np.float32)
    
        # Create a new file using the default properties.
        fid = h5py.h5f.create(test_filename)
        # Create the dataspace.  No maximum size parameter needed.
        space = h5py.h5s.create_simple(dims)
    
        # create a custom type with larger bias
        mytype = h5t.IEEE_F16LE.copy()
        mytype.set_fields(14, 9, 5, 0, 9)
        mytype.set_size(2)
        mytype.set_ebias(53)
        mytype.lock()
    
        dset = h5py.h5d.create(fid, dataset, mytype, space)
        dset.write(h5py.h5s.ALL, h5py.h5s.ALL, wdata)
    
        del dset
    
        # create a custom type with larger exponent
        mytype2 = h5t.IEEE_F16LE.copy()
        mytype2.set_fields(15, 9, 6, 0, 9)
        mytype2.set_size(2)
        mytype2.set_ebias(53)
        mytype2.lock()
    
        dset = h5py.h5d.create(fid, dataset2, mytype2, space)
        dset.write(h5py.h5s.ALL, h5py.h5s.ALL, wdata2)
    
        del dset
    
        # create a custom type which reimplements 16-bit floats
        mytype3 = h5t.IEEE_F16LE.copy()
        mytype3.set_fields(15, 10, 5, 0, 10)
        mytype3.set_size(2)
        mytype3.set_ebias(15)
        mytype3.lock()
    
        dset = h5py.h5d.create(fid, dataset3, mytype3, space)
        dset.write(h5py.h5s.ALL, h5py.h5s.ALL, wdata2)
    
        del dset
    
        # create a custom type with larger bias
        mytype4 = h5t.IEEE_F16LE.copy()
        mytype4.set_fields(15, 10, 5, 0, 10)
        mytype4.set_size(2)
        mytype4.set_ebias(258)
        mytype4.lock()
    
        dset = h5py.h5d.create(fid, dataset4, mytype4, space)
        dset.write(h5py.h5s.ALL, h5py.h5s.ALL, wdata2)
    
        del dset
    
        # create a dataset with long doubles
        dset = h5py.h5d.create(fid, dataset5, h5t.NATIVE_LDOUBLE, space)
        dset.write(h5py.h5s.ALL, h5py.h5s.ALL, wdata2)
    
        # Explicitly close and release resources.
        del space
        del dset
        del fid
    
        f = h5py.File(test_filename, 'r')
    
        # ebias promotion to float32
        values = f[dataset][:]
        np.testing.assert_array_equal(values, wdata)
        self.assertEqual(values.dtype, np.dtype('<f4'))
    
        # esize promotion to float32
        values = f[dataset2][:]
        np.testing.assert_array_equal(values, wdata2)
        self.assertEqual(values.dtype, np.dtype('<f4'))
    
        # regular half floats
        dset = f[dataset3]
        try:
            self.assertEqual(dset.dtype, np.dtype('<f2'))
        except AttributeError:
            self.assertEqual(dset.dtype, np.dtype('<f4'))
    
        # ebias promotion to float64
        dset = f[dataset4]
        self.assertEqual(dset.dtype, np.dtype('<f8'))
    
        # long double floats
        dset = f[dataset5]
>       self.assertEqual(dset.dtype, np.longdouble)
serial/build/lib.linux-ppc64le-3.10/h5py/tests/test_h5t.py:188: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
h5py/_objects.pyx:54: in h5py._objects.with_phil.wrapper
    ???
h5py/_objects.pyx:55: in h5py._objects.with_phil.wrapper
    ???
serial/build/lib.linux-ppc64le-3.10/h5py/_hl/dataset.py:487: in dtype
    return self.id.dtype
h5py/h5d.pyx:131: in h5py.h5d.DatasetID.dtype.__get__
    ???
h5py/h5d.pyx:134: in h5py.h5d.DatasetID.dtype.__get__
    ???
h5py/h5t.pyx:434: in h5py.h5t.TypeID.dtype.__get__
    ???
h5py/h5t.pyx:435: in h5py.h5t.TypeID.dtype.__get__
    ???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
>   ???
E   ValueError: Insufficient precision in available types to represent (127, 112, 15, 0, 112)
h5py/h5t.pyx:1079: ValueError

Version-Release number of selected component (if applicable):
h5py-3.6.0-1


We need to rebuild it for Python 3.11. The mass rebuild is happening now.

Quite a lot of packages requires this, hence setting the severity to high.


$ repoquery -q --repo=rawhide{,-source} --whatrequires python3-h5py --recursive
PyMca-0:5.6.7-6.fc37.src
PyMca-0:5.6.7-6.fc37.x86_64
diffoscope-0:213-1.fc37.src
espresso-0:4.1.4-12.fc37.src
libsonata-0:0.1.11-6.fc37.src
morphio-0:3.3.3-1.fc37.src
python-astropy-0:5.0.4-1.fc37.src
python-dask-0:2022.5.0-4.fc37~bootstrap.src
python-dipy-0:1.5.0-1.fc37.src
python-emcee-0:3.1.2-1.fc37.src
python-exdir-0:0.4.2-2.fc36.src
python-fsleyes-0:1.4.1-1.fc37.src
python-fsleyes-props-0:1.7.3-2.fc36.src
python-fslpy-0:3.9.0-1.fc37.src
python-h5io-0:0.1.2-8.fc36.src
python-hdf5storage-0:0.1.18-5.fc36.src
python-hdmf-0:3.1.1-6.fc36.src
python-klusta-0:3.0.16-19.fc36.src
python-lazy-ops-0:0.2.0-2.fc36.src
python-lfpy-0:2.2.4-1.fc37.src
python-mne-0:0.24.1-4.fc36.src
python-mne-bids-0:0.8-2.fc36.src
python-neo-0:0.10.2-1.fc37.src
python-nibabel-0:3.2.2-1.fc37.src
python-nixio-0:1.5.3-1.fc37.src
python-pybv-0:0.6.0-2.fc36.src
python-pymatreader-0:0.0.24-6.fc36.src
python-pynn-0:0.10.0-2.fc36.src
python-pynwb-0:2.0.0-7.fc36.src
python-pyqtgraph-0:0.12.4-2.fc37.src
python-pyscf-0:2.0.1-4.fc36.src
python-simframe-0:1.0.2-1.fc37.src
python-spyking-circus-0:1.1.0-2.fc37.src
python-zarr-0:2.11.3-1.fc37.src
python3-dipy-0:1.5.0-1.fc37.x86_64
python3-espresso-mpich-0:4.1.4-12.fc37.x86_64
python3-espresso-openmpi-0:4.1.4-12.fc37.x86_64
python3-fsleyes-0:1.4.1-1.fc37.noarch
python3-fsleyes-props-0:1.7.3-2.fc36.noarch
python3-fslpy-0:3.9.0-1.fc37.noarch
python3-h5io-0:0.1.2-8.fc36.noarch
python3-hdf5storage-0:0.1.18-5.fc36.noarch
python3-hdmf-0:3.1.1-6.fc36.noarch
python3-klusta-0:3.0.16-19.fc36.noarch
python3-lazy-ops-0:0.2.0-2.fc36.noarch
python3-lfpy-0:2.2.4-1.fc37.x86_64
python3-mne-0:0.24.1-4.fc36.noarch
python3-mne-bids-0:0.8-2.fc36.noarch
python3-nixio-0:1.5.3-1.fc37.noarch
python3-pymatreader-0:0.0.24-6.fc36.noarch
python3-pynwb-0:2.0.0-7.fc36.noarch
python3-pyscf-0:2.0.1-4.fc36.x86_64
python3-simframe-0:1.0.2-1.fc37.noarch
python3-spyking-circus-0:1.1.0-2.fc37.noarch
python3-tvb-data-0:1.5.9-8.fc36.noarch
veusz-0:3.4-2.fc36.src

Comment 1 Dan Horák 2022-06-14 16:13:37 UTC
The last successful build of h5py has been made with the ieee128 type as default, but the new type likely still plays a role in this failure as the log points to Float128 and long double.

Comment 2 Dan Horák 2022-06-14 16:28:59 UTC
and they have a special case for ppc64 and float128, see https://github.com/h5py/h5py/blob/master/h5py/h5t.pyx#L285
perhaps it just need to be removed ...

Comment 4 Dan Horák 2022-06-14 17:25:57 UTC
Terje, I think the fix from rawhide should go into f36 as well. The original successful build in f36 used the old nympy (not rebuilt with the ieee128 long double) thus it required the workaround from the h5py code I guess.

Comment 5 Fedora Update System 2022-06-14 19:15:49 UTC
FEDORA-2022-94490f62c1 has been submitted as an update to Fedora 36. https://bodhi.fedoraproject.org/updates/FEDORA-2022-94490f62c1

Comment 6 Fedora Update System 2022-06-15 01:11:31 UTC
FEDORA-2022-94490f62c1 has been pushed to the Fedora 36 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2022-94490f62c1`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2022-94490f62c1

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 7 Fedora Update System 2022-06-23 00:41:28 UTC
FEDORA-2022-94490f62c1 has been pushed to the Fedora 36 stable repository.
If problem still persists, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.