Bug 2395128 - Segmentation fault during open_dataset() with dask and netcdf4
Summary: Segmentation fault during open_dataset() with dask and netcdf4
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: python-xarray
Version: rawhide
Hardware: Unspecified
OS: Linux
high
medium
Target Milestone: ---
Assignee: Elliott Sales de Andrade
QA Contact: Fedora Extras Quality Assurance
URL: https://github.com/pydata/xarray/issu...
Whiteboard:
Depends On:
Blocks: 2372202
TreeView+ depends on / blocked
 
Reported: 2025-09-15 07:56 UTC by david08741
Modified: 2025-10-07 00:20 UTC (History)
4 users (show)

Fixed In Version: python-xarray-2025.9.0-3.fc43
Clone Of:
Environment:
Last Closed: 2025-10-07 00:20:27 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)

Description david08741 2025-09-15 07:56:27 UTC
dask is broken with xarray and netcdf, as netcdf is not thread safe.

Details in upsteam bugreport:
https://github.com/pydata/xarray/issues/9779

Adding the MRE as a test to the xarray build reproduces the issue in e.g. copr:
https://copr.fedorainfracloud.org/coprs/davidsch/testing/build/9550663/


Reproducible: Always

Steps to Reproduce:
1.Run upstream MRE in rawhide with python3.14
Actual Results:

+ PYTHONPATH=/builddir/build/BUILD/python-xarray-2025.4.0-build/BUILDROOT/usr/lib64/python3.14/site-packages:/builddir/build/BUILD/python-xarray-2025.4.0-build/BUILDROOT/usr/lib/python3.14/site-packages
+ python3 /builddir/build/SOURCES/9779_mre.py
/var/tmp/rpm-tmp.8NttXy: line 49:  2258 Segmentation fault         (core dumped) PYTHONPATH="${PYTHONPATH:-/builddir/build/BUILD/python-xarray-2025.4.0-build/BUILDROOT/usr/lib64/python3.14/site-packages:/builddir/build/BUILD/python-xarray-2025.4.0-build/BUILDROOT/usr/lib/python3.14/site-packages}" python3 /builddir/build/SOURCES/9779_mre.py


Expected Results:
no error

Comment 1 david08741 2025-09-15 08:01:32 UTC
This is blocking xbout rebuild. There an abort is triggered, and the I got these backtraces:

```
Current thread 0x00007fe240319bc0 [pytest] (most recent call first):
  Garbage-collecting
  File "/usr/lib/python3.14/site-packages/dask/array/core.py", line 1274 in blockdims_from_blockshape
  File "/usr/lib/python3.14/site-packages/dask/array/core.py", line 3232 in <genexpr>
  File "/usr/lib/python3.14/site-packages/dask/array/core.py", line 3229 in _convert_int_chunk_to_tuple
  File "/usr/lib/python3.14/site-packages/dask/array/core.py", line 3203 in normalize_chunks
  File "/usr/lib/python3.14/site-packages/xarray/namedarray/daskmanager.py", line 57 in normalize_chunks
  File "/usr/lib/python3.14/site-packages/xarray/structure/chunks.py", line 86 in _get_chunk
  File "/usr/lib/python3.14/site-packages/xarray/backends/api.py", line 366 in _chunk_ds
  File "/usr/lib/python3.14/site-packages/xarray/backends/api.py", line 402 in _dataset_from_backend_dataset
  File "/usr/lib/python3.14/site-packages/xarray/backends/api.py", line 693 in open_dataset
  File "/usr/lib/python3.14/site-packages/xarray/backends/api.py", line 1635 in open_mfdataset
  File "/builddir/build/BUILD/python-xbout-0.3.8-build/xbout-0.3.8/xbout/load.py", line 651 in _auto_open_mfboutdataset
  File "/builddir/build/BUILD/python-xbout-0.3.8-build/xbout-0.3.8/xbout/load.py", line 279 in open_boutdataset
  File "/builddir/build/BUILD/python-xbout-0.3.8-build/xbout-0.3.8/xbout/tests/test_animate.py", line 25 in create_test_file
[...]


Current thread's C stack trace (most recent call first):
  Binary file "/lib64/libpython3.14.so.1.0", at _Py_DumpStack+0x4c [0x7fe240723813]
  Binary file "/lib64/libpython3.14.so.1.0", at +0x121297 [0x7fe240724297]
  Binary file "/lib64/libc.so.6", at +0x1a2c0 [0x7fe2404292c0]
  Binary file "/lib64/libc.so.6", at +0x742bc [0x7fe2404832bc]
  Binary file "/lib64/libc.so.6", at gsignal+0x1e [0x7fe24042918e]
  Binary file "/lib64/libc.so.6", at abort+0x26 [0x7fe2404106d0]
  Binary file "/lib64/libc.so.6", at __assert_perror_fail+0x0 [0x7fe240410639]
  Binary file "/lib64/libnetcdf.so.22", at +0x11256 [0x7fe22997e256]
  Binary file "/lib64/libnetcdf.so.22", at nc4_close_hdf5_file+0x55 [0x7fe2299d42b5]
  Binary file "/lib64/libnetcdf.so.22", at NC4_close+0x68 [0x7fe2299d46b8]
  Binary file "/lib64/libnetcdf.so.22", at nc_close+0x40 [0x7fe22998c3b0]
  Binary file "/usr/lib64/python3.14/site-packages/netCDF4/_netCDF4.cpython-314-x86_64-linux-gnu.so", at +0x4a62d [0x7fe229b7062d]
  Binary file "/lib64/libpython3.14.so.1.0", at +0x1e6fa6 [0x7fe2407e9fa6]
  Binary file "/lib64/libpython3.14.so.1.0", at PyObject_VectorcallMethod+0x90 [0x7fe2407ed680]
  Binary file "/usr/lib64/python3.14/site-packages/netCDF4/_netCDF4.cpython-314-x86_64-linux-gnu.so", at +0x111560 [0x7fe229c37560]
  Binary file "/lib64/libpython3.14.so.1.0", at +0x1f78e6 [0x7fe2407fa8e6]
  Binary file "/lib64/libpython3.14.so.1.0", at +0x25e289 [0x7fe240861289]
[...]
```

Comment 2 Elliott Sales de Andrade 2025-09-18 06:33:02 UTC
If upstream doesn't have a fix, I'm not sure I can do anything about it. From the upstream issue, it looks like if you can modify the test to use `engine='h5netcdf'` then you might avoid the problem.

Comment 3 Fedora Update System 2025-10-04 14:24:06 UTC
FEDORA-2025-653d7f2ca9 (python-xarray-2025.9.0-3.fc43 and python-xbout-0.3.8-3.fc43) has been submitted as an update to Fedora 43.
https://bodhi.fedoraproject.org/updates/FEDORA-2025-653d7f2ca9

Comment 4 Fedora Update System 2025-10-05 01:23:51 UTC
FEDORA-2025-653d7f2ca9 has been pushed to the Fedora 43 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2025-653d7f2ca9`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2025-653d7f2ca9

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 5 Fedora Update System 2025-10-07 00:20:27 UTC
FEDORA-2025-653d7f2ca9 (python-xarray-2025.9.0-3.fc43 and python-xbout-0.3.8-3.fc43) has been pushed to the Fedora 43 stable repository.
If problem still persists, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.