Bug 2325425
Summary: | python-pydantic-core fails to build with Python 3.14: AssertionError: Regex pattern did not match. + AttributeError: module 'typing' has no attribute 'ByteString' | ||
---|---|---|---|
Product: | [Fedora] Fedora | Reporter: | Karolina Surma <ksurma> |
Component: | python-pydantic-core | Assignee: | Maxwell G <maxwell> |
Status: | ASSIGNED --- | QA Contact: | |
Severity: | unspecified | Docs Contact: | |
Priority: | unspecified | ||
Version: | rawhide | CC: | code, fti-bugs, ksurma, maxwell, mhroncok, python-packagers-sig, rust-sig |
Target Milestone: | --- | ||
Target Release: | --- | ||
Hardware: | Unspecified | ||
OS: | Unspecified | ||
Whiteboard: | |||
Fixed In Version: | Doc Type: | If docs needed, set a value | |
Doc Text: | Story Points: | --- | |
Clone Of: | Environment: | ||
Last Closed: | Type: | Bug | |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: | |||
Bug Depends On: | 2372054 | ||
Bug Blocks: | 2322407, 2339432, 2339435, 2353126, 2371692, 2371713, 2371714, 2371715, 2371719, 2371731, 2371761, 2371766, 2371776, 2371822, 2371850, 2371871, 2371885, 2371886, 2371887, 2371888, 2371889, 2371890, 2371891, 2371892, 2371938, 2371965, 2371978, 2371982, 2372011, 2372012, 2372013, 2372014, 2372017, 2372031, 2372039, 2372049, 2372050, 2372056, 2372057, 2372060, 2372083, 2372085, 2372089, 2372090, 2372106, 2372111, 2372147, 2372150, 2372153, 2372173, 2372186, 2372233, 2372239, 2371947, 2372055 |
Description
Karolina Surma
2024-11-12 10:03:57 UTC
This bug appears to have been reported against 'rawhide' during the Fedora Linux 42 development cycle. Changing version to 42. With Python 3.14.0a5 there are different failures.
1) AssertionErrors in test_isinstance[python] and test_isinstance[json] as reported above.
2) Segmentation fault in tests/test_json.py::test_partial_parse, sometimes accompanied by a garbage output from the test itself, example:
_________________________________________________ test_partial_parse __________________________________________________
def test_partial_parse():
with pytest.raises(ValueError, match='EOF while parsing a string at line 1 column 15'):
from_json('["aa", "bb", "c')
> assert from_json('["aa", "bb", "c', allow_partial=True) == ['aa', 'bb']
E AssertionError: assert ['Ø[', 'fo'] == ['aa', 'bb']
E
E At index 0 diff: 'Ø[' != 'aa'
E
E Full diff:
E [
E - 'aa',
E ? ^^
E + 'Ø[',
E ? ^^
E - 'bb',
E ? ^^
E + 'fo',
E ? ^^
E ]
tests/test_json.py:374: AssertionError
3) Segmentation fault in tests/validators/test_json.py::test_any - same as above
It’s hard to report things like this upstream when one can’t even build for Python 3.14 in a virtual environment, e.g.: $ git clone https://github.com/pydantic/pydantic-core.git $ cd pydantic-core $ uv venv --python 3.14 $ . .venv/bin/activate (pydantic-core) $ PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 uv pip install -e . Resolved 2 packages in 257ms × Failed to build `pydantic-core @ file:///home/ben/src/forks/pydantic-core` ├─▶ The build backend returned an error ╰─▶ Call to `maturin.build_wheel` failed (exit status: 1) [stdout] Running `maturin pep517 build-wheel -i /home/ben/.cache/uv/builds-v0/.tmpvltD2Y/bin/python --compatibility off` [stderr] 📦 Including license file "/home/ben/src/forks/pydantic-core/LICENSE" 🍹 Building a mixed python/rust project 🔗 Found pyo3 bindings 🐍 Found CPython 3.14 at /home/ben/.cache/uv/builds-v0/.tmpvltD2Y/bin/python 📡 Using build options features, bindings from pyproject.toml Compiling pyo3-build-config v0.23.5 Compiling pyo3-macros-backend v0.23.5 Compiling pyo3-ffi v0.23.5 Compiling pyo3 v0.23.5 Compiling jiter v0.8.2 Compiling pydantic-core v2.31.1 (/home/ben/src/forks/pydantic-core) Compiling pyo3-macros v0.23.5 error[E0425]: cannot find function, tuple struct or tuple variant `PyUnicode_New` in module `pyo3::ffi` --> /home/ben/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/jiter-0.8.2/src/py_string_cache.rs:216:26 | 216 | let ptr = pyo3::ffi::PyUnicode_New(s.len() as isize, 127); | ^^^^^^^^^^^^^ help: a function with a similar name exists: `PySlice_New` | ::: /home/ben/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/pyo3-ffi-0.23.5/src/sliceobject.rs:50:5 | 50 | / pub fn PySlice_New( 51 | | start: *mut PyObject, 52 | | stop: *mut PyObject, 53 | | step: *mut PyObject, 54 | | ) -> *mut PyObject; | |_______________________- similarly named function `PySlice_New` defined here error[E0425]: cannot find function, tuple struct or tuple variant `PyUnicode_KIND` in module `pyo3::ffi` --> /home/ben/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/jiter-0.8.2/src/py_string_cache.rs:218:33 | 218 | debug_assert_eq!(pyo3::ffi::PyUnicode_KIND(ptr), pyo3::ffi::PyUnicode_1BYTE_KIND); | ^^^^^^^^^^^^^^ help: a function with a similar name exists: `PyUnicode_Find` | ::: /home/ben/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/pyo3-ffi-0.23.5/src/unicodeobject.rs:303:5 | 303 | / pub fn PyUnicode_Find( 304 | | str: *mut PyObject, 305 | | substr: *mut PyObject, 306 | | start: Py_ssize_t, 307 | | end: Py_ssize_t, 308 | | direction: c_int, 309 | | ) -> Py_ssize_t; | |____________________- similarly named function `PyUnicode_Find` defined here error[E0425]: cannot find value `PyUnicode_1BYTE_KIND` in module `pyo3::ffi` --> /home/ben/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/jiter-0.8.2/src/py_string_cache.rs:218:65 | 218 | debug_assert_eq!(pyo3::ffi::PyUnicode_KIND(ptr), pyo3::ffi::PyUnicode_1BYTE_KIND); | ^^^^^^^^^^^^^^^^^^^^ not found in `pyo3::ffi` error[E0425]: cannot find function, tuple struct or tuple variant `PyUnicode_DATA` in module `pyo3::ffi` --> /home/ben/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/jiter-0.8.2/src/py_string_cache.rs:219:31 | 219 | let data_ptr = pyo3::ffi::PyUnicode_DATA(ptr).cast(); | ^^^^^^^^^^^^^^ help: a function with a similar name exists: `PyUnicode_Find` | ::: /home/ben/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/pyo3-ffi-0.23.5/src/unicodeobject.rs:303:5 | 303 | / pub fn PyUnicode_Find( 304 | | str: *mut PyObject, 305 | | substr: *mut PyObject, 306 | | start: Py_ssize_t, 307 | | end: Py_ssize_t, 308 | | direction: c_int, 309 | | ) -> Py_ssize_t; | |____________________- similarly named function `PyUnicode_Find` defined here For more information about this error, try `rustc --explain E0425`. error: could not compile `jiter` (lib) due to 4 previous errors warning: build failed, waiting for other jobs to finish... 💥 maturin failed Caused by: Failed to build a native library through cargo Caused by: Cargo build finished with "exit status: 101": `env -u CARGO PYO3_ENVIRONMENT_SIGNATURE="cpython-3.14-64bit" PYO3_PYTHON="/home/ben/.cache/uv/builds-v0/.tmpvltD2Y/bin/python" PYTHON_SYS_EXECUTABLE="/home/ben/.cache/uv/builds-v0/.tmpvltD2Y/bin/python" "cargo" "rustc" "--features" "pyo3/extension-module" "--message-format" "json-render-diagnostics" "--manifest-path" "/home/ben/src/forks/pydantic-core/Cargo.toml" "--release" "--lib" "--crate-type" "cdylib"` Error: command ['maturin', 'pep517', 'build-wheel', '-i', '/home/ben/.cache/uv/builds-v0/.tmpvltD2Y/bin/python', '--compatibility', 'off'] returned non-zero exit status 1 hint: This usually indicates a problem with the package or the build environment. Pydantic is also complicated and subtle enough that it tends to be difficult to fix issues downstream. Fortunately, upstream does tend to be proactive about supporting new Python versions, just rather later in the release cycle. Segmentation faults and garbage output could be due to pyo3-ffi not yet having fully correct object layouts for Python 3.14, as discussed in https://github.com/PyO3/pyo3/issues/3555. Noted that PyO3 and Pydantic do have some of the same maintainers, particularly David Hewitt, so the projects collaborate closely. Overall, just as we saw with bug 2259626, I think it’s going to be hard to do much about this until later in the Python development cycle. I am aware pydantic developers prefer to wait until the beta versions of Python. No pressure on reporting things upstream too soon. Fortunately pydantic isn't that big of a blocker in the grand scheme of things. I just prefer to update the relevant information in the downstream bug reports when periodically revisiting the dependency tree of the internal rebuild, so that things are easier to find and mitigate, once fixes appear. (In reply to Karolina Surma from comment #4) > I am aware pydantic developers prefer to wait until the beta versions of > Python. No pressure on reporting things upstream too soon. Fortunately > pydantic isn't that big of a blocker in the grand scheme of things. I just > prefer to update the relevant information in the downstream bug reports when > periodically revisiting the dependency tree of the internal rebuild, so that > things are easier to find and mitigate, once fixes appear. I do appreciate the updates. Knowing the latest status is helpful even if I can’t do much about it yet. I’m preparing to ship Pydantic 2.11[1] with pydantic-core 2.33.1[2] in Rawhide. I didn’t necessarily expect this to bring Python 3.14 support but, but I decided to at least check to see if anything had changed. Since building python-inline-snapshot for Python 3.14 is still blocked by dependencies – at least python-executing – I used --without inline_snapshot_tests to get a local mock build of pydantic-core 2.33.1 for Python 3.14. [1] https://src.fedoraproject.org/rpms/python-pydantic/pull-request/43 [2] https://src.fedoraproject.org/rpms/python-pydantic-core/pull-request/21 + /usr/bin/pytest --ignore=tests/benchmarks --ignore=tests/validators/test_allow_partial.py -k '' -rs Fatal Python error: Segmentation fault Current thread 0x00007f1ee0e65bc0 [pytest] (most recent call first): File "/builddir/build/BUILD/python-pydantic-core-2.33.1-build/pydantic-core-2.33.1/tests/serializers/test_any.py", line 59 in __init__ File "/builddir/build/BUILD/python-pydantic-core-2.33.1-build/pydantic-core-2.33.1/tests/serializers/test_any.py", line 78 in <module> File "/usr/lib/python3.14/site-packages/_pytest/assertion/rewrite.py", line 184 in exec_module File "<frozen importlib._bootstrap>", line 938 in _load_unlocked File "<frozen importlib._bootstrap>", line 1342 in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 1371 in _find_and_load File "<frozen importlib._bootstrap>", line 1398 in _gcd_import File "/usr/lib64/python3.14/importlib/__init__.py", line 88 in import_module File "/usr/lib/python3.14/site-packages/_pytest/pathlib.py", line 587 in import_path File "/usr/lib/python3.14/site-packages/_pytest/python.py", line 493 in importtestmodule File "/usr/lib/python3.14/site-packages/_pytest/python.py", line 546 in _getobj File "/usr/lib/python3.14/site-packages/_pytest/python.py", line 284 in obj File "/usr/lib/python3.14/site-packages/_pytest/python.py", line 562 in _register_setup_module_fixture File "/usr/lib/python3.14/site-packages/_pytest/python.py", line 549 in collect File "/usr/lib/python3.14/site-packages/_pytest/runner.py", line 389 in collect File "/usr/lib/python3.14/site-packages/_pytest/runner.py", line 341 in from_call File "/usr/lib/python3.14/site-packages/_pytest/runner.py", line 391 in pytest_make_collect_report File "/usr/lib/python3.14/site-packages/pluggy/_callers.py", line 103 in _multicall File "/usr/lib/python3.14/site-packages/pluggy/_manager.py", line 120 in _hookexec File "/usr/lib/python3.14/site-packages/pluggy/_hooks.py", line 513 in __call__ File "/usr/lib/python3.14/site-packages/_pytest/runner.py", line 567 in collect_one_node File "/usr/lib/python3.14/site-packages/_pytest/main.py", line 835 in _collect_one_node File "/usr/lib/python3.14/site-packages/_pytest/main.py", line 970 in genitems File "/usr/lib/python3.14/site-packages/_pytest/main.py", line 975 in genitems File "/usr/lib/python3.14/site-packages/_pytest/main.py", line 975 in genitems File "/usr/lib/python3.14/site-packages/_pytest/main.py", line 809 in perform_collect File "/usr/lib/python3.14/site-packages/_pytest/main.py", line 347 in pytest_collection File "/usr/lib/python3.14/site-packages/pluggy/_callers.py", line 103 in _multicall File "/usr/lib/python3.14/site-packages/pluggy/_manager.py", line 120 in _hookexec File "/usr/lib/python3.14/site-packages/pluggy/_hooks.py", line 513 in __call__ File "/usr/lib/python3.14/site-packages/_pytest/main.py", line 336 in _main File "/usr/lib/python3.14/site-packages/_pytest/main.py", line 283 in wrap_session File "/usr/lib/python3.14/site-packages/_pytest/main.py", line 330 in pytest_cmdline_main File "/usr/lib/python3.14/site-packages/pluggy/_callers.py", line 103 in _multicall File "/usr/lib/python3.14/site-packages/pluggy/_manager.py", line 120 in _hookexec File "/usr/lib/python3.14/site-packages/pluggy/_hooks.py", line 513 in __call__ File "/usr/lib/python3.14/site-packages/_pytest/config/__init__.py", line 175 in main File "/usr/lib/python3.14/site-packages/_pytest/config/__init__.py", line 201 in console_main File "/usr/bin/pytest", line 8 in <module> That seems to be worse than before, but I trust upstream will investigate Python 3.14 in due time. There’s now a tracking issue https://github.com/pydantic/pydantic/issues/11613. Other related tracking issues/PR’s: https://github.com/PyO3/pyo3/issues/4662 https://github.com/PyO3/pyo3/pull/4811 PyO3 0.25 is releasing today with Python 3.14 support. I know Pydantic will Python 3.14 issues of its own, but a compatible PyO3 helps clear the way for upstream to actually work on them. With b1, in a clean environment, I can't even get that far (build --without inline_snapshot_tests): cargo:rerun-if-changed=generate_self_schema.py --- stderr Traceback (most recent call last): File "/builddir/build/BUILD/python-pydantic-core-2.33.2-build/pydantic-core-2.33.2/generate_self_schema.py", line 245, in <module> main() ~~~~^^ File "/builddir/build/BUILD/python-pydantic-core-2.33.2-build/pydantic-core-2.33.2/generate_self_schema.py", line 212, in main m = re.search(r"Literal\['(.+?)']", type_.__forward_arg__) ^^^^^^^^^^^^^^^^^^^^^ AttributeError: 'str' object has no attribute '__forward_arg__' thread 'main' panicked at build.rs:29:9: generate_self_schema.py failed with exit status: 1 Since 2.35.0, pydantic-core supports Python 3.15, but Pydantic itself is not ready yet. We need to update these together. *** Bug 2372055 has been marked as a duplicate of this bug. *** |