dnf fails to build with Python 3.14.0a1. 1: ERROR: test_another_process (tests.test_lock.ProcessLockTest.test_another_process) 1: ---------------------------------------------------------------------- 1: Traceback (most recent call last): 1: File "/builddir/build/BUILD/dnf-4.21.1-build/dnf-4.21.1/tests/test_lock.py", line 113, in test_another_process 1: process.start() 1: ~~~~~~~~~~~~~^^ 1: File "/usr/lib64/python3.14/multiprocessing/process.py", line 121, in start 1: self._popen = self._Popen(self) 1: ~~~~~~~~~~~^^^^^^ 1: File "/usr/lib64/python3.14/multiprocessing/context.py", line 224, in _Popen 1: return _default_context.get_context().Process._Popen(process_obj) 1: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^ 1: File "/usr/lib64/python3.14/multiprocessing/context.py", line 300, in _Popen 1: return Popen(process_obj) 1: File "/usr/lib64/python3.14/multiprocessing/popen_forkserver.py", line 35, in __init__ 1: super().__init__(process_obj) 1: ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^ 1: File "/usr/lib64/python3.14/multiprocessing/popen_fork.py", line 20, in __init__ 1: self._launch(process_obj) 1: ~~~~~~~~~~~~^^^^^^^^^^^^^ 1: File "/usr/lib64/python3.14/multiprocessing/popen_forkserver.py", line 47, in _launch 1: reduction.dump(process_obj, buf) 1: ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^ 1: File "/usr/lib64/python3.14/multiprocessing/reduction.py", line 60, in dump 1: ForkingPickler(file, protocol).dump(obj) 1: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^ 1: TypeError: cannot pickle '_thread.RLock' object 1: when serializing dict item 'thread_lock' 1: when serializing dnf.lock.ProcessLock state 1: when serializing dnf.lock.ProcessLock object 1: when serializing dict item 'lock' 1: when serializing tests.test_lock.OtherProcess state 1: when serializing tests.test_lock.OtherProcess object 1: 1: ====================================================================== 1: ERROR: test_another_process_blocking (tests.test_lock.ProcessLockTest.test_another_process_blocking) 1: ---------------------------------------------------------------------- 1: Traceback (most recent call last): 1: File "/builddir/build/BUILD/dnf-4.21.1-build/dnf-4.21.1/tests/test_lock.py", line 123, in test_another_process_blocking 1: process.start() 1: ~~~~~~~~~~~~~^^ 1: File "/usr/lib64/python3.14/multiprocessing/process.py", line 121, in start 1: self._popen = self._Popen(self) 1: ~~~~~~~~~~~^^^^^^ 1: File "/usr/lib64/python3.14/multiprocessing/context.py", line 224, in _Popen 1: return _default_context.get_context().Process._Popen(process_obj) 1: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^ 1: File "/usr/lib64/python3.14/multiprocessing/context.py", line 300, in _Popen 1: return Popen(process_obj) 1: File "/usr/lib64/python3.14/multiprocessing/popen_forkserver.py", line 35, in __init__ 1: super().__init__(process_obj) 1: ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^ 1: File "/usr/lib64/python3.14/multiprocessing/popen_fork.py", line 20, in __init__ 1: self._launch(process_obj) 1: ~~~~~~~~~~~~^^^^^^^^^^^^^ 1: File "/usr/lib64/python3.14/multiprocessing/popen_forkserver.py", line 47, in _launch 1: reduction.dump(process_obj, buf) 1: ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^ 1: File "/usr/lib64/python3.14/multiprocessing/reduction.py", line 60, in dump 1: ForkingPickler(file, protocol).dump(obj) 1: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^ 1: TypeError: cannot pickle '_thread.RLock' object 1: when serializing dict item 'thread_lock' 1: when serializing dnf.lock.ProcessLock state 1: when serializing dnf.lock.ProcessLock object 1: when serializing dict item 'lock' 1: when serializing tests.test_lock.OtherProcess state 1: when serializing tests.test_lock.OtherProcess object 1: 1: ---------------------------------------------------------------------- 1: Ran 804 tests in 6.090s 1: 1: FAILED (errors=2, skipped=1) 1: <sys>:0: DeprecationWarning: builtin type swigvarlink has no __module__ attribute 1/1 Test #1: test .............................***Failed 6.56 sec https://docs.python.org/3.14/whatsnew/3.14.html For the build logs, see: https://copr-be.cloud.fedoraproject.org/results/@python/python3.14/fedora-rawhide-x86_64/08189199-dnf/ For all our attempts to build dnf with Python 3.14, see: https://copr.fedorainfracloud.org/coprs/g/python/python3.14/package/dnf/ Testing and mass rebuild of packages is happening in copr. You can follow these instructions to test locally in mock if your package builds with Python 3.14: https://copr.fedorainfracloud.org/coprs/g/python/python3.14/ Let us know here if you have any questions. Python 3.14 is planned to be included in Fedora 43. To make that update smoother, we're building Fedora packages with all pre-releases of Python 3.14. A build failure prevents us from testing all dependent packages (transitive [Build]Requires), so if this package is required a lot, it's important for us to get it fixed soon. We'd appreciate help from the people who know this package best, but if you don't want to work on this now, let us know so we can try to work around it on our side.
This patch makes the failures go away: diff --git a/tests/test_lock.py b/tests/test_lock.py index ce9806b4..d45557e9 100644 --- a/tests/test_lock.py +++ b/tests/test_lock.py @@ -42,6 +42,15 @@ import tests.support from tests.support import mock +# The tests here are not compatible with the forkserver method, +# which is the default on Python 3.14+. +# See https://github.com/python/cpython/issues/125714 +if multiprocessing.get_start_method() == 'forkserver': + mp_context = multiprocessing.get_context(method='fork') +else: + mp_context = multiprocessing.get_context() + + class ConcurrencyMixin(object): def __init__(self, lock): self.lock = lock @@ -61,11 +70,11 @@ class OtherThread(ConcurrencyMixin, threading.Thread): self.queue = dnf.pycomp.Queue(1) -class OtherProcess(ConcurrencyMixin, multiprocessing.Process): +class OtherProcess(ConcurrencyMixin, mp_context.Process): def __init__(self, lock): ConcurrencyMixin.__init__(self, lock) - multiprocessing.Process.__init__(self) - self.queue = multiprocessing.Queue(1) + mp_context.Process.__init__(self) + self.queue = mp_context.Queue(1) TARGET = os.path.join(tests.support.USER_RUNDIR, 'unit-test.pid') However, I still get: + ctest -VV UpdateCTestConfiguration from :/builddir/build/BUILD/dnf-4.22.0-build/dnf-4.22.0/build-py3/DartConfiguration.tcl UpdateCTestConfiguration from :/builddir/build/BUILD/dnf-4.22.0-build/dnf-4.22.0/build-py3/DartConfiguration.tcl Test project /builddir/build/BUILD/dnf-4.22.0-build/dnf-4.22.0/build-py3 Constructing a list of tests Done constructing a list of tests Updating test list for fixtures Added 0 tests to meet fixture requirements Checking test dependency graph... Checking test dependency graph end test 1 Start 1: test 1: Test command: /usr/bin/python3 "-m" "unittest" "discover" "-s" "tests" "-t" "/builddir/build/BUILD/dnf-4.22.0-build/dnf-4.22.0" 1: Working Directory: /builddir/build/BUILD/dnf-4.22.0-build/dnf-4.22.0 1: Environment variables: 1: PYTHONPATH=/builddir/build/BUILD/dnf-4.22.0-build/dnf-4.22.0/ 1: ASAN_OPTIONS=verify_asan_link_order=0,detect_leaks=0 1: Test timeout computed to be: 10000000 1: ....History database cannot be created, using in-memory database instead: SQLite error on "/tmp/tests/history.sqlite": Open failed: unable to open database file 1: ......History database cannot be created, using in-memory database instead: SQLite error on "/tmp/tests/history.sqlite": Open failed: unable to open database file 1: ...EHistory database cannot be created, using in-memory database instead: SQLite error on "/tmp/tests/history.sqlite": Open failed: unable to open database file 1: .History database cannot be created, using in-memory database instead: SQLite error on "/tmp/tests/history.sqlite": Open failed: unable to open database file 1: ...EHistory database cannot be created, using in-memory database instead: SQLite error on "/tmp/tests/history.sqlite": Open failed: unable to open database file 1: . 1/1 Test #1: test .............................***Exception: SegFault 0.54 sec 0% tests passed, 1 tests failed out of 1 Total Test time (real) = 0.54 sec The following tests FAILED: 1 - test (SEGFAULT) Errors while running CTest Output from these tests are in: /builddir/build/BUILD/dnf-4.22.0-build/dnf-4.22.0/build-py3/Testing/Temporary/LastTest.log That log has: Start testing: Nov 25 14:01 CET ---------------------------------------------------------- 1/1 Testing: test 1/1 Test: test Command: "/usr/bin/python3" "-m" "unittest" "discover" "-s" "tests" "-t" "/builddir/build/BUILD/dnf-4.22.0-build/dnf-4.22.0" Directory: /builddir/build/BUILD/dnf-4.22.0-build/dnf-4.22.0 "test" start time: Nov 25 14:01 CET Output: ---------------------------------------------------------- ....History database cannot be created, using in-memory database instead: SQLite error on "/tmp/tests/history.sqlite": Open failed: unable to open database file ......History database cannot be created, using in-memory database instead: SQLite error on "/tmp/tests/history.sqlite": Open failed: unable to open database file ...EHistory database cannot be created, using in-memory database instead: SQLite error on "/tmp/tests/history.sqlite": Open failed: unable to open database file .History database cannot be created, using in-memory database instead: SQLite error on "/tmp/tests/history.sqlite": Open failed: unable to open database file ...EHistory database cannot be created, using in-memory database instead: SQLite error on "/tmp/tests/history.sqlite": Open failed: unable to open database file . <end of output> Test time = 0.54 sec ---------------------------------------------------------- Test Failed. "test" end time: Nov 25 14:01 CET "test" time elapsed: 00:00:00 ---------------------------------------------------------- End testing: Nov 25 14:01 CET
Has something relevant changed in Python 3.14 in the mean time? I tried to reproduce it with current Copr repository (python3-3.14.0~a2-1.fc42.x86_64) and I do not observe the _thread.RLock exception. Though I confirm the segfault.
Not that I know of, but perhaps the segfault just happens sooner?
I wonder if the segfault is bz2330562. Considering the failures in https://github.com/rpm-software-management/dnf/pull/2172 look very similar.
Nope, it faults even with python3-libdnf 0.73.4-2.fc42
With bz2331665 fixed, we are back at: 1: ====================================================================== 1: ERROR: test_another_process_blocking (tests.test_lock.ProcessLockTest.test_another_process_blocking) 1: ---------------------------------------------------------------------- 1: Traceback (most recent call last): 1: File "/builddir/build/BUILD/dnf-4.22.0-build/dnf-4.22.0/tests/test_lock.py", line 123, in test_another_process_blocking 1: process.start() 1: ~~~~~~~~~~~~~^^ 1: File "/usr/lib64/python3.14/multiprocessing/process.py", line 121, in start 1: self._popen = self._Popen(self) 1: ~~~~~~~~~~~^^^^^^ 1: File "/usr/lib64/python3.14/multiprocessing/context.py", line 224, in _Popen 1: return _default_context.get_context().Process._Popen(process_obj) 1: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^ 1: File "/usr/lib64/python3.14/multiprocessing/context.py", line 300, in _Popen 1: return Popen(process_obj) 1: File "/usr/lib64/python3.14/multiprocessing/popen_forkserver.py", line 35, in __init__ 1: super().__init__(process_obj) 1: ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^ 1: File "/usr/lib64/python3.14/multiprocessing/popen_fork.py", line 20, in __init__ 1: self._launch(process_obj) 1: ~~~~~~~~~~~~^^^^^^^^^^^^^ 1: File "/usr/lib64/python3.14/multiprocessing/popen_forkserver.py", line 47, in _launch 1: reduction.dump(process_obj, buf) 1: ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^ 1: File "/usr/lib64/python3.14/multiprocessing/reduction.py", line 60, in dump 1: ForkingPickler(file, protocol).dump(obj) 1: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^ 1: TypeError: cannot pickle '_thread.RLock' object 1: when serializing dict item 'thread_lock' 1: when serializing dnf.lock.ProcessLock state 1: when serializing dnf.lock.ProcessLock object 1: when serializing dict item 'lock' 1: when serializing tests.test_lock.OtherProcess state 1: when serializing tests.test_lock.OtherProcess object 1: 1: ---------------------------------------------------------------------- 1: Ran 804 tests in 4.890s 1: 1: FAILED (errors=2, skipped=1) 1: <sys>:0: DeprecationWarning: builtin type swigvarlink has no __module__ attribute 1/1 Test #1: test .............................***Failed 5.30 sec
And my attempt from comment #1 works: https://github.com/rpm-software-management/dnf/pull/2173
Merged upstream. I guess you need this fix in Fedora 42 to unblock your upgrade of Python there. I will apply it to Fedora 42.
> I guess you need this fix in Fedora 42 to unblock your upgrade of Python there. Dist git was fine, but no harm in shipping a rawhide update. Thanks.