Bug 2323186
| Summary: | dnf fails to build with Python 3.14: TypeError: cannot pickle '_thread.RLock' object | ||
|---|---|---|---|
| Product: | [Fedora] Fedora | Reporter: | Karolina Surma <ksurma> |
| Component: | dnf | Assignee: | Petr Pisar <ppisar> |
| Status: | CLOSED RAWHIDE | QA Contact: | Fedora Extras Quality Assurance <extras-qa> |
| Severity: | unspecified | Docs Contact: | |
| Priority: | high | ||
| Version: | rawhide | CC: | daniel.mach, jmracek, jrohel, ksurma, mblaha, mhroncok, packaging-team-maint, pkratoch, ppisar, rpm-software-management |
| Target Milestone: | --- | Keywords: | Patch, Triaged |
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | dnf-4.22.0-3.fc42 | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2025-01-07 10:36:33 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | |||
| Bug Blocks: | 2322407 | ||
|
Description
Karolina Surma
2024-11-01 15:10:07 UTC
This patch makes the failures go away: diff --git a/tests/test_lock.py b/tests/test_lock.py index ce9806b4..d45557e9 100644 --- a/tests/test_lock.py +++ b/tests/test_lock.py @@ -42,6 +42,15 @@ import tests.support from tests.support import mock +# The tests here are not compatible with the forkserver method, +# which is the default on Python 3.14+. +# See https://github.com/python/cpython/issues/125714 +if multiprocessing.get_start_method() == 'forkserver': + mp_context = multiprocessing.get_context(method='fork') +else: + mp_context = multiprocessing.get_context() + + class ConcurrencyMixin(object): def __init__(self, lock): self.lock = lock @@ -61,11 +70,11 @@ class OtherThread(ConcurrencyMixin, threading.Thread): self.queue = dnf.pycomp.Queue(1) -class OtherProcess(ConcurrencyMixin, multiprocessing.Process): +class OtherProcess(ConcurrencyMixin, mp_context.Process): def __init__(self, lock): ConcurrencyMixin.__init__(self, lock) - multiprocessing.Process.__init__(self) - self.queue = multiprocessing.Queue(1) + mp_context.Process.__init__(self) + self.queue = mp_context.Queue(1) TARGET = os.path.join(tests.support.USER_RUNDIR, 'unit-test.pid') However, I still get: + ctest -VV UpdateCTestConfiguration from :/builddir/build/BUILD/dnf-4.22.0-build/dnf-4.22.0/build-py3/DartConfiguration.tcl UpdateCTestConfiguration from :/builddir/build/BUILD/dnf-4.22.0-build/dnf-4.22.0/build-py3/DartConfiguration.tcl Test project /builddir/build/BUILD/dnf-4.22.0-build/dnf-4.22.0/build-py3 Constructing a list of tests Done constructing a list of tests Updating test list for fixtures Added 0 tests to meet fixture requirements Checking test dependency graph... Checking test dependency graph end test 1 Start 1: test 1: Test command: /usr/bin/python3 "-m" "unittest" "discover" "-s" "tests" "-t" "/builddir/build/BUILD/dnf-4.22.0-build/dnf-4.22.0" 1: Working Directory: /builddir/build/BUILD/dnf-4.22.0-build/dnf-4.22.0 1: Environment variables: 1: PYTHONPATH=/builddir/build/BUILD/dnf-4.22.0-build/dnf-4.22.0/ 1: ASAN_OPTIONS=verify_asan_link_order=0,detect_leaks=0 1: Test timeout computed to be: 10000000 1: ....History database cannot be created, using in-memory database instead: SQLite error on "/tmp/tests/history.sqlite": Open failed: unable to open database file 1: ......History database cannot be created, using in-memory database instead: SQLite error on "/tmp/tests/history.sqlite": Open failed: unable to open database file 1: ...EHistory database cannot be created, using in-memory database instead: SQLite error on "/tmp/tests/history.sqlite": Open failed: unable to open database file 1: .History database cannot be created, using in-memory database instead: SQLite error on "/tmp/tests/history.sqlite": Open failed: unable to open database file 1: ...EHistory database cannot be created, using in-memory database instead: SQLite error on "/tmp/tests/history.sqlite": Open failed: unable to open database file 1: . 1/1 Test #1: test .............................***Exception: SegFault 0.54 sec 0% tests passed, 1 tests failed out of 1 Total Test time (real) = 0.54 sec The following tests FAILED: 1 - test (SEGFAULT) Errors while running CTest Output from these tests are in: /builddir/build/BUILD/dnf-4.22.0-build/dnf-4.22.0/build-py3/Testing/Temporary/LastTest.log That log has: Start testing: Nov 25 14:01 CET ---------------------------------------------------------- 1/1 Testing: test 1/1 Test: test Command: "/usr/bin/python3" "-m" "unittest" "discover" "-s" "tests" "-t" "/builddir/build/BUILD/dnf-4.22.0-build/dnf-4.22.0" Directory: /builddir/build/BUILD/dnf-4.22.0-build/dnf-4.22.0 "test" start time: Nov 25 14:01 CET Output: ---------------------------------------------------------- ....History database cannot be created, using in-memory database instead: SQLite error on "/tmp/tests/history.sqlite": Open failed: unable to open database file ......History database cannot be created, using in-memory database instead: SQLite error on "/tmp/tests/history.sqlite": Open failed: unable to open database file ...EHistory database cannot be created, using in-memory database instead: SQLite error on "/tmp/tests/history.sqlite": Open failed: unable to open database file .History database cannot be created, using in-memory database instead: SQLite error on "/tmp/tests/history.sqlite": Open failed: unable to open database file ...EHistory database cannot be created, using in-memory database instead: SQLite error on "/tmp/tests/history.sqlite": Open failed: unable to open database file . <end of output> Test time = 0.54 sec ---------------------------------------------------------- Test Failed. "test" end time: Nov 25 14:01 CET "test" time elapsed: 00:00:00 ---------------------------------------------------------- End testing: Nov 25 14:01 CET Has something relevant changed in Python 3.14 in the mean time? I tried to reproduce it with current Copr repository (python3-3.14.0~a2-1.fc42.x86_64) and I do not observe the _thread.RLock exception. Though I confirm the segfault. Not that I know of, but perhaps the segfault just happens sooner? I wonder if the segfault is bz2330562. Considering the failures in https://github.com/rpm-software-management/dnf/pull/2172 look very similar. Nope, it faults even with python3-libdnf 0.73.4-2.fc42 With bz2331665 fixed, we are back at: 1: ====================================================================== 1: ERROR: test_another_process_blocking (tests.test_lock.ProcessLockTest.test_another_process_blocking) 1: ---------------------------------------------------------------------- 1: Traceback (most recent call last): 1: File "/builddir/build/BUILD/dnf-4.22.0-build/dnf-4.22.0/tests/test_lock.py", line 123, in test_another_process_blocking 1: process.start() 1: ~~~~~~~~~~~~~^^ 1: File "/usr/lib64/python3.14/multiprocessing/process.py", line 121, in start 1: self._popen = self._Popen(self) 1: ~~~~~~~~~~~^^^^^^ 1: File "/usr/lib64/python3.14/multiprocessing/context.py", line 224, in _Popen 1: return _default_context.get_context().Process._Popen(process_obj) 1: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^ 1: File "/usr/lib64/python3.14/multiprocessing/context.py", line 300, in _Popen 1: return Popen(process_obj) 1: File "/usr/lib64/python3.14/multiprocessing/popen_forkserver.py", line 35, in __init__ 1: super().__init__(process_obj) 1: ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^ 1: File "/usr/lib64/python3.14/multiprocessing/popen_fork.py", line 20, in __init__ 1: self._launch(process_obj) 1: ~~~~~~~~~~~~^^^^^^^^^^^^^ 1: File "/usr/lib64/python3.14/multiprocessing/popen_forkserver.py", line 47, in _launch 1: reduction.dump(process_obj, buf) 1: ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^ 1: File "/usr/lib64/python3.14/multiprocessing/reduction.py", line 60, in dump 1: ForkingPickler(file, protocol).dump(obj) 1: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^ 1: TypeError: cannot pickle '_thread.RLock' object 1: when serializing dict item 'thread_lock' 1: when serializing dnf.lock.ProcessLock state 1: when serializing dnf.lock.ProcessLock object 1: when serializing dict item 'lock' 1: when serializing tests.test_lock.OtherProcess state 1: when serializing tests.test_lock.OtherProcess object 1: 1: ---------------------------------------------------------------------- 1: Ran 804 tests in 4.890s 1: 1: FAILED (errors=2, skipped=1) 1: <sys>:0: DeprecationWarning: builtin type swigvarlink has no __module__ attribute 1/1 Test #1: test .............................***Failed 5.30 sec And my attempt from comment #1 works: https://github.com/rpm-software-management/dnf/pull/2173 Merged upstream. I guess you need this fix in Fedora 42 to unblock your upgrade of Python there. I will apply it to Fedora 42. > I guess you need this fix in Fedora 42 to unblock your upgrade of Python there.
Dist git was fine, but no harm in shipping a rawhide update. Thanks.
|