Bug 2323186 - dnf fails to build with Python 3.14: TypeError: cannot pickle '_thread.RLock' object
Summary: dnf fails to build with Python 3.14: TypeError: cannot pickle '_thread.RLock'...
Keywords:
Status: CLOSED RAWHIDE
Alias: None
Product: Fedora
Classification: Fedora
Component: dnf
Version: rawhide
Hardware: Unspecified
OS: Unspecified
high
unspecified
Target Milestone: ---
Assignee: Petr Pisar
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: PYTHON3.14
TreeView+ depends on / blocked
 
Reported: 2024-11-01 15:10 UTC by Karolina Surma
Modified: 2025-01-07 10:36 UTC (History)
10 users (show)

Fixed In Version: dnf-4.22.0-3.fc42
Clone Of:
Environment:
Last Closed: 2025-01-07 10:36:33 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github rpm-software-management dnf pull 2173 0 None open Tests: Avoid the multiprocessing forkserver method 2024-12-11 16:33:03 UTC

Description Karolina Surma 2024-11-01 15:10:07 UTC
dnf fails to build with Python 3.14.0a1.

1: ERROR: test_another_process (tests.test_lock.ProcessLockTest.test_another_process)
1: ----------------------------------------------------------------------
1: Traceback (most recent call last):
1:   File "/builddir/build/BUILD/dnf-4.21.1-build/dnf-4.21.1/tests/test_lock.py", line 113, in test_another_process
1:     process.start()
1:     ~~~~~~~~~~~~~^^
1:   File "/usr/lib64/python3.14/multiprocessing/process.py", line 121, in start
1:     self._popen = self._Popen(self)
1:                   ~~~~~~~~~~~^^^^^^
1:   File "/usr/lib64/python3.14/multiprocessing/context.py", line 224, in _Popen
1:     return _default_context.get_context().Process._Popen(process_obj)
1:            ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
1:   File "/usr/lib64/python3.14/multiprocessing/context.py", line 300, in _Popen
1:     return Popen(process_obj)
1:   File "/usr/lib64/python3.14/multiprocessing/popen_forkserver.py", line 35, in __init__
1:     super().__init__(process_obj)
1:     ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
1:   File "/usr/lib64/python3.14/multiprocessing/popen_fork.py", line 20, in __init__
1:     self._launch(process_obj)
1:     ~~~~~~~~~~~~^^^^^^^^^^^^^
1:   File "/usr/lib64/python3.14/multiprocessing/popen_forkserver.py", line 47, in _launch
1:     reduction.dump(process_obj, buf)
1:     ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
1:   File "/usr/lib64/python3.14/multiprocessing/reduction.py", line 60, in dump
1:     ForkingPickler(file, protocol).dump(obj)
1:     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^
1: TypeError: cannot pickle '_thread.RLock' object
1: when serializing dict item 'thread_lock'
1: when serializing dnf.lock.ProcessLock state
1: when serializing dnf.lock.ProcessLock object
1: when serializing dict item 'lock'
1: when serializing tests.test_lock.OtherProcess state
1: when serializing tests.test_lock.OtherProcess object
1: 
1: ======================================================================
1: ERROR: test_another_process_blocking (tests.test_lock.ProcessLockTest.test_another_process_blocking)
1: ----------------------------------------------------------------------
1: Traceback (most recent call last):
1:   File "/builddir/build/BUILD/dnf-4.21.1-build/dnf-4.21.1/tests/test_lock.py", line 123, in test_another_process_blocking
1:     process.start()
1:     ~~~~~~~~~~~~~^^
1:   File "/usr/lib64/python3.14/multiprocessing/process.py", line 121, in start
1:     self._popen = self._Popen(self)
1:                   ~~~~~~~~~~~^^^^^^
1:   File "/usr/lib64/python3.14/multiprocessing/context.py", line 224, in _Popen
1:     return _default_context.get_context().Process._Popen(process_obj)
1:            ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
1:   File "/usr/lib64/python3.14/multiprocessing/context.py", line 300, in _Popen
1:     return Popen(process_obj)
1:   File "/usr/lib64/python3.14/multiprocessing/popen_forkserver.py", line 35, in __init__
1:     super().__init__(process_obj)
1:     ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
1:   File "/usr/lib64/python3.14/multiprocessing/popen_fork.py", line 20, in __init__
1:     self._launch(process_obj)
1:     ~~~~~~~~~~~~^^^^^^^^^^^^^
1:   File "/usr/lib64/python3.14/multiprocessing/popen_forkserver.py", line 47, in _launch
1:     reduction.dump(process_obj, buf)
1:     ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
1:   File "/usr/lib64/python3.14/multiprocessing/reduction.py", line 60, in dump
1:     ForkingPickler(file, protocol).dump(obj)
1:     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^
1: TypeError: cannot pickle '_thread.RLock' object
1: when serializing dict item 'thread_lock'
1: when serializing dnf.lock.ProcessLock state
1: when serializing dnf.lock.ProcessLock object
1: when serializing dict item 'lock'
1: when serializing tests.test_lock.OtherProcess state
1: when serializing tests.test_lock.OtherProcess object
1: 
1: ----------------------------------------------------------------------
1: Ran 804 tests in 6.090s
1: 
1: FAILED (errors=2, skipped=1)
1: <sys>:0: DeprecationWarning: builtin type swigvarlink has no __module__ attribute
1/1 Test #1: test .............................***Failed    6.56 sec

https://docs.python.org/3.14/whatsnew/3.14.html

For the build logs, see:
https://copr-be.cloud.fedoraproject.org/results/@python/python3.14/fedora-rawhide-x86_64/08189199-dnf/

For all our attempts to build dnf with Python 3.14, see:
https://copr.fedorainfracloud.org/coprs/g/python/python3.14/package/dnf/

Testing and mass rebuild of packages is happening in copr.
You can follow these instructions to test locally in mock if your package builds with Python 3.14:
https://copr.fedorainfracloud.org/coprs/g/python/python3.14/

Let us know here if you have any questions.

Python 3.14 is planned to be included in Fedora 43.
To make that update smoother, we're building Fedora packages with all pre-releases of Python 3.14.
A build failure prevents us from testing all dependent packages (transitive [Build]Requires),
so if this package is required a lot, it's important for us to get it fixed soon.

We'd appreciate help from the people who know this package best,
but if you don't want to work on this now, let us know so we can try to work around it on our side.

Comment 1 Miro Hrončok 2024-11-25 13:03:13 UTC
This patch makes the failures go away:

 diff --git a/tests/test_lock.py b/tests/test_lock.py
index ce9806b4..d45557e9 100644
--- a/tests/test_lock.py
+++ b/tests/test_lock.py
@@ -42,6 +42,15 @@ import tests.support
 from tests.support import mock
 
 
+# The tests here are not compatible with the forkserver method,
+# which is the default on Python 3.14+.
+# See https://github.com/python/cpython/issues/125714
+if multiprocessing.get_start_method() == 'forkserver':
+    mp_context = multiprocessing.get_context(method='fork')
+else:
+    mp_context = multiprocessing.get_context()
+
+
 class ConcurrencyMixin(object):
     def __init__(self, lock):
         self.lock = lock
@@ -61,11 +70,11 @@ class OtherThread(ConcurrencyMixin, threading.Thread):
         self.queue = dnf.pycomp.Queue(1)
 
 
-class OtherProcess(ConcurrencyMixin, multiprocessing.Process):
+class OtherProcess(ConcurrencyMixin, mp_context.Process):
     def __init__(self, lock):
         ConcurrencyMixin.__init__(self, lock)
-        multiprocessing.Process.__init__(self)
-        self.queue = multiprocessing.Queue(1)
+        mp_context.Process.__init__(self)
+        self.queue = mp_context.Queue(1)
 
 
 TARGET = os.path.join(tests.support.USER_RUNDIR, 'unit-test.pid')


However, I still get:

+ ctest -VV
UpdateCTestConfiguration  from :/builddir/build/BUILD/dnf-4.22.0-build/dnf-4.22.0/build-py3/DartConfiguration.tcl
UpdateCTestConfiguration  from :/builddir/build/BUILD/dnf-4.22.0-build/dnf-4.22.0/build-py3/DartConfiguration.tcl
Test project /builddir/build/BUILD/dnf-4.22.0-build/dnf-4.22.0/build-py3
Constructing a list of tests
Done constructing a list of tests
Updating test list for fixtures
Added 0 tests to meet fixture requirements
Checking test dependency graph...
Checking test dependency graph end
test 1
    Start 1: test

1: Test command: /usr/bin/python3 "-m" "unittest" "discover" "-s" "tests" "-t" "/builddir/build/BUILD/dnf-4.22.0-build/dnf-4.22.0"
1: Working Directory: /builddir/build/BUILD/dnf-4.22.0-build/dnf-4.22.0
1: Environment variables: 
1:  PYTHONPATH=/builddir/build/BUILD/dnf-4.22.0-build/dnf-4.22.0/
1:  ASAN_OPTIONS=verify_asan_link_order=0,detect_leaks=0
1: Test timeout computed to be: 10000000
1: ....History database cannot be created, using in-memory database instead: SQLite error on "/tmp/tests/history.sqlite": Open failed: unable to open database file
1: ......History database cannot be created, using in-memory database instead: SQLite error on "/tmp/tests/history.sqlite": Open failed: unable to open database file
1: ...EHistory database cannot be created, using in-memory database instead: SQLite error on "/tmp/tests/history.sqlite": Open failed: unable to open database file
1: .History database cannot be created, using in-memory database instead: SQLite error on "/tmp/tests/history.sqlite": Open failed: unable to open database file
1: ...EHistory database cannot be created, using in-memory database instead: SQLite error on "/tmp/tests/history.sqlite": Open failed: unable to open database file
1: .
1/1 Test #1: test .............................***Exception: SegFault  0.54 sec

0% tests passed, 1 tests failed out of 1

Total Test time (real) =   0.54 sec

The following tests FAILED:
	  1 - test (SEGFAULT)
Errors while running CTest
Output from these tests are in: /builddir/build/BUILD/dnf-4.22.0-build/dnf-4.22.0/build-py3/Testing/Temporary/LastTest.log


That log has:


Start testing: Nov 25 14:01 CET
----------------------------------------------------------
1/1 Testing: test
1/1 Test: test
Command: "/usr/bin/python3" "-m" "unittest" "discover" "-s" "tests" "-t" "/builddir/build/BUILD/dnf-4.22.0-build/dnf-4.22.0"
Directory: /builddir/build/BUILD/dnf-4.22.0-build/dnf-4.22.0
"test" start time: Nov 25 14:01 CET
Output:
----------------------------------------------------------
....History database cannot be created, using in-memory database instead: SQLite error on "/tmp/tests/history.sqlite": Open failed: unable to open database file
......History database cannot be created, using in-memory database instead: SQLite error on "/tmp/tests/history.sqlite": Open failed: unable to open database file
...EHistory database cannot be created, using in-memory database instead: SQLite error on "/tmp/tests/history.sqlite": Open failed: unable to open database file
.History database cannot be created, using in-memory database instead: SQLite error on "/tmp/tests/history.sqlite": Open failed: unable to open database file
...EHistory database cannot be created, using in-memory database instead: SQLite error on "/tmp/tests/history.sqlite": Open failed: unable to open database file
.
<end of output>
Test time =   0.54 sec
----------------------------------------------------------
Test Failed.
"test" end time: Nov 25 14:01 CET
"test" time elapsed: 00:00:00
----------------------------------------------------------

End testing: Nov 25 14:01 CET

Comment 2 Petr Pisar 2024-12-03 13:18:49 UTC
Has something relevant changed in Python 3.14 in the mean time? I tried to reproduce it with current Copr repository (python3-3.14.0~a2-1.fc42.x86_64) and I do not observe the _thread.RLock exception. Though I confirm the segfault.

Comment 3 Miro Hrončok 2024-12-03 13:24:15 UTC
Not that I know of, but perhaps the segfault just happens sooner?

Comment 4 Miro Hrončok 2024-12-10 14:24:51 UTC
I wonder if the segfault is bz2330562. Considering the failures in https://github.com/rpm-software-management/dnf/pull/2172 look very similar.

Comment 5 Miro Hrončok 2024-12-10 14:28:58 UTC
Nope, it faults even with python3-libdnf 0.73.4-2.fc42

Comment 6 Miro Hrončok 2024-12-11 16:29:55 UTC
With bz2331665 fixed, we are back at:

1: ======================================================================
1: ERROR: test_another_process_blocking (tests.test_lock.ProcessLockTest.test_another_process_blocking)
1: ----------------------------------------------------------------------
1: Traceback (most recent call last):
1:   File "/builddir/build/BUILD/dnf-4.22.0-build/dnf-4.22.0/tests/test_lock.py", line 123, in test_another_process_blocking
1:     process.start()
1:     ~~~~~~~~~~~~~^^
1:   File "/usr/lib64/python3.14/multiprocessing/process.py", line 121, in start
1:     self._popen = self._Popen(self)
1:                   ~~~~~~~~~~~^^^^^^
1:   File "/usr/lib64/python3.14/multiprocessing/context.py", line 224, in _Popen
1:     return _default_context.get_context().Process._Popen(process_obj)
1:            ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
1:   File "/usr/lib64/python3.14/multiprocessing/context.py", line 300, in _Popen
1:     return Popen(process_obj)
1:   File "/usr/lib64/python3.14/multiprocessing/popen_forkserver.py", line 35, in __init__
1:     super().__init__(process_obj)
1:     ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
1:   File "/usr/lib64/python3.14/multiprocessing/popen_fork.py", line 20, in __init__
1:     self._launch(process_obj)
1:     ~~~~~~~~~~~~^^^^^^^^^^^^^
1:   File "/usr/lib64/python3.14/multiprocessing/popen_forkserver.py", line 47, in _launch
1:     reduction.dump(process_obj, buf)
1:     ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
1:   File "/usr/lib64/python3.14/multiprocessing/reduction.py", line 60, in dump
1:     ForkingPickler(file, protocol).dump(obj)
1:     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^
1: TypeError: cannot pickle '_thread.RLock' object
1: when serializing dict item 'thread_lock'
1: when serializing dnf.lock.ProcessLock state
1: when serializing dnf.lock.ProcessLock object
1: when serializing dict item 'lock'
1: when serializing tests.test_lock.OtherProcess state
1: when serializing tests.test_lock.OtherProcess object
1: 
1: ----------------------------------------------------------------------
1: Ran 804 tests in 4.890s
1: 
1: FAILED (errors=2, skipped=1)
1: <sys>:0: DeprecationWarning: builtin type swigvarlink has no __module__ attribute
1/1 Test #1: test .............................***Failed    5.30 sec

Comment 7 Miro Hrončok 2024-12-11 16:33:04 UTC
And my attempt from comment #1 works: https://github.com/rpm-software-management/dnf/pull/2173

Comment 8 Petr Pisar 2025-01-07 09:20:26 UTC
Merged upstream.

I guess you need this fix in Fedora 42 to unblock your upgrade of Python there. I will apply it to Fedora 42.

Comment 9 Miro Hrončok 2025-01-07 10:36:33 UTC
> I guess you need this fix in Fedora 42 to unblock your upgrade of Python there.

Dist git was fine, but no harm in shipping a rawhide update. Thanks.


Note You need to log in before you can comment on or make changes to this bug.