Created attachment 1562747 [details] Full log from Copr python-jep-3.8.2-2 fails to build with Python 3.8 due to various test failures, all of which produce: RuntimeError: fork not supported for subinterpreters Snippet: ERROR: test_exception_cause (test_exceptions.TestExceptions) ---------------------------------------------------------------------- Traceback (most recent call last): File "/builddir/build/BUILD/jep-3.8.2/src/test/python/test_exceptions.py", line 84, in test_exception_cause jep_pipe(build_java_process_cmd('jep.test.TestExceptionCause')) File "/usr/lib64/python3.8/contextlib.py", line 240, in helper return _GeneratorContextManager(func, args, kwds) File "/usr/lib64/python3.8/contextlib.py", line 83, in __init__ self.gen = func(*args, **kwds) File "/builddir/build/BUILD/jep-3.8.2/src/test/python/jep_pipe.py", line 32, in jep_pipe p = subprocess.Popen(argv, stdout=PIPE, stderr=PIPE) File "/usr/lib64/python3.8/subprocess.py", line 830, in __init__ self._execute_child(args, executable, preexec_fn, close_fds, File "/usr/lib64/python3.8/subprocess.py", line 1583, in _execute_child self.pid = _posixsubprocess.fork_exec( RuntimeError: fork not supported for subinterpreters Full log attached
See https://github.com/python/cpython/pull/9279 and https://bugs.python.org/issue34651
Fixed in rawhide. Please retest. https://src.fedoraproject.org/rpms/python-jep/c/b04f360bf3702db89466c8e18d8a34ddff7d74f2?branch=master
No, there's need for another patch. Having to investigate even deeper. Maybe the fork is forbidden because we run python as main interpreter inside mock? Meaning we can't use fork at all?
Of course you can use fork in mock. You cannot use fork in a subinterpreter. I suggest you bring this upstream.
Upstream is aware about the issue with subprocess. Personally, I don't feel competent enough with the internals of Cython to have a good base to discuss with upstream. Let's hope upstream can fix it when python 3.8 is in a stable state and adds official support. > @unittest.skipIf(sys.platform.startswith("win"), "subprocess complications on Windows") $ grep -l 'subprocess complications' src/test/python/*.py src/test/python/test_exceptions.py src/test/python/test_getvalue.py src/test/python/test_invoke.py src/test/python/test_iterators.py src/test/python/test_numpy.py src/test/python/test_preinits.py src/test/python/test_shared_interpreter.py src/test/python/test_shared_modules.py src/test/python/test_synchronized.py At least, we could disable all those tests as well in Fedora. But i don't like that idea because all those tests are randomized across all the test files mixed with native python. In my opinion, some tests with java code should be separately called with a java process instead as they're written in java what fails here by calls to subprocess Popen. Upstream decided for unknown reasons to call all tests from python, propably to have them all accessible with python unittest. $ grep -l jep_pipe src/test/python/*.py src/test/python/jep_pipe.py src/test/python/test_exceptions.py src/test/python/test_getvalue.py src/test/python/test_invoke.py src/test/python/test_iterators.py src/test/python/test_numpy.py src/test/python/test_preinits.py src/test/python/test_run_script.py src/test/python/test_shared_interpreter.py src/test/python/test_shared_modules.py src/test/python/test_synchronized.py
Obviously, there's need to work with upstream to get a good patch. Any help is appreciated. JPype is the preferred way how to call java from inside python. There's JPackage to import (custom) classes and call methods, but the official documentation is stingy about how to do it for custom classes like tests are. https://jpype.readthedocs.io/en/latest/userguide.html#jpackage-class
FTR Besides JPype (mostly recommended in official discussions), there's also Py4J. Though, Py4J needs some kind of wrapper with so called entry points to import any java classes. https://www.py4j.org/getting_started.html#writing-the-java-program Third option: Use pyjnius. But I fail as well there to get any working sample with the native java tests provided in python-jep. https://pyjnius.readthedocs.io/en/stable/quickstart.html#a-minimal-example That proposed wrapper logic means, we'd have to write even more java code to fix our initial issue with Popen and prevent any programatically forbidden forks as explained, to properly execute any java inside the main thread of initial python interpreter.
Another option is to also look deeper into os.exec*(). https://bugs.python.org/issue34651#msg343364 https://www.python-kurs.eu/forking.php
Changing to assigned, as verified means that a fix is actually verified.
No glue how to easily fix in near future. Let's hope for upstream.
This bug appears to have been reported against 'rawhide' during the Fedora 31 development cycle. Changing version to '31'.
This bug appears to have been reported against 'rawhide' during the Fedora 31 development cycle. Changing version to 31.
The coordinated rebuild of Python 3.8 has started in the `f32-python` side tag. If you figure out how to rebuild this package, please don't rebuild it in regular rawhide, but use the side tag instead: on branch master: $ fedpkg build --target=f32-python To wait for a build to show up in the side tag, do: $ koji wait-repo f32-python --build=<nvr> Where <nvr> is name-version-release of the source package, e.g. python-foo-1.1-2.fc32. An updated mock config is posted at: http://copr.fedorainfracloud.org/coprs/g/python/python3.8/ Note that it will take a while before the essential packages are rebuilt, so don't expect all your dependencies to be available right away. Thanks. Let us know if you need up to date info, or if you have any questions. PS this message is mass posted to all the bugs that block the PYTHON38 bug. If this is also a Fedora 31 FTBFS bug and you manage to fix it, you can do a f31 build as usual: on branch f31: $ fedpkg build
The f32-python side tag has been merged. In order to rebuild the package, do it in regular rawhide, but please wait until python3-3.8 is tagged: $ koji wait-repo f32-build --build python3-3.8.0~b3-3.fc32 If your built already started in f32-python, after it is finished, please tag it to rawhide with: $ koji tag-build f32-pending <nvr> For example: $ koji tag-build f32-pending libreoffice-6.3.0.4-3.fc32 Thanks! (This comment is mass posted to all bugzillas blocking the PYTHON38 tracking bug.)
(Python 3.8 has landed in the rawhide buildroot.)
Fixed in rawhide with new upstream release. All tests pass successfully. For more information, see changelog provided by upstream.