Bug 2161636 - python-pytest-xdist: FTBFS in Fedora Rawhide due to %pytest setting PYTEST_XDIST_AUTO_NUM_WORKERS
Summary: python-pytest-xdist: FTBFS in Fedora Rawhide due to %pytest setting PYTEST_XD...
Keywords:
Status: CLOSED RAWHIDE
Alias: None
Product: Fedora
Classification: Fedora
Component: python-pytest-xdist
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Scott Talbert
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On: 2161702
Blocks: F38FTBFS PYTHON3.12
TreeView+ depends on / blocked
 
Reported: 2023-01-17 12:56 UTC by Tomáš Hrnčiar
Modified: 2023-01-18 00:24 UTC (History)
3 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2023-01-18 00:24:43 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github pytest-dev pytest-xdist pull 870 0 None Draft Tests: Unset PYTEST_XDIST_AUTO_NUM_WORKERS when the behavior without the envvar is asserted 2023-01-17 17:04:29 UTC

Description Tomáš Hrnčiar 2023-01-17 12:56:12 UTC
python-pytest-xdist fails to build with Python 3.12.0a3.


=================================== FAILURES ===================================
____________________________ test_auto_detect_cpus _____________________________

pytester = <Pytester PosixPath('/tmp/pytest-of-mockbuild/pytest-0/test_auto_detect_cpus0')>
monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0x7f71eee6ffe0>

    def test_auto_detect_cpus(
        pytester: pytest.Pytester, monkeypatch: pytest.MonkeyPatch
    ) -> None:
        from xdist.plugin import pytest_cmdline_main as check_options
    
        with suppress(ImportError):
            import psutil
    
            monkeypatch.setattr(psutil, "cpu_count", lambda logical=True: None)
    
        if hasattr(os, "sched_getaffinity"):
            monkeypatch.setattr(os, "sched_getaffinity", lambda _pid: set(range(99)))
        elif hasattr(os, "cpu_count"):
            monkeypatch.setattr(os, "cpu_count", lambda: 99)
        else:
            import multiprocessing
    
            monkeypatch.setattr(multiprocessing, "cpu_count", lambda: 99)
    
        config = pytester.parseconfigure("-n2")
        assert config.getoption("numprocesses") == 2
    
        config = pytester.parseconfigure("-nauto")
        check_options(config)
>       assert config.getoption("numprocesses") == 99
E       AssertionError: assert 2 == 99
E        +  where 2 = <bound method Config.getoption of <_pytest.config.Config object at 0x7f71ec606390>>('numprocesses')
E        +    where <bound method Config.getoption of <_pytest.config.Config object at 0x7f71ec606390>> = <_pytest.config.Config object at 0x7f71ec606390>.getoption

/builddir/build/BUILD/pytest-xdist-3.1.0/testing/test_plugin.py:76: AssertionError
_________________________ test_auto_detect_cpus_psutil _________________________

pytester = <Pytester PosixPath('/tmp/pytest-of-mockbuild/pytest-0/test_auto_detect_cpus_psutil0')>
monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0x7f71ecdf7080>

    def test_auto_detect_cpus_psutil(
        pytester: pytest.Pytester, monkeypatch: pytest.MonkeyPatch
    ) -> None:
        from xdist.plugin import pytest_cmdline_main as check_options
    
        psutil = pytest.importorskip("psutil")
    
        monkeypatch.setattr(psutil, "cpu_count", lambda logical=True: 84 if logical else 42)
    
        config = pytester.parseconfigure("-nauto")
        check_options(config)
>       assert config.getoption("numprocesses") == 42
E       AssertionError: assert 2 == 42
E        +  where 2 = <bound method Config.getoption of <_pytest.config.Config object at 0x7f71ec4c25a0>>('numprocesses')
E        +    where <bound method Config.getoption of <_pytest.config.Config object at 0x7f71ec4c25a0>> = <_pytest.config.Config object at 0x7f71ec4c25a0>.getoption

/builddir/build/BUILD/pytest-xdist-3.1.0/testing/test_plugin.py:108: AssertionError
___________________________ test_auto_detect_cpus_os ___________________________

pytester = <Pytester PosixPath('/tmp/pytest-of-mockbuild/pytest-0/test_auto_detect_cpus_os0')>
monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0x7f71eed641a0>
monkeypatch_3_cpus = None

    def test_auto_detect_cpus_os(
        pytester: pytest.Pytester, monkeypatch: pytest.MonkeyPatch, monkeypatch_3_cpus
    ) -> None:
        from xdist.plugin import pytest_cmdline_main as check_options
    
        config = pytester.parseconfigure("-nauto")
        check_options(config)
>       assert config.getoption("numprocesses") == 3
E       AssertionError: assert 2 == 3
E        +  where 2 = <bound method Config.getoption of <_pytest.config.Config object at 0x7f71ed323050>>('numprocesses')
E        +    where <bound method Config.getoption of <_pytest.config.Config object at 0x7f71ed323050>> = <_pytest.config.Config object at 0x7f71ed323050>.getoption

/builddir/build/BUILD/pytest-xdist-3.1.0/testing/test_plugin.py:122: AssertionError
_______________________ test_hook_auto_num_workers_none ________________________

pytester = <Pytester PosixPath('/tmp/pytest-of-mockbuild/pytest-0/test_hook_auto_num_workers_none0')>
monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0x7f71eee8b7d0>
monkeypatch_3_cpus = None

    def test_hook_auto_num_workers_none(
        pytester: pytest.Pytester, monkeypatch: pytest.MonkeyPatch, monkeypatch_3_cpus
    ) -> None:
        # Returning None from a hook to skip it is pytest behavior,
        # but we document it so let's test it.
        from xdist.plugin import pytest_cmdline_main as check_options
    
        pytester.makeconftest(
            """
            def pytest_xdist_auto_num_workers():
                return None
        """
        )
        config = pytester.parseconfigure("-nauto")
        check_options(config)
>       assert config.getoption("numprocesses") == 3
E       AssertionError: assert 2 == 3
E        +  where 2 = <bound method Config.getoption of <_pytest.config.Config object at 0x7f71ec4c1df0>>('numprocesses')
E        +    where <bound method Config.getoption of <_pytest.config.Config object at 0x7f71ec4c1df0>> = <_pytest.config.Config object at 0x7f71ec4c1df0>.getoption

/builddir/build/BUILD/pytest-xdist-3.1.0/testing/test_plugin.py:189: AssertionError
=========================== short test summary info ============================
SKIPPED [3] ../../../../usr/lib/python3.12/site-packages/_pytest/pytester.py:1534: could not import 'pexpect': No module named 'pexpect'
SKIPPED [1] testing/acceptance_test.py:772: pytest 7.2.0 does not have the pytest_warning_captured hook.
SKIPPED [1] ../../../../usr/lib/python3.12/site-packages/_pytest/config/__init__.py:1550: no 'gspecs' option found
XFAIL testing/acceptance_test.py::TestDistEach::test_simple_diffoutput - reason: [NOTRUN] other python versions might not have pytest installed
XFAIL testing/acceptance_test.py::test_terminate_on_hangingnode
XFAIL testing/acceptance_test.py::test_session_hooks - reason: [NOTRUN] works if run outside test suite
XFAIL testing/acceptance_test.py::TestNodeFailure::test_each_multiple - #20: xdist race condition on node restart
XFAIL testing/test_dsession.py::TestDistReporter::test_rsync_printing
XFAIL testing/test_dsession.py::test_pytest_issue419 - duplicate test ids not supported yet
XFAIL testing/test_looponfail.py::TestLooponFailing::test_looponfail_removed_test - broken by pytest 3.1+
XFAIL testing/test_remote.py::test_remoteinitconfig - #59
XFAIL testing/test_remote.py::TestWorkerInteractor::test_happy_run_events_converted - reason: implement a simple test for event production
XFAIL testing/test_workermanage.py::TestNodeManager::test_rsync_roots_no_roots - reason: [NOTRUN] 
XFAIL testing/test_workermanage.py::test_unserialize_warning_msg[Nested] - Nested warning classes are not supported.
FAILED testing/test_plugin.py::test_auto_detect_cpus - AssertionError: assert...
FAILED testing/test_plugin.py::test_auto_detect_cpus_psutil - AssertionError:...
FAILED testing/test_plugin.py::test_auto_detect_cpus_os - AssertionError: ass...
FAILED testing/test_plugin.py::test_hook_auto_num_workers_none - AssertionErr...
======= 4 failed, 160 passed, 5 skipped, 11 xfailed in 60.21s (0:01:00) ========

https://docs.python.org/3.12/whatsnew/3.12.html

For the build logs, see:
https://copr-be.cloud.fedoraproject.org/results/@python/python3.12/fedora-rawhide-x86_64/05239856-python-pytest-xdist/

For all our attempts to build python-pytest-xdist with Python 3.12, see:
https://copr.fedorainfracloud.org/coprs/g/python/python3.12/package/python-pytest-xdist/

Testing and mass rebuild of packages is happening in copr. You can follow these instructions to test locally in mock if your package builds with Python 3.12:
https://copr.fedorainfracloud.org/coprs/g/python/python3.12/

Let us know here if you have any questions.

Python 3.12 is planned to be included in Fedora 39. To make that update smoother, we're building Fedora packages with all pre-releases of Python 3.12.
A build failure prevents us from testing all dependent packages (transitive [Build]Requires), so if this package is required a lot, it's important for us to get it fixed soon.
We'd appreciate help from the people who know this package best, but if you don't want to work on this now, let us know so we can try to work around it on our side.

Comment 1 Tomáš Hrnčiar 2023-01-17 13:40:48 UTC
Apologies, the failure isn't directly related to Python 3.12. The package is also failing on rawhide.

Comment 2 Miro Hrončok 2023-01-17 17:04:29 UTC
The following PR should fix this: https://github.com/pytest-dev/pytest-xdist/pull/870

Comment 3 Scott Talbert 2023-01-17 17:12:19 UTC
(In reply to Miro Hrončok from comment #2)
> The following PR should fix this:
> https://github.com/pytest-dev/pytest-xdist/pull/870

Thanks, Miro!


Note You need to log in before you can comment on or make changes to this bug.