Bug 2279997 - python-pytest-rerunfailures fails to build with pytest 8: AssertionError: E assert False
Summary: python-pytest-rerunfailures fails to build with pytest 8: AssertionError: E a...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: python-pytest-rerunfailures
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Scott K Logan
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: 2256331
TreeView+ depends on / blocked
 
Reported: 2024-05-10 12:35 UTC by Tomáš Hrnčiar
Modified: 2024-12-31 03:10 UTC (History)
3 users (show)

Fixed In Version: python-pytest-rerunfailures-15.0-1.fc41
Clone Of:
Environment:
Last Closed: 2024-12-31 03:10:55 UTC
Type: Bug
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github pytest-dev pytest-rerunfailures issues 267 0 None open Tests fail against main branch of pytest 2024-05-14 14:03:53 UTC

Description Tomáš Hrnčiar 2024-05-10 12:35:37 UTC 
python-pytest-rerunfailures fails to build with pytest 8.

=================================== FAILURES ===================================
_________________ test_run_session_teardown_once_after_reruns __________________

testdir = <Testdir local('/tmp/pytest-of-mockbuild/pytest-0/test_run_session_teardown_once_after_reruns0')>

    def test_run_session_teardown_once_after_reruns(testdir):
        testdir.makepyfile(
            """
            import logging
            import pytest
    
            from unittest import TestCase
    
            @pytest.fixture(scope='session', autouse=True)
            def session_fixture():
                logging.info('session setup')
                yield
                logging.info('session teardown')
    
            @pytest.fixture(scope='class', autouse=True)
            def class_fixture():
                logging.info('class setup')
                yield
                logging.info('class teardown')
    
            @pytest.fixture(scope='function', autouse=True)
            def function_fixture():
                logging.info('function setup')
                yield
                logging.info('function teardown')
    
            @pytest.fixture(scope='function')
            def function_skip_fixture():
                logging.info('skip fixture setup')
                pytest.skip('some reason')
                yield
                logging.info('skip fixture teardown')
    
            @pytest.fixture(scope='function')
            def function_setup_fail_fixture():
                logging.info('fail fixture setup')
                assert False
                yield
                logging.info('fail fixture teardown')
    
            class TestFirstPassLastFail:
    
                @staticmethod
                def test_1():
                    logging.info("TestFirstPassLastFail 1")
    
                @staticmethod
                def test_2():
                    logging.info("TestFirstPassLastFail 2")
                    assert False
    
            class TestFirstFailLastPass:
    
                @staticmethod
                def test_1():
                    logging.info("TestFirstFailLastPass 1")
                    assert False
    
                @staticmethod
                def test_2():
                    logging.info("TestFirstFailLastPass 2")
    
            class TestSkipFirst:
                @staticmethod
                @pytest.mark.skipif(True, reason='Some reason')
                def test_1():
                    logging.info("TestSkipFirst 1")
                    assert False
    
                @staticmethod
                def test_2():
                    logging.info("TestSkipFirst 2")
                    assert False
    
            class TestSkipLast:
                @staticmethod
                def test_1():
                    logging.info("TestSkipLast 1")
                    assert False
    
                @staticmethod
                @pytest.mark.skipif(True, reason='Some reason')
                def test_2():
                    logging.info("TestSkipLast 2")
                    assert False
    
            class TestSkipFixture:
                @staticmethod
                def test_1(function_skip_fixture):
                    logging.info("TestSkipFixture 1")
    
            class TestSetupFailed:
                @staticmethod
                def test_1(function_setup_fail_fixture):
                    logging.info("TestSetupFailed 1")
    
            class TestTestCaseFailFirstFailLast(TestCase):
    
                @staticmethod
                def test_1():
                    logging.info("TestTestCaseFailFirstFailLast 1")
                    assert False
    
                @staticmethod
                def test_2():
                    logging.info("TestTestCaseFailFirstFailLast 2")
                    assert False
    
            class TestTestCaseSkipFirst(TestCase):
    
                @staticmethod
                @pytest.mark.skipif(True, reason='Some reason')
                def test_1():
                    logging.info("TestTestCaseSkipFirst 1")
                    assert False
    
                @staticmethod
                def test_2():
                    logging.info("TestTestCaseSkipFirst 2")
                    assert False
    
            class TestTestCaseSkipLast(TestCase):
    
                @staticmethod
                def test_1():
                    logging.info("TestTestCaseSkipLast 1")
                    assert False
    
                @staticmethod
                @pytest.mark.skipif(True, reason="Some reason")
                def test_2():
                    logging.info("TestTestCaseSkipLast 2")
                    assert False"""
        )
        import logging
    
        logging.info = mock.MagicMock()
    
        result = testdir.runpytest("--reruns", "2")
        expected_calls = [
            mock.call("session setup"),
            # TestFirstPassLastFail
            mock.call("class setup"),
            mock.call("function setup"),
            mock.call("TestFirstPassLastFail 1"),
            mock.call("function teardown"),
            mock.call("function setup"),
            mock.call("TestFirstPassLastFail 2"),
            mock.call("function teardown"),
            mock.call("function setup"),
            mock.call("TestFirstPassLastFail 2"),
            mock.call("function teardown"),
            mock.call("function setup"),
            mock.call("TestFirstPassLastFail 2"),
            mock.call("function teardown"),
            mock.call("class teardown"),
            # TestFirstFailLastPass
            mock.call("class setup"),
            mock.call("function setup"),
            mock.call("TestFirstFailLastPass 1"),
            mock.call("function teardown"),
            mock.call("function setup"),
            mock.call("TestFirstFailLastPass 1"),
            mock.call("function teardown"),
            mock.call("function setup"),
            mock.call("TestFirstFailLastPass 1"),
            mock.call("function teardown"),
            mock.call("function setup"),
            mock.call("TestFirstFailLastPass 2"),
            mock.call("function teardown"),
            mock.call("class teardown"),
            # TestSkipFirst
            mock.call("class setup"),
            mock.call("function setup"),
            mock.call("TestSkipFirst 2"),
            mock.call("function teardown"),
            mock.call("function setup"),
            mock.call("TestSkipFirst 2"),
            mock.call("function teardown"),
            mock.call("function setup"),
            mock.call("TestSkipFirst 2"),
            mock.call("function teardown"),
            mock.call("class teardown"),
            # TestSkipLast
            mock.call("class setup"),
            mock.call("function setup"),
            mock.call("TestSkipLast 1"),
            mock.call("function teardown"),
            mock.call("function setup"),
            mock.call("TestSkipLast 1"),
            mock.call("function teardown"),
            mock.call("function setup"),
            mock.call("TestSkipLast 1"),
            mock.call("function teardown"),
            mock.call("class teardown"),
            # TestSkipFixture
            mock.call("class setup"),
            mock.call("function setup"),
            mock.call("skip fixture setup"),
            mock.call("function teardown"),
            mock.call("class teardown"),
            # TestSetupFailed
            mock.call("class setup"),
            mock.call("function setup"),
            mock.call("fail fixture setup"),
            mock.call("function teardown"),
            mock.call("function setup"),
            mock.call("fail fixture setup"),
            mock.call("function teardown"),
            mock.call("function setup"),
            mock.call("fail fixture setup"),
            mock.call("function teardown"),
            mock.call("class teardown"),
            # TestTestCaseFailFirstFailLast
            mock.call("class setup"),
            mock.call("function setup"),
            mock.call("TestTestCaseFailFirstFailLast 1"),
            mock.call("function teardown"),
            mock.call("function setup"),
            mock.call("TestTestCaseFailFirstFailLast 1"),
            mock.call("function teardown"),
            mock.call("function setup"),
            mock.call("TestTestCaseFailFirstFailLast 1"),
            mock.call("function teardown"),
            mock.call("function setup"),
            mock.call("TestTestCaseFailFirstFailLast 2"),
            mock.call("function teardown"),
            mock.call("function setup"),
            mock.call("TestTestCaseFailFirstFailLast 2"),
            mock.call("function teardown"),
            mock.call("function setup"),
            mock.call("TestTestCaseFailFirstFailLast 2"),
            mock.call("function teardown"),
            mock.call("class teardown"),
            # TestTestCaseSkipFirst
            mock.call("class setup"),
            mock.call("function setup"),
            mock.call("TestTestCaseSkipFirst 2"),
            mock.call("function teardown"),
            mock.call("function setup"),
            mock.call("TestTestCaseSkipFirst 2"),
            mock.call("function teardown"),
            mock.call("function setup"),
            mock.call("TestTestCaseSkipFirst 2"),
            mock.call("function teardown"),
            mock.call("class teardown"),
            # TestTestCaseSkipLast
            mock.call("class setup"),
            mock.call("function setup"),
            mock.call("TestTestCaseSkipLast 1"),
            mock.call("function teardown"),
            mock.call("function setup"),
            mock.call("TestTestCaseSkipLast 1"),
            mock.call("function teardown"),
            mock.call("function setup"),
            mock.call("TestTestCaseSkipLast 1"),
            mock.call("function teardown"),
            mock.call("class teardown"),
            mock.call("session teardown"),
        ]
    
>       logging.info.assert_has_calls(expected_calls, any_order=False)

/builddir/build/BUILD/pytest-rerunfailures-12.0/test_pytest_rerunfailures.py:1076: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <MagicMock id='140508184277600'>
calls = [call('session setup'), call('class setup'), call('function setup'), call('TestFirstPassLastFail 1'), call('function teardown'), call('function setup'), ...]
any_order = False

    def assert_has_calls(self, calls, any_order=False):
        """assert the mock has been called with the specified calls.
        The `mock_calls` list is checked for the calls.
    
        If `any_order` is False (the default) then the calls must be
        sequential. There can be extra calls before or after the
        specified calls.
    
        If `any_order` is True then the calls can be in any order, but
        they must all appear in `mock_calls`."""
        expected = [self._call_matcher(c) for c in calls]
        cause = next((e for e in expected if isinstance(e, Exception)), None)
        all_calls = _CallList(self._call_matcher(c) for c in self.mock_calls)
        if not any_order:
            if expected not in all_calls:
                if cause is None:
                    problem = 'Calls not found.'
                else:
                    problem = ('Error processing expected calls.\n'
                               'Errors: {}').format(
                                   [e if isinstance(e, Exception) else None
                                    for e in expected])
>               raise AssertionError(
                    f'{problem}\n'
                    f'Expected: {_CallList(calls)}'
                    f'{self._calls_repr(prefix="  Actual").rstrip(".")}'
                ) from cause
E               AssertionError: Calls not found.
E               Expected: [call('session setup'),
E                call('class setup'),
E                call('function setup'),
E                call('TestFirstPassLastFail 1'),
E                call('function teardown'),
E                call('function setup'),
E                call('TestFirstPassLastFail 2'),
E                call('function teardown'),
E                call('function setup'),
E                call('TestFirstPassLastFail 2'),
E                call('function teardown'),
E                call('function setup'),
E                call('TestFirstPassLastFail 2'),
E                call('function teardown'),
E                call('class teardown'),
E                call('class setup'),
E                call('function setup'),
E                call('TestFirstFailLastPass 1'),
E                call('function teardown'),
E                call('function setup'),
E                call('TestFirstFailLastPass 1'),
E                call('function teardown'),
E                call('function setup'),
E                call('TestFirstFailLastPass 1'),
E                call('function teardown'),
E                call('function setup'),
E                call('TestFirstFailLastPass 2'),
E                call('function teardown'),
E                call('class teardown'),
E                call('class setup'),
E                call('function setup'),
E                call('TestSkipFirst 2'),
E                call('function teardown'),
E                call('function setup'),
E                call('TestSkipFirst 2'),
E                call('function teardown'),
E                call('function setup'),
E                call('TestSkipFirst 2'),
E                call('function teardown'),
E                call('class teardown'),
E                call('class setup'),
E                call('function setup'),
E                call('TestSkipLast 1'),
E                call('function teardown'),
E                call('function setup'),
E                call('TestSkipLast 1'),
E                call('function teardown'),
E                call('function setup'),
E                call('TestSkipLast 1'),
E                call('function teardown'),
E                call('class teardown'),
E                call('class setup'),
E                call('function setup'),
E                call('skip fixture setup'),
E                call('function teardown'),
E                call('class teardown'),
E                call('class setup'),
E                call('function setup'),
E                call('fail fixture setup'),
E                call('function teardown'),
E                call('function setup'),
E                call('fail fixture setup'),
E                call('function teardown'),
E                call('function setup'),
E                call('fail fixture setup'),
E                call('function teardown'),
E                call('class teardown'),
E                call('class setup'),
E                call('function setup'),
E                call('TestTestCaseFailFirstFailLast 1'),
E                call('function teardown'),
E                call('function setup'),
E                call('TestTestCaseFailFirstFailLast 1'),
E                call('function teardown'),
E                call('function setup'),
E                call('TestTestCaseFailFirstFailLast 1'),
E                call('function teardown'),
E                call('function setup'),
E                call('TestTestCaseFailFirstFailLast 2'),
E                call('function teardown'),
E                call('function setup'),
E                call('TestTestCaseFailFirstFailLast 2'),
E                call('function teardown'),
E                call('function setup'),
E                call('TestTestCaseFailFirstFailLast 2'),
E                call('function teardown'),
E                call('class teardown'),
E                call('class setup'),
E                call('function setup'),
E                call('TestTestCaseSkipFirst 2'),
E                call('function teardown'),
E                call('function setup'),
E                call('TestTestCaseSkipFirst 2'),
E                call('function teardown'),
E                call('function setup'),
E                call('TestTestCaseSkipFirst 2'),
E                call('function teardown'),
E                call('class teardown'),
E                call('class setup'),
E                call('function setup'),
E                call('TestTestCaseSkipLast 1'),
E                call('function teardown'),
E                call('function setup'),
E                call('TestTestCaseSkipLast 1'),
E                call('function teardown'),
E                call('function setup'),
E                call('TestTestCaseSkipLast 1'),
E                call('function teardown'),
E                call('class teardown'),
E                call('session teardown')]
E                 Actual: [call('session setup'),
E                call('class setup'),
E                call('function setup'),
E                call('TestFirstPassLastFail 1'),
E                call('function teardown'),
E                call('function setup'),
E                call('TestFirstPassLastFail 2'),
E                call('function teardown'),
E                call('function setup'),
E                call('TestFirstPassLastFail 2'),
E                call('function teardown'),
E                call('function setup'),
E                call('TestFirstPassLastFail 2'),
E                call('function teardown'),
E                call('function setup'),
E                call('TestFirstFailLastPass 1'),
E                call('function teardown'),
E                call('function setup'),
E                call('TestFirstFailLastPass 1'),
E                call('function teardown'),
E                call('function setup'),
E                call('TestFirstFailLastPass 1'),
E                call('function teardown'),
E                call('function setup'),
E                call('TestFirstFailLastPass 2'),
E                call('function teardown'),
E                call('function setup'),
E                call('TestSkipFirst 2'),
E                call('function teardown'),
E                call('function setup'),
E                call('TestSkipFirst 2'),
E                call('function teardown'),
E                call('function setup'),
E                call('TestSkipFirst 2'),
E                call('function teardown'),
E                call('function setup'),
E                call('TestSkipLast 1'),
E                call('function teardown'),
E                call('function setup'),
E                call('TestSkipLast 1'),
E                call('function teardown'),
E                call('function setup'),
E                call('TestSkipLast 1'),
E                call('function teardown'),
E                call('function setup'),
E                call('skip fixture setup'),
E                call('function teardown'),
E                call('function setup'),
E                call('fail fixture setup'),
E                call('function teardown'),
E                call('function setup'),
E                call('fail fixture setup'),
E                call('function teardown'),
E                call('function setup'),
E                call('fail fixture setup'),
E                call('function teardown'),
E                call('function setup'),
E                call('TestTestCaseFailFirstFailLast 1'),
E                call('function teardown'),
E                call('function setup'),
E                call('TestTestCaseFailFirstFailLast 1'),
E                call('function teardown'),
E                call('function setup'),
E                call('TestTestCaseFailFirstFailLast 1'),
E                call('function teardown'),
E                call('function setup'),
E                call('TestTestCaseFailFirstFailLast 2'),
E                call('function teardown'),
E                call('function setup'),
E                call('TestTestCaseFailFirstFailLast 2'),
E                call('function teardown'),
E                call('function setup'),
E                call('TestTestCaseFailFirstFailLast 2'),
E                call('function teardown'),
E                call('function setup'),
E                call('TestTestCaseSkipFirst 2'),
E                call('function teardown'),
E                call('function setup'),
E                call('TestTestCaseSkipFirst 2'),
E                call('function teardown'),
E                call('function setup'),
E                call('TestTestCaseSkipFirst 2'),
E                call('function teardown'),
E                call('function setup'),
E                call('TestTestCaseSkipLast 1'),
E                call('function teardown'),
E                call('function setup'),
E                call('TestTestCaseSkipLast 1'),
E                call('function teardown'),
E                call('function setup'),
E                call('TestTestCaseSkipLast 1'),
E                call('function teardown')]

/usr/lib64/python3.12/unittest/mock.py:981: AssertionError
----------------------------- Captured stdout call -----------------------------
============================= test session starts ==============================
platform linux -- Python 3.12.3, pytest-8.2.0, pluggy-1.5.0
rootdir: /tmp/pytest-of-mockbuild/pytest-0/test_run_session_teardown_once_after_reruns0
plugins: rerunfailures-12.0, xdist-3.5.0
collected 16 items

test_run_session_teardown_once_after_reruns.py .RRFRRF.sRRFRRFssRRERRFRR [ 75%]
FsRRFRRFs                                                                [100%]

==================================== ERRORS ====================================
___________________ ERROR at setup of TestSetupFailed.test_1 ___________________

    @pytest.fixture(scope='function')
    def function_setup_fail_fixture():
        logging.info('fail fixture setup')
>       assert False
E       assert False

test_run_session_teardown_once_after_reruns.py:34: AssertionError
=================================== FAILURES ===================================
_________________________ TestFirstPassLastFail.test_2 _________________________

    @staticmethod
    def test_2():
        logging.info("TestFirstPassLastFail 2")
>       assert False
E       assert False

test_run_session_teardown_once_after_reruns.py:47: AssertionError
_________________________ TestFirstFailLastPass.test_1 _________________________

    @staticmethod
    def test_1():
        logging.info("TestFirstFailLastPass 1")
>       assert False
E       assert False

test_run_session_teardown_once_after_reruns.py:54: AssertionError
_____________________________ TestSkipFirst.test_2 _____________________________

    @staticmethod
    def test_2():
        logging.info("TestSkipFirst 2")
>       assert False
E       assert False

test_run_session_teardown_once_after_reruns.py:70: AssertionError
_____________________________ TestSkipLast.test_1 ______________________________

    @staticmethod
    def test_1():
        logging.info("TestSkipLast 1")
>       assert False
E       assert False

test_run_session_teardown_once_after_reruns.py:76: AssertionError
_____________________ TestTestCaseFailFirstFailLast.test_1 _____________________

    @staticmethod
    def test_1():
        logging.info("TestTestCaseFailFirstFailLast 1")
>       assert False
E       assert False

test_run_session_teardown_once_after_reruns.py:99: AssertionError
_____________________ TestTestCaseFailFirstFailLast.test_2 _____________________

    @staticmethod
    def test_2():
        logging.info("TestTestCaseFailFirstFailLast 2")
>       assert False
E       assert False

test_run_session_teardown_once_after_reruns.py:104: AssertionError
_________________________ TestTestCaseSkipFirst.test_2 _________________________

    @staticmethod
    def test_2():
        logging.info("TestTestCaseSkipFirst 2")
>       assert False
E       assert False

test_run_session_teardown_once_after_reruns.py:117: AssertionError
_________________________ TestTestCaseSkipLast.test_1 __________________________

    @staticmethod
    def test_1():
        logging.info("TestTestCaseSkipLast 1")
>       assert False
E       assert False

test_run_session_teardown_once_after_reruns.py:124: AssertionError
=============================== warnings summary ===============================
../../../../usr/lib/python3.12/site-packages/_pytest/config/__init__.py:1285
  /usr/lib/python3.12/site-packages/_pytest/config/__init__.py:1285: PytestAssertRewriteWarning: Module already imported so cannot be rewritten: pytest_rerunfailures
    self._mark_plugins_for_rewrite(hook)

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
FAILED test_run_session_teardown_once_after_reruns.py::TestFirstPassLastFail::test_2
FAILED test_run_session_teardown_once_after_reruns.py::TestFirstFailLastPass::test_1
FAILED test_run_session_teardown_once_after_reruns.py::TestSkipFirst::test_2
FAILED test_run_session_teardown_once_after_reruns.py::TestSkipLast::test_1
FAILED test_run_session_teardown_once_after_reruns.py::TestTestCaseFailFirstFailLast::test_1
FAILED test_run_session_teardown_once_after_reruns.py::TestTestCaseFailFirstFailLast::test_2
FAILED test_run_session_teardown_once_after_reruns.py::TestTestCaseSkipFirst::test_2
FAILED test_run_session_teardown_once_after_reruns.py::TestTestCaseSkipLast::test_1
ERROR test_run_session_teardown_once_after_reruns.py::TestSetupFailed::test_1
===== 8 failed, 2 passed, 5 skipped, 1 warning, 1 error, 18 rerun in 0.09s =====
=========================== short test summary info ============================
FAILED test_pytest_rerunfailures.py::test_run_session_teardown_once_after_reruns
=================== 1 failed, 99 passed, 1 skipped in 4.17s ====================

https://docs.pytest.org/en/stable/changelog.html

For the build logs, see:
https://copr-be.cloud.fedoraproject.org/results/thrnciar/pytest/fedora-rawhide-x86_64/07389533-python-pytest-rerunfailures/

For all our attempts to build python-pytest-rerunfailures with pytest 8, see:
https://copr.fedorainfracloud.org/coprs/thrnciar/pytest/package/python-pytest-rerunfailures/

Let us know here if you have any questions.

Pytest 8 is planned to be included in Fedora 41. And this bugzilla is a
heads up before we merge new pytest into rawhide. For more info see a Fedora Change
proposal https://fedoraproject.org/wiki/Changes/Pytest_8

We'd appreciate help from the people who know this package best,
but if you don't want to work on this now, let us know so we can try to work around it on our side.
Comment 1 Tomáš Hrnčiar 2024-11-20 07:52:15 UTC 
This problem was fixed upstream, updating to 15.0 should help.
Comment 2 Fedora Update System 2024-12-22 19:11:11 UTC 
FEDORA-2024-ffdd7a5aae (python-pytest-rerunfailures-15.0-1.fc41) has been submitted as an update to Fedora 41.
https://bodhi.fedoraproject.org/updates/FEDORA-2024-ffdd7a5aae
Comment 3 Fedora Update System 2024-12-23 02:00:33 UTC 
FEDORA-2024-ffdd7a5aae has been pushed to the Fedora 41 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2024-ffdd7a5aae`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2024-ffdd7a5aae

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.
Comment 4 Fedora Update System 2024-12-31 03:10:55 UTC 
FEDORA-2024-ffdd7a5aae (python-pytest-rerunfailures-15.0-1.fc41) has been pushed to the Fedora 41 stable repository.
If problem still persists, please make note of it in this bug report.
Note You need to log in before you can comment on or make changes to this bug.