Bug 2166612 - python-TestSlide FTBFS with Pygments 2.14
Summary: python-TestSlide FTBFS with Pygments 2.14
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: python-TestSlide
Version: 38
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Davide Cavalca
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On: 2179100
Blocks: 2157264
TreeView+ depends on / blocked
 
Reported: 2023-02-02 10:35 UTC by Karolina Surma
Modified: 2023-07-07 01:28 UTC (History)
4 users (show)

Fixed In Version: python-TestSlide-2.7.1-1.fc39 python-TestSlide-2.7.1-2.fc38
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2023-06-26 16:51:07 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github facebook TestSlide issues 345 0 None open Test failures with Pygments 2.14 2023-02-03 12:28:02 UTC

Description Karolina Surma 2023-02-02 10:35:49 UTC
Description of problem:
python-TestSlide fails to build with Pygments 2.14 we want to bring to Rawhide. 
It's likely related to some output changes brought by new Pygments. I checked the impact in a copr repository, see links below.

Version-Release number of selected component (if applicable):
2.7.0-7

How reproducible:
Always

Steps to Reproduce:
$ mock -r fedora-rawhide-x86_64 --addrepo=https://download.copr.fedorainfracloud.org/results/ksurma/pygments-2.14/fedora-rawhide-x86_64/ --no-clean your.src.rpm
$ mock -r fedora-rawhide-x86_64 --addrepo=https://download.copr.fedorainfracloud.org/results/ksurma/pygments-2.14/fedora-rawhide-x86_64/  shell

Actual results:
https://download.copr.fedorainfracloud.org/results/ksurma/pygments-2.14/fedora-rawhide-x86_64/05362858-python-TestSlide/builder-live.log.gz


=================================== FAILURES ===================================
__________ TestCliDocumentFormatter.test_prints_exceptions_with_cause __________

self = <tests.cli_unittest.TestCliDocumentFormatter testMethod=test_prints_exceptions_with_cause>

    def test_prints_exceptions_with_cause(self):
>       self.run_testslide(
            tty_stdout=True,
            expected_return_code=1,
            expected_in_stdout=(
                '      File \033[36m"tests/sample_tests.py"\033[39;49;00m, line \033[34m76\033[39;49;00m, in test_failing\r\n'
                '        \033[34mraise\033[39;49;00m \033[36mAssertionError\033[39;49;00m(\033[33m"\033[39;49;00m\033[33mThird\033[39;49;00m\033[33m"\033[39;49;00m) \033[34mfrom\033[39;49;00m \033[04m\033[36mcause\033[39;49;00m\r\n'
                "\033[0m\033[31m      Caused by \033[0m\033[0m\033[31mAssertionError: Second\033[0m\r\n"
                '        File \033[36m"tests/sample_tests.py"\033[39;49;00m, line \033[34m74\033[39;49;00m, in test_failing\r\n'
                '          \033[34mraise\033[39;49;00m \033[36mAssertionError\033[39;49;00m(\033[33m"\033[39;49;00m\033[33mSecond\033[39;49;00m\033[33m"\033[39;49;00m) \033[34mfrom\033[39;49;00m \033[04m\033[36mcause\033[39;49;00m\r\n'
                "\033[0m\033[31m        Caused by \033[0m\033[0m\033[31mAssertionError: First\033[0m\r\n"
                '          File \033[36m"tests/sample_tests.py"\033[39;49;00m, line \033[34m72\033[39;49;00m, in test_failing\r\n'
                '            \033[34mraise\033[39;49;00m \033[36mAssertionError\033[39;49;00m(\033[33m"\033[39;49;00m\033[33mFirst\033[39;49;00m\033[33m"\033[39;49;00m)\r\n'
            ),
        )

tests/cli_unittest.py:284: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/cli_unittest.py:137: in run_testslide
    self.assertTrue(
E   AssertionError: False is not true : Command ['/usr/bin/python3', '-m', 'testslide.cli', '--show-testslide-stack-trace', '--format', 'documentation', '/builddir/build/BUILD/TestSlide-2.7.0/tests/sample_tests.py'] expected to have have in its stdout:
E   
E         File "tests/sample_tests.py", line 76, in test_failing
E           raise AssertionError("Third") from cause
E         Caused by AssertionError: Second
E           File "tests/sample_tests.py", line 74, in test_failing
E             raise AssertionError("Second") from cause
E           Caused by AssertionError: First
E             File "tests/sample_tests.py", line 72, in test_failing
E               raise AssertionError("First")
E   
E   
E   But output was different:
E   top context
E     passing example
E     failing example: SimulatedFailure: test failure (extra)
E     *focused example
E     skipped example
E     unittest SkipTest
E     nested context
E       passing nested example
E   tests.sample_tests.SampleTestCase
E     test_failing: AssertionError: Third
E     test_passing
E     test_skipped
E   
E   Failures:
E   
E     1) top context: failing example
E       1) SimulatedFailure: test failure (extra)
E         File "testslide/runner.py", line 789, in run
E           self._run_example(example)
E         File "testslide/runner.py", line 773, in _run_example
E           _ExampleRunner(example, self.formatter).run()
E         File "testslide/__init__.py", line 540, in run
E           self._sync_run_all_hooks_and_example(context_data)
E         File "testslide/__init__.py", line 512, in _sync_run_all_hooks_and_example
E           aggregated_exceptions.raise_correct_exception()
E         File "testslide/__init__.py", line 289, in raise_correct_exception
E           raise self.exceptions[0]
E         File "testslide/__init__.py", line 273, in catch
E           yield
E         File "testslide/__init__.py", line 503, in _sync_run_all_hooks_and_example
E           self._fail_if_coroutine_function(self.example.code, context_data)
E         File "testslide/__init__.py", line 471, in _fail_if_coroutine_function
E           return func(*args, **kwargs)
E         File "tests/sample_tests.py", line 42, in failing_example
E           raise SimulatedFailure("test failure", "(extra)")
E   
E     2) tests.sample_tests.SampleTestCase: test_failing
E       1) AssertionError: Third
E         File "testslide/runner.py", line 789, in run
E           self._run_example(example)
E         File "testslide/runner.py", line 773, in _run_example
E           _ExampleRunner(example, self.formatter).run()
E         File "testslide/__init__.py", line 540, in run
E           self._sync_run_all_hooks_and_example(context_data)
E         File "testslide/__init__.py", line 512, in _sync_run_all_hooks_and_example
E           aggregated_exceptions.raise_correct_exception()
E         File "testslide/__init__.py", line 289, in raise_correct_exception
E           raise self.exceptions[0]
E         File "testslide/__init__.py", line 273, in catch
E           yield
E         File "testslide/__init__.py", line 503, in _sync_run_all_hooks_and_example
E           self._fail_if_coroutine_function(self.example.code, context_data)
E         File "testslide/__init__.py", line 471, in _fail_if_coroutine_function
E           return func(*args, **kwargs)
E         File "testslide/cli.py", line 121, in example_code
E           with test_result() as result:
E         File "/usr/lib64/python3.11/contextlib.py", line 144, in __exit__
E           next(self.gen)
E         File "testslide/cli.py", line 110, in test_result
E           result.aggregated_exceptions.raise_correct_exception()
E         File "testslide/__init__.py", line 289, in raise_correct_exception
E           raise self.exceptions[0]
E         File "/usr/lib64/python3.11/unittest/case.py", line 57, in testPartExecutor
E           yield
E         File "/usr/lib64/python3.11/unittest/case.py", line 623, in run
E           self._callTestMethod(testMethod)
E         File "/usr/lib64/python3.11/unittest/case.py", line 579, in _callTestMethod
E           if method() is not None:
E         File "tests/sample_tests.py", line 76, in test_failing
E           raise AssertionError("Third") from cause
E         Caused by AssertionError: Second
E           File "tests/sample_tests.py", line 74, in test_failing
E             raise AssertionError("Second") from cause
E           Caused by AssertionError: First
E             File "tests/sample_tests.py", line 72, in test_failing
E               raise AssertionError("First")
E   
E   Executed 9 examples in 0.3s:
E     Successful: 4
E     Failed: 2
E     Skipped: 3
E     Not executed: 0
E   https://testslide.readthedocs.io/
__________ TestCliProgressFormatter.test_prints_exceptions_with_cause __________

self = <tests.cli_unittest.TestCliProgressFormatter testMethod=test_prints_exceptions_with_cause>

    def test_prints_exceptions_with_cause(self):
>       self.run_testslide(
            tty_stdout=True,
            expected_return_code=1,
            expected_in_stdout=(
                '      File \033[36m"tests/sample_tests.py"\033[39;49;00m, line \033[34m76\033[39;49;00m, in test_failing\r\n'
                '        \033[34mraise\033[39;49;00m \033[36mAssertionError\033[39;49;00m(\033[33m"\033[39;49;00m\033[33mThird\033[39;49;00m\033[33m"\033[39;49;00m) \033[34mfrom\033[39;49;00m \033[04m\033[36mcause\033[39;49;00m\r\n'
                "\033[0m\033[31m      Caused by \033[0m\033[0m\033[31mAssertionError: Second\033[0m\r\n"
                '        File \033[36m"tests/sample_tests.py"\033[39;49;00m, line \033[34m74\033[39;49;00m, in test_failing\r\n'
                '          \033[34mraise\033[39;49;00m \033[36mAssertionError\033[39;49;00m(\033[33m"\033[39;49;00m\033[33mSecond\033[39;49;00m\033[33m"\033[39;49;00m) \033[34mfrom\033[39;49;00m \033[04m\033[36mcause\033[39;49;00m\r\n'
                "\033[0m\033[31m        Caused by \033[0m\033[0m\033[31mAssertionError: First\033[0m\r\n"
                '          File \033[36m"tests/sample_tests.py"\033[39;49;00m, line \033[34m72\033[39;49;00m, in test_failing\r\n'
                '            \033[34mraise\033[39;49;00m \033[36mAssertionError\033[39;49;00m(\033[33m"\033[39;49;00m\033[33mFirst\033[39;49;00m\033[33m"\033[39;49;00m)\r\n'
            ),
        )

tests/cli_unittest.py:284: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/cli_unittest.py:137: in run_testslide
    self.assertTrue(
E   AssertionError: False is not true : Command ['/usr/bin/python3', '-m', 'testslide.cli', '--show-testslide-stack-trace', '/builddir/build/BUILD/TestSlide-2.7.0/tests/sample_tests.py', '--format', 'progress'] expected to have have in its stdout:
E   
E         File "tests/sample_tests.py", line 76, in test_failing
E           raise AssertionError("Third") from cause
E         Caused by AssertionError: Second
E           File "tests/sample_tests.py", line 74, in test_failing
E             raise AssertionError("Second") from cause
E           Caused by AssertionError: First
E             File "tests/sample_tests.py", line 72, in test_failing
E               raise AssertionError("First")
E   
E   
E   But output was different:
E   .F.SS.F.S
E   Failures:
E   
E     1) top context: failing example
E       1) SimulatedFailure: test failure (extra)
E         File "testslide/runner.py", line 789, in run
E           self._run_example(example)
E         File "testslide/runner.py", line 773, in _run_example
E           _ExampleRunner(example, self.formatter).run()
E         File "testslide/__init__.py", line 540, in run
E           self._sync_run_all_hooks_and_example(context_data)
E         File "testslide/__init__.py", line 512, in _sync_run_all_hooks_and_example
E           aggregated_exceptions.raise_correct_exception()
E         File "testslide/__init__.py", line 289, in raise_correct_exception
E           raise self.exceptions[0]
E         File "testslide/__init__.py", line 273, in catch
E           yield
E         File "testslide/__init__.py", line 503, in _sync_run_all_hooks_and_example
E           self._fail_if_coroutine_function(self.example.code, context_data)
E         File "testslide/__init__.py", line 471, in _fail_if_coroutine_function
E           return func(*args, **kwargs)
E         File "tests/sample_tests.py", line 42, in failing_example
E           raise SimulatedFailure("test failure", "(extra)")
E   
E     2) tests.sample_tests.SampleTestCase: test_failing
E       1) AssertionError: Third
E         File "testslide/runner.py", line 789, in run
E           self._run_example(example)
E         File "testslide/runner.py", line 773, in _run_example
E           _ExampleRunner(example, self.formatter).run()
E         File "testslide/__init__.py", line 540, in run
E           self._sync_run_all_hooks_and_example(context_data)
E         File "testslide/__init__.py", line 512, in _sync_run_all_hooks_and_example
E           aggregated_exceptions.raise_correct_exception()
E         File "testslide/__init__.py", line 289, in raise_correct_exception
E           raise self.exceptions[0]
E         File "testslide/__init__.py", line 273, in catch
E           yield
E         File "testslide/__init__.py", line 503, in _sync_run_all_hooks_and_example
E           self._fail_if_coroutine_function(self.example.code, context_data)
E         File "testslide/__init__.py", line 471, in _fail_if_coroutine_function
E           return func(*args, **kwargs)
E         File "testslide/cli.py", line 121, in example_code
E           with test_result() as result:
E         File "/usr/lib64/python3.11/contextlib.py", line 144, in __exit__
E           next(self.gen)
E         File "testslide/cli.py", line 110, in test_result
E           result.aggregated_exceptions.raise_correct_exception()
E         File "testslide/__init__.py", line 289, in raise_correct_exception
E           raise self.exceptions[0]
E         File "/usr/lib64/python3.11/unittest/case.py", line 57, in testPartExecutor
E           yield
E         File "/usr/lib64/python3.11/unittest/case.py", line 623, in run
E           self._callTestMethod(testMethod)
E         File "/usr/lib64/python3.11/unittest/case.py", line 579, in _callTestMethod
E           if method() is not None:
E         File "tests/sample_tests.py", line 76, in test_failing
E           raise AssertionError("Third") from cause
E         Caused by AssertionError: Second
E           File "tests/sample_tests.py", line 74, in test_failing
E             raise AssertionError("Second") from cause
E           Caused by AssertionError: First
E             File "tests/sample_tests.py", line 72, in test_failing
E               raise AssertionError("First")
____________ TestCliLongFormatter.test_prints_exceptions_with_cause ____________

self = <tests.cli_unittest.TestCliLongFormatter testMethod=test_prints_exceptions_with_cause>

    def test_prints_exceptions_with_cause(self):
>       self.run_testslide(
            tty_stdout=True,
            expected_return_code=1,
            expected_in_stdout=(
                '      File \033[36m"tests/sample_tests.py"\033[39;49;00m, line \033[34m76\033[39;49;00m, in test_failing\r\n'
                '        \033[34mraise\033[39;49;00m \033[36mAssertionError\033[39;49;00m(\033[33m"\033[39;49;00m\033[33mThird\033[39;49;00m\033[33m"\033[39;49;00m) \033[34mfrom\033[39;49;00m \033[04m\033[36mcause\033[39;49;00m\r\n'
                "\033[0m\033[31m      Caused by \033[0m\033[0m\033[31mAssertionError: Second\033[0m\r\n"
                '        File \033[36m"tests/sample_tests.py"\033[39;49;00m, line \033[34m74\033[39;49;00m, in test_failing\r\n'
                '          \033[34mraise\033[39;49;00m \033[36mAssertionError\033[39;49;00m(\033[33m"\033[39;49;00m\033[33mSecond\033[39;49;00m\033[33m"\033[39;49;00m) \033[34mfrom\033[39;49;00m \033[04m\033[36mcause\033[39;49;00m\r\n'
                "\033[0m\033[31m        Caused by \033[0m\033[0m\033[31mAssertionError: First\033[0m\r\n"
                '          File \033[36m"tests/sample_tests.py"\033[39;49;00m, line \033[34m72\033[39;49;00m, in test_failing\r\n'
                '            \033[34mraise\033[39;49;00m \033[36mAssertionError\033[39;49;00m(\033[33m"\033[39;49;00m\033[33mFirst\033[39;49;00m\033[33m"\033[39;49;00m)\r\n'
            ),
        )

tests/cli_unittest.py:284: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/cli_unittest.py:137: in run_testslide
    self.assertTrue(
E   AssertionError: False is not true : Command ['/usr/bin/python3', '-m', 'testslide.cli', '--show-testslide-stack-trace', '--format', 'long', '/builddir/build/BUILD/TestSlide-2.7.0/tests/sample_tests.py'] expected to have have in its stdout:
E   
E         File "tests/sample_tests.py", line 76, in test_failing
E           raise AssertionError("Third") from cause
E         Caused by AssertionError: Second
E           File "tests/sample_tests.py", line 74, in test_failing
E             raise AssertionError("Second") from cause
E           Caused by AssertionError: First
E             File "tests/sample_tests.py", line 72, in test_failing
E               raise AssertionError("First")
E   
E   
E   But output was different:
E   top context: passing example
E   top context: failing example: SimulatedFailure: test failure (extra)
E   top context: *focused example
E   top context: skipped example
E   top context: unittest SkipTest
E   top context, nested context: passing nested example
E   tests.sample_tests.SampleTestCase: test_failing: AssertionError: Third
E   tests.sample_tests.SampleTestCase: test_passing
E   tests.sample_tests.SampleTestCase: test_skipped
E   
E   Failures:
E   
E     1) top context: failing example
E       1) SimulatedFailure: test failure (extra)
E         File "testslide/runner.py", line 789, in run
E           self._run_example(example)
E         File "testslide/runner.py", line 773, in _run_example
E           _ExampleRunner(example, self.formatter).run()
E         File "testslide/__init__.py", line 540, in run
E           self._sync_run_all_hooks_and_example(context_data)
E         File "testslide/__init__.py", line 512, in _sync_run_all_hooks_and_example
E           aggregated_exceptions.raise_correct_exception()
E         File "testslide/__init__.py", line 289, in raise_correct_exception
E           raise self.exceptions[0]
E         File "testslide/__init__.py", line 273, in catch
E           yield
E         File "testslide/__init__.py", line 503, in _sync_run_all_hooks_and_example
E           self._fail_if_coroutine_function(self.example.code, context_data)
E         File "testslide/__init__.py", line 471, in _fail_if_coroutine_function
E           return func(*args, **kwargs)
E         File "tests/sample_tests.py", line 42, in failing_example
E           raise SimulatedFailure("test failure", "(extra)")
E   
E     2) tests.sample_tests.SampleTestCase: test_failing
E       1) AssertionError: Third
E         File "testslide/runner.py", line 789, in run
E           self._run_example(example)
E         File "testslide/runner.py", line 773, in _run_example
E           _ExampleRunner(example, self.formatter).run()
E         File "testslide/__init__.py", line 540, in run
E           self._sync_run_all_hooks_and_example(context_data)
E         File "testslide/__init__.py", line 512, in _sync_run_all_hooks_and_example
E           aggregated_exceptions.raise_correct_exception()
E         File "testslide/__init__.py", line 289, in raise_correct_exception
E           raise self.exceptions[0]
E         File "testslide/__init__.py", line 273, in catch
E           yield
E         File "testslide/__init__.py", line 503, in _sync_run_all_hooks_and_example
E           self._fail_if_coroutine_function(self.example.code, context_data)
E         File "testslide/__init__.py", line 471, in _fail_if_coroutine_function
E           return func(*args, **kwargs)
E         File "testslide/cli.py", line 121, in example_code
E           with test_result() as result:
E         File "/usr/lib64/python3.11/contextlib.py", line 144, in __exit__
E           next(self.gen)
E         File "testslide/cli.py", line 110, in test_result
E           result.aggregated_exceptions.raise_correct_exception()
E         File "testslide/__init__.py", line 289, in raise_correct_exception
E           raise self.exceptions[0]
E         File "/usr/lib64/python3.11/unittest/case.py", line 57, in testPartExecutor
E           yield
E         File "/usr/lib64/python3.11/unittest/case.py", line 623, in run
E           self._callTestMethod(testMethod)
E         File "/usr/lib64/python3.11/unittest/case.py", line 579, in _callTestMethod
E           if method() is not None:
E         File "tests/sample_tests.py", line 76, in test_failing
E           raise AssertionError("Third") from cause
E         Caused by AssertionError: Second
E           File "tests/sample_tests.py", line 74, in test_failing
E             raise AssertionError("Second") from cause
E           Caused by AssertionError: First
E             File "tests/sample_tests.py", line 72, in test_failing
E               raise AssertionError("First")
E   
E   Executed 9 examples in 0.3s:
E     Successful: 4
E     Failed: 2
E     Skipped: 3
E     Not executed: 0
E   https://testslide.readthedocs.io/
=========================== short test summary info ============================
FAILED tests/cli_unittest.py::TestCliDocumentFormatter::test_prints_exceptions_with_cause
FAILED tests/cli_unittest.py::TestCliProgressFormatter::test_prints_exceptions_with_cause
FAILED tests/cli_unittest.py::TestCliLongFormatter::test_prints_exceptions_with_cause

Expected results:
Success

Comment 1 Davide Cavalca 2023-02-03 12:28:03 UTC
Reported upstream in https://github.com/facebook/TestSlide/issues/345

Comment 2 Ben Cotton 2023-02-07 15:12:24 UTC
This bug appears to have been reported against 'rawhide' during the Fedora Linux 38 development cycle.
Changing version to 38.

Comment 4 Fedora Update System 2023-06-26 16:48:11 UTC
FEDORA-2023-a2d687e102 has been submitted as an update to Fedora 39. https://bodhi.fedoraproject.org/updates/FEDORA-2023-a2d687e102

Comment 5 Fedora Update System 2023-06-26 16:51:07 UTC
FEDORA-2023-a2d687e102 has been pushed to the Fedora 39 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 6 Fedora Update System 2023-06-26 17:40:52 UTC
FEDORA-2023-becc1c0336 has been submitted as an update to Fedora 38. https://bodhi.fedoraproject.org/updates/FEDORA-2023-becc1c0336

Comment 7 Fedora Update System 2023-06-27 10:55:39 UTC
FEDORA-2023-becc1c0336 has been pushed to the Fedora 38 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2023-becc1c0336`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2023-becc1c0336

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 8 Fedora Update System 2023-06-29 01:46:14 UTC
FEDORA-2023-becc1c0336 has been pushed to the Fedora 38 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2023-becc1c0336`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2023-becc1c0336

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 9 Fedora Update System 2023-07-07 01:28:33 UTC
FEDORA-2023-becc1c0336 has been pushed to the Fedora 38 stable repository.
If problem still persists, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.