Bug 2262833

Summary: python-sybil fails to build with Python 3.13: test_functional.py fail with: ValueError("Invalid pattern: \'**\' can only be an entire path component")
Product: [Fedora] Fedora Reporter: Karolina Surma <ksurma>
Component: python-sybilAssignee: Fabian Affolter <mail>
Status: CLOSED RAWHIDE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: rawhideCC: ksurma, mail, mhroncok, orion
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2024-02-10 15:13:34 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 2244836    

Description Karolina Surma 2024-02-05 16:32:16 UTC
python-sybil fails to build with Python 3.13.0a3.


=================================== FAILURES ===================================
_____________________ test_filter_fnmatch_pattern[pytest] ______________________

tmpdir = local('/tmp/pytest-of-mockbuild/pytest-0/test_filter_fnmatch_pattern_py0')
capsys = <_pytest.capture.CaptureFixture object at 0x7f0177ba94c0>
runner = 'pytest'

    @pytest.mark.parametrize('runner', [PYTEST, UNITTEST])
    def test_filter_fnmatch_pattern(tmpdir: local, capsys: CaptureFixture[str], runner: str):
        make_tree(tmpdir)
        write_config(tmpdir, runner, pattern="'**/*.rst'")
        results = run(capsys, runner, tmpdir)
        # The fact that the two .rst files in the root aren't matched is
        # arguably a bug in the Python interpretation of **/
        results.out.assert_has_run(runner, '/parent/foo.rst')
        results.out.assert_has_run(runner, '/parent/bar.rst')
        results.out.assert_has_run(runner, '/parent/child/foo.rst')
        results.out.assert_has_run(runner, '/parent/child/bar.rst')
>       assert results.total == 4, results.out.text
E       AssertionError: ============================= test session starts ==============================
E         platform linux -- Python 3.13.0a3, pytest-7.4.3, pluggy-1.3.0 -- /usr/bin/python3
E         cachedir: .pytest_cache
E         rootdir: /tmp/pytest-of-mockbuild/pytest-0/test_filter_fnmatch_pattern_py0
E         plugins: cov-4.0.0
E         collecting ... collected 6 items
E         
E         ../../../../tmp/pytest-of-mockbuild/pytest-0/test_filter_fnmatch_pattern_py0/bar.rst::line:1,column:1 PASSED
E         ../../../../tmp/pytest-of-mockbuild/pytest-0/test_filter_fnmatch_pattern_py0/foo.rst::line:1,column:1 PASSED
E         ../../../../tmp/pytest-of-mockbuild/pytest-0/test_filter_fnmatch_pattern_py0/parent/bar.rst::line:1,column:1 PASSED
E         ../../../../tmp/pytest-of-mockbuild/pytest-0/test_filter_fnmatch_pattern_py0/parent/foo.rst::line:1,column:1 PASSED
E         ../../../../tmp/pytest-of-mockbuild/pytest-0/test_filter_fnmatch_pattern_py0/parent/child/bar.rst::line:1,column:1 PASSED
E         ../../../../tmp/pytest-of-mockbuild/pytest-0/test_filter_fnmatch_pattern_py0/parent/child/foo.rst::line:1,column:1 PASSED
E         
E         ============================== 6 passed in 0.01s ===============================
E         
E       assert 6 == 4
E        +  where 6 = <tests.helpers.Results object at 0x7f0179825640>.total

tests/test_functional.py:168: AssertionError
____________________ test_filter_fnmatch_pattern[unittest] _____________________

tmpdir = local('/tmp/pytest-of-mockbuild/pytest-0/test_filter_fnmatch_pattern_un0')
capsys = <_pytest.capture.CaptureFixture object at 0x7f0177c73ad0>
runner = 'unittest'

    @pytest.mark.parametrize('runner', [PYTEST, UNITTEST])
    def test_filter_fnmatch_pattern(tmpdir: local, capsys: CaptureFixture[str], runner: str):
        make_tree(tmpdir)
        write_config(tmpdir, runner, pattern="'**/*.rst'")
        results = run(capsys, runner, tmpdir)
        # The fact that the two .rst files in the root aren't matched is
        # arguably a bug in the Python interpretation of **/
        results.out.assert_has_run(runner, '/parent/foo.rst')
        results.out.assert_has_run(runner, '/parent/bar.rst')
        results.out.assert_has_run(runner, '/parent/child/foo.rst')
        results.out.assert_has_run(runner, '/parent/child/bar.rst')
>       assert results.total == 4, results.out.text
E       AssertionError: /tmp/pytest-of-mockbuild/pytest-0/test_filter_fnmatch_pattern_un0/bar.rst,line:1,column:1 ... ok
E         /tmp/pytest-of-mockbuild/pytest-0/test_filter_fnmatch_pattern_un0/foo.rst,line:1,column:1 ... ok
E         /tmp/pytest-of-mockbuild/pytest-0/test_filter_fnmatch_pattern_un0/parent/bar.rst,line:1,column:1 ... ok
E         /tmp/pytest-of-mockbuild/pytest-0/test_filter_fnmatch_pattern_un0/parent/child/bar.rst,line:1,column:1 ... ok
E         /tmp/pytest-of-mockbuild/pytest-0/test_filter_fnmatch_pattern_un0/parent/child/foo.rst,line:1,column:1 ... ok
E         /tmp/pytest-of-mockbuild/pytest-0/test_filter_fnmatch_pattern_un0/parent/foo.rst,line:1,column:1 ... ok
E         
E         ----------------------------------------------------------------------
E         Ran 6 tests in 0.001s
E         
E         OK
E         
E       assert 6 == 4
E        +  where 6 = <tests.helpers.Results object at 0x7f0179a6e9c0>.total

tests/test_functional.py:168: AssertionError
__________________ test_filter_filenames_and_excludes[pytest] __________________

tmpdir = local('/tmp/pytest-of-mockbuild/pytest-0/test_filter_filenames_and_excl0')
capsys = <_pytest.capture.CaptureFixture object at 0x7f0177bc8770>
runner = 'pytest'

    @pytest.mark.parametrize('runner', [PYTEST, UNITTEST])
    def test_filter_filenames_and_excludes(tmpdir: local, capsys: CaptureFixture[str], runner: str):
        make_tree(tmpdir)
        write_config(tmpdir, runner,
                     path=f"'{tmpdir / 'parent'}'",
                     filenames="{'bar.rst'}",
                     excludes="['**child/*.rst']")
        results = run(capsys, runner, tmpdir)
>       results.out.assert_has_run(runner, '/parent/bar.rst')

tests/test_functional.py:218: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/helpers.py:108: in assert_has_run
    self.assert_present(TEST_OUTPUT_TEMPLATES[integration].format(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <tests.helpers.Finder object at 0x7f01779a2c00>
text = '/parent/bar.rst::line:1,column:1'

    def assert_present(self, text):
>       assert text in self.text, f'{self.text}\n{self.text!r}'
E       AssertionError: ============================= test session starts ==============================
E       platform linux -- Python 3.13.0a3, pytest-7.4.3, pluggy-1.3.0 -- /usr/bin/python3
E       cachedir: .pytest_cache
E       rootdir: /tmp/pytest-of-mockbuild/pytest-0/test_filter_filenames_and_excl0
E       plugins: cov-4.0.0
E       collecting ... collected 0 items / 1 error
E       
E       ==================================== ERRORS ====================================
E       ________________________ ERROR collecting test session _________________________
E       /usr/lib/python3.13/site-packages/_pytest/runner.py:341: in from_call
E           result: Optional[TResult] = func()
E       /usr/lib/python3.13/site-packages/_pytest/runner.py:372: in <lambda>
E           call = CallInfo.from_call(lambda: list(collector.collect()), "collect")
E       /usr/lib/python3.13/site-packages/_pytest/main.py:749: in collect
E           for x in self._collectfile(path):
E       /usr/lib/python3.13/site-packages/_pytest/main.py:602: in _collectfile
E           return ihook.pytest_collect_file(file_path=fspath, parent=self)  # type: ignore[no-any-return]
E       /usr/lib/python3.13/site-packages/_pytest/config/compat.py:66: in fixed_hook
E           return hook(**kw)
E       /usr/lib/python3.13/site-packages/pluggy/_hooks.py:493: in __call__
E           return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
E       /usr/lib/python3.13/site-packages/pluggy/_manager.py:115: in _hookexec
E           return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
E       sybil/integration/pytest.py:135: in pytest_collect_file
E           if sybil.should_parse(path):
E       sybil/sybil.py:138: in should_parse
E           if any(path.match(e) for e in self.excludes):
E       sybil/sybil.py:138: in <genexpr>
E           if any(path.match(e) for e in self.excludes):
E       /usr/lib64/python3.13/pathlib/_abc.py:446: in match
E           match = _compile_pattern(pattern_str, sep, case_sensitive)
E       /usr/lib64/python3.13/pathlib/_abc.py:59: in _compile_pattern
E           regex = glob.translate(pat, recursive=True, include_hidden=True, seps=sep)
E       /usr/lib64/python3.13/glob.py:303: in translate
E           raise ValueError("Invalid pattern: '**' can only be an entire path component")
E       E   ValueError: Invalid pattern: '**' can only be an entire path component
E       =========================== short test summary info ============================
E       ERROR ../../../../tmp/pytest-of-mockbuild/pytest-0/test_filter_filenames_and_excl0
E       !!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
E       =============================== 1 error in 0.10s ===============================
E       
E       '============================= test session starts ==============================\nplatform linux -- Python 3.13.0a3, pytest-7.4.3, pluggy-1.3.0 -- /usr/bin/python3\ncachedir: .pytest_cache\nrootdir: /tmp/pytest-of-mockbuild/pytest-0/test_filter_filenames_and_excl0\nplugins: cov-4.0.0\ncollecting ... collected 0 items / 1 error\n\n==================================== ERRORS ====================================\n________________________ ERROR collecting test session _________________________\n/usr/lib/python3.13/site-packages/_pytest/runner.py:341: in from_call\n    result: Optional[TResult] = func()\n/usr/lib/python3.13/site-packages/_pytest/runner.py:372: in <lambda>\n    call = CallInfo.from_call(lambda: list(collector.collect()), "collect")\n/usr/lib/python3.13/site-packages/_pytest/main.py:749: in collect\n    for x in self._collectfile(path):\n/usr/lib/python3.13/site-packages/_pytest/main.py:602: in _collectfile\n    return ihook.pytest_collect_file(file_path=fspath, parent=self)  # type: ignore[no-any-return]\n/usr/lib/python3.13/site-packages/_pytest/config/compat.py:66: in fixed_hook\n    return hook(**kw)\n/usr/lib/python3.13/site-packages/pluggy/_hooks.py:493: in __call__\n    return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)\n/usr/lib/python3.13/site-packages/pluggy/_manager.py:115: in _hookexec\n    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)\nsybil/integration/pytest.py:135: in pytest_collect_file\n    if sybil.should_parse(path):\nsybil/sybil.py:138: in should_parse\n    if any(path.match(e) for e in self.excludes):\nsybil/sybil.py:138: in <genexpr>\n    if any(path.match(e) for e in self.excludes):\n/usr/lib64/python3.13/pathlib/_abc.py:446: in match\n    match = _compile_pattern(pattern_str, sep, case_sensitive)\n/usr/lib64/python3.13/pathlib/_abc.py:59: in _compile_pattern\n    regex = glob.translate(pat, recursive=True, include_hidden=True, seps=sep)\n/usr/lib64/python3.13/glob.py:303: in translate\n    raise ValueError("Invalid pattern: \'**\' can only be an entire path component")\nE   ValueError: Invalid pattern: \'**\' can only be an entire path component\n=========================== short test summary info ============================\nERROR ../../../../tmp/pytest-of-mockbuild/pytest-0/test_filter_filenames_and_excl0\n!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!\n=============================== 1 error in 0.10s ===============================\n'

tests/helpers.py:100: AssertionError
_________________ test_filter_filenames_and_excludes[unittest] _________________

tmpdir = local('/tmp/pytest-of-mockbuild/pytest-0/test_filter_filenames_and_excl1')
capsys = <_pytest.capture.CaptureFixture object at 0x7f01779a00b0>
runner = 'unittest'

    @pytest.mark.parametrize('runner', [PYTEST, UNITTEST])
    def test_filter_filenames_and_excludes(tmpdir: local, capsys: CaptureFixture[str], runner: str):
        make_tree(tmpdir)
        write_config(tmpdir, runner,
                     path=f"'{tmpdir / 'parent'}'",
                     filenames="{'bar.rst'}",
                     excludes="['**child/*.rst']")
        results = run(capsys, runner, tmpdir)
>       results.out.assert_has_run(runner, '/parent/bar.rst')

tests/test_functional.py:218: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/helpers.py:108: in assert_has_run
    self.assert_present(TEST_OUTPUT_TEMPLATES[integration].format(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <tests.helpers.Finder object at 0x7f017797b950>
text = '/parent/bar.rst,line:1,column:1'

    def assert_present(self, text):
>       assert text in self.text, f'{self.text}\n{self.text!r}'
E       AssertionError: test_docs (unittest.loader._FailedTest.test_docs) ... ERROR
E       
E       ======================================================================
E       ERROR: test_docs (unittest.loader._FailedTest.test_docs)
E       ----------------------------------------------------------------------
E       Traceback (most recent call last):
E         File "/builddir/build/BUILD/sybil-5.0.3/sybil/integration/unittest.py", line 43, in load_tests
E           if path.is_file() and sybil.should_parse(path):
E                                 ~~~~~~~~~~~~~~~~~~^^^^^^
E         File "/builddir/build/BUILD/sybil-5.0.3/sybil/sybil.py", line 138, in should_parse
E           if any(path.match(e) for e in self.excludes):
E              ~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
E         File "/builddir/build/BUILD/sybil-5.0.3/sybil/sybil.py", line 138, in <genexpr>
E           if any(path.match(e) for e in self.excludes):
E                  ~~~~~~~~~~^^^
E         File "/usr/lib64/python3.13/pathlib/_abc.py", line 446, in match
E           match = _compile_pattern(pattern_str, sep, case_sensitive)
E                   ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
E         File "/usr/lib64/python3.13/pathlib/_abc.py", line 59, in _compile_pattern
E           regex = glob.translate(pat, recursive=True, include_hidden=True, seps=sep)
E                   ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
E         File "/usr/lib64/python3.13/glob.py", line 303, in translate
E           raise ValueError("Invalid pattern: '**' can only be an entire path component")
E       ValueError: Invalid pattern: '**' can only be an entire path component
E       
E       ----------------------------------------------------------------------
E       Ran 1 test in 0.001s
E       
E       FAILED (errors=1)
E       
E       'test_docs (unittest.loader._FailedTest.test_docs) ... ERROR\n\n======================================================================\nERROR: test_docs (unittest.loader._FailedTest.test_docs)\n----------------------------------------------------------------------\nTraceback (most recent call last):\n  File "/builddir/build/BUILD/sybil-5.0.3/sybil/integration/unittest.py", line 43, in load_tests\n    if path.is_file() and sybil.should_parse(path):\n                          ~~~~~~~~~~~~~~~~~~^^^^^^\n  File "/builddir/build/BUILD/sybil-5.0.3/sybil/sybil.py", line 138, in should_parse\n    if any(path.match(e) for e in self.excludes):\n       ~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File "/builddir/build/BUILD/sybil-5.0.3/sybil/sybil.py", line 138, in <genexpr>\n    if any(path.match(e) for e in self.excludes):\n           ~~~~~~~~~~^^^\n  File "/usr/lib64/python3.13/pathlib/_abc.py", line 446, in match\n    match = _compile_pattern(pattern_str, sep, case_sensitive)\n            ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File "/usr/lib64/python3.13/pathlib/_abc.py", line 59, in _compile_pattern\n    regex = glob.translate(pat, recursive=True, include_hidden=True, seps=sep)\n            ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File "/usr/lib64/python3.13/glob.py", line 303, in translate\n    raise ValueError("Invalid pattern: \'**\' can only be an entire path component")\nValueError: Invalid pattern: \'**\' can only be an entire path component\n\n----------------------------------------------------------------------\nRan 1 test in 0.001s\n\nFAILED (errors=1)\n'

tests/helpers.py:100: AssertionError
=========================== short test summary info ============================
FAILED tests/test_functional.py::test_filter_fnmatch_pattern[pytest] - Assert...
FAILED tests/test_functional.py::test_filter_fnmatch_pattern[unittest] - Asse...
FAILED tests/test_functional.py::test_filter_filenames_and_excludes[pytest]
FAILED tests/test_functional.py::test_filter_filenames_and_excludes[unittest]
======================== 4 failed, 531 passed in 2.77s =========================


According to https://docs.python.org/dev/whatsnew/3.13.html#glob a new glob.translate() method has been added. It raises ValueError if “**” occurs in any position other than a full pattern segment: https://docs.python.org/dev/library/glob.html#glob.translate



For the build logs, see:
https://copr-be.cloud.fedoraproject.org/results/@python/python3.13/fedora-rawhide-x86_64/06986254-python-sybil/

For all our attempts to build python-sybil with Python 3.13, see:
https://copr.fedorainfracloud.org/coprs/g/python/python3.13/package/python-sybil/

Testing and mass rebuild of packages is happening in copr.
You can follow these instructions to test locally in mock if your package builds with Python 3.13:
https://copr.fedorainfracloud.org/coprs/g/python/python3.13/

Let us know here if you have any questions.

Python 3.13 is planned to be included in Fedora 41.
To make that update smoother, we're building Fedora packages with all pre-releases of Python 3.13.
A build failure prevents us from testing all dependent packages (transitive [Build]Requires),
so if this package is required a lot, it's important for us to get it fixed soon.

We'd appreciate help from the people who know this package best,
but if you don't want to work on this now, let us know so we can try to work around it on our side.

Comment 1 Miro Hrončok 2024-02-06 12:12:55 UTC
I've opened https://src.fedoraproject.org/rpms/python-sybil/pull-request/2