Bug 1935212

Summary: %_pyproject_builddir sometimes leaks to pytest collection
Product: [Fedora] Fedora Reporter: Miro Hrončok <mhroncok>
Component: pyproject-rpm-macrosAssignee: Miro Hrončok <mhroncok>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: rawhideCC: lbalhar, mhroncok, pviktori, python-sig
Target Milestone: ---Keywords: Reopened
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: pyproject-rpm-macros-0-41.fc35 pyproject-rpm-macros-0-42.fc34 pyproject-rpm-macros-0-42.fc33 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-07-04 01:07:15 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Miro Hrončok 2021-03-04 14:39:28 UTC 
For the purpose of debuginfo/debugsource generation, we instruct pip to build out-of-tree builds inside a hidden directory in the builddir. This directory is %_pyproject_builddir (defined to .pyproject-builddir).

When pytest collects tests, it ignores hidden directories by default, because the `.*` pattern is ignored.

However, if upstream sets https://docs.pytest.org/en/stable/reference.html#confval-norecursedirs it replaces the default. If upstream does not explicitly list `.*`, hidden directories are not ignored. E.g. like this:

https://github.com/jaraco/jaraco.path/blob/49088ff54e73ef86f0ecd2206b07931b5801973a/pytest.ini#L2

    norecursedirs=dist build .tox .eggs

When this happens, pytest collects the files from both %_pyproject_builddir and the actual location, causing a mismatch:
    _____________________ ERROR collecting tests/test_path.py ______________________
    /usr/lib/python3.9/site-packages/_pytest/runner.py:311: in from_call
        result: Optional[TResult] = func()
    /usr/lib/python3.9/site-packages/_pytest/runner.py:341: in <lambda>
        call = CallInfo.from_call(lambda: list(collector.collect()), "collect")
    /usr/lib/python3.9/site-packages/_pytest/doctest.py:532: in collect
        module = import_path(self.fspath)
    /usr/lib/python3.9/site-packages/_pytest/pathlib.py:551: in import_path
        raise ImportPathMismatchError(module_name, module_file, path)
    E   _pytest.pathlib.ImportPathMismatchError: ('test_path', '/builddir/build/BUILD/jaraco.path-3.3.1/.pyproject-builddir/pip-req-build-dc2m0ci1/tests/test_path.py', PosixPath('/builddir/build/BUILD/jaraco.path-3.3.1/tests/test_path.py'))


The way to currently workaround this in the specfile is to use:

    %pytest --ignore %_pyproject_builddir

Or, in the case of pytest in tox:

    %tox -- -- --ignore %_pyproject_builddir

We should make sure this is not necessary.
Comment 1 Miro Hrončok 2021-03-04 14:43:28 UTC 
A "simple" solution might be to pass PYTEST_ADDOPTS='--ignore %_pyproject_builddir' in %pytest and %tox.

https://docs.pytest.org/en/stable/reference.html#envvar-PYTEST_ADDOPTS


%tox is defined in pyproject-rpm-macros, the fix is trivial.


%pytest is defined in python-rpm-macros, so unless we want to spaghetti the two macro sets together, we can adapt it to do %{?_pytest_addopts:PYTEST_ADDOPTS='%{_pytest_addopts}'} and define %_pytest_addopts in pyproject_rpm_macros.
Comment 2 Lumír Balhar 2021-06-24 19:04:06 UTC 
Another related issue I've just met is when %_pyproject_builddir contains a copy of conftest.py which defines CLI arguments for pytest - then the arguments are defined twice which leads to: ValueError: option names {'--run-integration'} already added.
Comment 3 Miro Hrončok 2021-06-24 19:13:44 UTC 
(In reply to Lumír Balhar from comment #2)
> Another related issue I've just met is when %_pyproject_builddir contains a
> copy of conftest.py which defines CLI arguments for pytest - then the
> arguments are defined twice which leads to: ValueError: option names
> {'--run-integration'} already added.

Just to clarify, adding --ignore %_pyproject_builddir solves that problem as well?
Comment 4 Lumír Balhar 2021-06-25 07:29:49 UTC 
(In reply to Miro Hrončok from comment #3)
> (In reply to Lumír Balhar from comment #2)
> > Another related issue I've just met is when %_pyproject_builddir contains a
> > copy of conftest.py which defines CLI arguments for pytest - then the
> > arguments are defined twice which leads to: ValueError: option names
> > {'--run-integration'} already added.
> 
> Just to clarify, adding --ignore %_pyproject_builddir solves that problem as
> well?

Yes, sorry for not mentioning it before.
Comment 5 Miro Hrončok 2021-06-26 14:33:53 UTC 
https://src.fedoraproject.org/rpms/python-rpm-macros/pull-request/102
https://src.fedoraproject.org/rpms/pyproject-rpm-macros/pull-request/168
Comment 6 Fedora Update System 2021-06-28 19:54:42 UTC 
FEDORA-2021-872314c50e has been submitted as an update to Fedora 34. https://bodhi.fedoraproject.org/updates/FEDORA-2021-872314c50e
Comment 7 Fedora Update System 2021-06-28 19:54:46 UTC 
FEDORA-2021-f5e7993ed1 has been submitted as an update to Fedora 33. https://bodhi.fedoraproject.org/updates/FEDORA-2021-f5e7993ed1
Comment 8 Fedora Update System 2021-06-28 19:55:43 UTC 
FEDORA-2021-02f02941bb has been pushed to the Fedora 35 stable repository.
If problem still persists, please make note of it in this bug report.
Comment 9 Fedora Update System 2021-06-29 01:32:18 UTC 
FEDORA-2021-872314c50e has been pushed to the Fedora 34 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2021-872314c50e`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2021-872314c50e

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.
Comment 10 Fedora Update System 2021-06-29 02:09:43 UTC 
FEDORA-2021-f5e7993ed1 has been pushed to the Fedora 33 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2021-f5e7993ed1`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2021-f5e7993ed1

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.
Comment 11 Fedora Update System 2021-06-29 16:30:15 UTC 
FEDORA-2021-f5e7993ed1 has been submitted as an update to Fedora 33. https://bodhi.fedoraproject.org/updates/FEDORA-2021-f5e7993ed1
Comment 12 Fedora Update System 2021-06-30 14:19:51 UTC 
FEDORA-2021-f5e7993ed1 has been pushed to the Fedora 33 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2021-f5e7993ed1`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2021-f5e7993ed1

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.
Comment 13 Fedora Update System 2021-06-30 14:23:55 UTC 
FEDORA-2021-872314c50e has been pushed to the Fedora 34 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2021-872314c50e`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2021-872314c50e

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.
Comment 14 Fedora Update System 2021-07-04 01:07:15 UTC 
FEDORA-2021-872314c50e has been pushed to the Fedora 34 stable repository.
If problem still persists, please make note of it in this bug report.
Comment 15 Fedora Update System 2021-07-08 01:07:45 UTC 
FEDORA-2021-f5e7993ed1 has been pushed to the Fedora 33 stable repository.
If problem still persists, please make note of it in this bug report.
Comment 16 Miro Hrončok 2021-07-09 14:55:37 UTC 
Followup:

https://src.fedoraproject.org/rpms/python-jaraco-path/pull-request/1
https://src.fedoraproject.org/rpms/python-build/pull-request/1
https://src.fedoraproject.org/rpms/python-pipdeptree/pull-request/3
https://src.fedoraproject.org/rpms/python-junit_xml/pull-request/1