Bug 2166888 - %pyproject_buildrequires leaks output from subprocesses to stdout, creating bogus BuildRequires
Summary: %pyproject_buildrequires leaks output from subprocesses to stdout, creating b...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: pyproject-rpm-macros
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Miro Hrončok
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2023-02-03 11:20 UTC by Miro Hrončok
Modified: 2023-03-01 03:09 UTC (History)
4 users (show)

Fixed In Version: pyproject-rpm-macros-1.6.1-1.fc38 pyproject-rpm-macros-1.6.1-1.fc37 pyproject-rpm-macros-1.6.2-1.fc36
Clone Of:
Environment:
Last Closed: 2023-02-07 10:27:15 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Miro Hrončok 2023-02-03 11:20:05 UTC
The %pyproject_buildrequires macro is designed to be used in the %generate_buildrequires section. As such the entire stdout is treated as BuildRequires. Anything the macro needs to put into the logs must be put to stderr.

Since this macro calls third-party code, the implementation in pyproject_buildrequires.py uses contextlib.redirect_stdout to redirect all stdout from PEP 517 hooks to stderr.

Using contextlib.redirect_stdout however has limitations. It does not redirect stdout of subprocesses.

If a setup.py script has for example:

  rv = os.system('/usr/bin/patch -N -p3 -d build/lib < lib/py-lmdb/env-copy-txn.patch')

(From https://github.com/jnwatson/py-lmdb/blob/py-lmdb_1.0.0/setup.py#L117)


The stdout of /usr/bin/patch leaks to stdout of %pyproject_buildrequires:


  [lmdb-1.0.0]$ /usr/bin/python3 -Bs /usr/lib/rpm/redhat/pyproject_buildrequires.py --python3_pkgversion 3 2>/dev/null
  python3dist(setuptools) >= 40.8
  python3dist(wheel)
  patching file lmdb.h
  patching file mdb.c
  python3dist(wheel)
  patching file lmdb.h
  patching file mdb.c

This results in DNF errors like this:

  No matching package to install: 'lmdb.h'
  No matching package to install: 'mdb.c'
  No matching package to install: 'patching'

Moreover, it results in bogus BuildRequires that may exist (e.g. "file").


To fix this from within pyproject_buildrequires.py, we might need to play with file descriptors.

Alternatively, we might write the desired BuildRequires to a file instead of stdout and run the script with >&2 from within the macro, catting the generated file to stdout at the end.

Comment 1 Miro Hrončok 2023-02-03 12:42:45 UTC
A WIP pull request in https://src.fedoraproject.org/rpms/pyproject-rpm-macros/pull-request/353

Comment 2 Fedora Update System 2023-02-07 10:24:56 UTC
FEDORA-2023-e710cff39f has been submitted as an update to Fedora 38. https://bodhi.fedoraproject.org/updates/FEDORA-2023-e710cff39f

Comment 3 Fedora Update System 2023-02-07 10:27:15 UTC
FEDORA-2023-e710cff39f has been pushed to the Fedora 38 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 4 Fedora Update System 2023-02-07 10:27:49 UTC
FEDORA-2023-b9d9b8dfae has been submitted as an update to Fedora 37. https://bodhi.fedoraproject.org/updates/FEDORA-2023-b9d9b8dfae

Comment 5 Fedora Update System 2023-02-07 10:28:50 UTC
FEDORA-2023-f706deb47c has been submitted as an update to Fedora 36. https://bodhi.fedoraproject.org/updates/FEDORA-2023-f706deb47c

Comment 6 Fedora Update System 2023-02-08 01:29:17 UTC
FEDORA-2023-b9d9b8dfae has been pushed to the Fedora 37 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2023-b9d9b8dfae`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2023-b9d9b8dfae

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 7 Fedora Update System 2023-02-08 01:40:25 UTC
FEDORA-2023-f706deb47c has been pushed to the Fedora 36 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2023-f706deb47c`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2023-f706deb47c

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 8 Fedora Update System 2023-02-13 00:41:14 UTC
FEDORA-2023-b9d9b8dfae has been pushed to the Fedora 37 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 9 Fedora Update System 2023-02-14 03:25:10 UTC
FEDORA-2023-f706deb47c has been pushed to the Fedora 36 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2023-f706deb47c`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2023-f706deb47c

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 10 Fedora Update System 2023-03-01 03:09:35 UTC
FEDORA-2023-f706deb47c has been pushed to the Fedora 36 stable repository.
If problem still persists, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.