Bug 1685582

Summary: Implement macros for PEP 518 & PEP 517 build systems
Product: [Fedora] Fedora Reporter: Petr Viktorin (pviktori) <pviktori>
Component: python3Assignee: Patrik Kopkan <pkopkan>
Status: CLOSED RAWHIDE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 31CC: cstratak, dmalcolm, dominik, j, mcyprian, mhroncok, pviktori, rkuska, shcherbina.iryna, tomspur, torsava
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-05-29 07:15:37 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1732822    
Bug Blocks: 1335203, 1575270    

Description Petr Viktorin (pviktori) 2019-03-05 14:47:23 UTC
Python is moving away from Setuptools as the only realistically possible build system for packages. We should update Fedora packaging to the new standards.

Comment 1 Petr Viktorin (pviktori) 2019-03-05 14:47:48 UTC
Blocking a few bugs that we can fix together with this.

Comment 2 Jason Tibbitts 2019-03-05 15:48:46 UTC
But what actually needs to change?  What would a specfile look like (without any fancy macros) for something using each of these new buildsystems?  How uniform is the packaging expected to be?

Given an example spec and some idea of how much things are expected to deviate from it, we can start to cook up macros where they would be helpful and look at whether there would be benefit from various RPM features (including ones that aren't yet released).

Comment 3 Miro Hrončok 2019-03-05 16:33:47 UTC
We are working towards answers for those questions. I'm meeting with Petr IRL this week to discuss this.

> How uniform is the packaging expected to be?

The PEPs actually define an uniform layer on top of custom tools with fallback to the old de facto standard (setup.py with setuptools). So the answer is "very uniform".

Comment 4 Miro Hrončok 2019-03-14 09:30:59 UTC
This is more or less what we have:

%pyproject_wheel: <flags...> %{__python3} -m pip wheel --no-deps --use-pep517 --no-build-isolation --progress-bar off --verbose .
+ conditional --global-option and --build-option based on %{?py_setup_args} and/or other macros.

%pyproject_install: %{__python3} -m pip install --root %{buildroot}  --strip-file-prefix %{buildroot} --no-deps  --progress-bar off *.whl


Problems:

1) We cannot tell what's the wheel name yet. We need to ask pip to provide us the filename, so we can parse it form the output and store it for %pyproject_install. We should probably say this in non-normative "Recommendations for build frontends" in PEP 517.

2) The setup.py --executable option is gone and we need to figure out what to do instead to set shebangs to /usr/bin/python3 -s.


We've also discussed BuildRequires generators briefly (should be possible with PEP 517), but decided not to block the above on that.

Next steps:

1) contact pip and/or amend PEP 517 about the wheel filename
2) examine shebang options, maybe use pathfix.py after install

Comment 5 Miro Hrončok 2019-03-14 18:05:14 UTC
> 1) contact pip and/or amend PEP 517 about the wheel filename

https://discuss.python.org/t/pep-517-build-frontends-to-inform-users-about-wheel-basename/1026

Comment 6 Miro Hrončok 2019-04-03 23:12:09 UTC
setuptools 40.9.0 got released today:

https://setuptools.readthedocs.io/en/latest/history.html#v40-9-0

"Added support for setup.cfg-only projects when using the setuptools.build_meta backend. Projects that have enabled PEP 517 no longer need to have a setup.py and can use the purely declarative setup.cfg configuration file instead."

Comment 7 Patrik Kopkan 2019-04-18 08:52:21 UTC
(In reply to Miro Hrončok from comment #4)
 
> Next steps:
> 
> 1) contact pip and/or amend PEP 517 about the wheel filename
PR for option to save the basenames of wheel: https://github.com/pypa/pip/pull/6377
> 2) examine shebang options, maybe use pathfix.py after install
after few package built by these macros seems there is not problem with shebangs.
Need to make more research.

Comment 8 Miro Hrončok 2019-04-18 09:00:05 UTC
> there is not problem with shebangs

Yet I'd pretty much love to keep using the -s option in shebangs of our packages.

Comment 9 Miro Hrončok 2019-04-18 09:45:09 UTC
I've just talked to Patrik IRL. We decided to add a flag to pathfix.py that can be sued to... add simple flags to shebangs.

Something like:

pathfix.py -i /usr/bin/python3 -f s

And it will convert shebangs like this:

  python -> /usr/bin/python3 -s

  python -s -> /usr/bin/python3 -s

  python -u -> /usr/bin/python3 -su

  python -us -> /usr/bin/python3 -us

  python -W xxx -> /usr/bin/python3 -sW xxx


Petrik will prep a proof of concept.

Comment 10 Miro Hrončok 2019-04-18 09:47:08 UTC
More details: This will for now only support simple flags without values. In a way, it will exactly fit our use case and we will not need to handle complicated flag merges like proposed in bz1335203.

Comment 11 Patrik Kopkan 2019-05-12 14:15:17 UTC
Please check out if this may be the wanted result https://github.com/PatrikKopkan/cpython/commits/master

Comment 12 Miro Hrončok 2019-05-13 07:59:50 UTC
I've supplied some comments to the GitHub commits. However I don't think it is a best way to provide feedback. Could you open a GitHub pull request *against your own fork*?

Comment 13 Miro Hrončok 2019-07-08 11:47:42 UTC
See https://src.fedoraproject.org/rpms/pyproject-rpm-macros

Comment 14 Ben Cotton 2019-08-13 17:09:49 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 31 development cycle.
Changing version to 31.