Bug 1340885 - rpm: Please backport new virtual provides generator for Python packages
Summary: rpm: Please backport new virtual provides generator for Python packages
Keywords:
Status: CLOSED RAWHIDE
Alias: None
Product: Fedora
Classification: Fedora
Component: rpm
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Packaging Maintenance Team
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-05-30 14:43 UTC by Tomas Orsava
Modified: 2016-08-15 10:07 UTC (History)
9 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-06-13 14:34:04 UTC


Attachments (Terms of Use)

Description Tomas Orsava 2016-05-30 14:43:32 UTC
Hi!
Upstream has made an important change to what virtual provides are generated for Python packages, and we would like to ask you to backport this functionality to Fedora. If you could put it in F24 it would be greatly appreciated, otherwise rawhide is acceptable.

The Python-maintenance and Copr teams are currently working on an automatic way to create Fedora packages from the Python Packaging Index (PyPI) [0][1]. There is a hard problem of matching PyPI dependencies to Fedora packages, which this upstream RPM change handily solves [2].

The necessary changes are concentrated in only one file (`scripts/pythonrggs.py`, renamed in the process to `scripts/pythondistdeps.py`) and done in two commits [C1,2]. The change was also discussed in a pull request [PR].

Note that these changes were done in November 2015/March 2016, and therefore are not yet contained in the latest upstream release (rpm-4.14.0-rc1) from September 2015.

Options: Please leave default values (i.e. no `--majorver-provides`).

Thank you!


[0] https://github.com/fedora-python/pyp2rpm
[1] https://copr.fedorainfracloud.org/coprs/msuchy/PyPi3-for-Fedora/builds/
[2] https://github.com/fedora-python/pyp2rpm/issues/22

[C1] https://github.com/rpm-software-management/rpm/commit/29abb07fbf6b9ea255bd26e492c104eac8d2370f
[C2] https://github.com/rpm-software-management/rpm/commit/f8a75ae6a206e2fcc53c7e91ccca2ccc306fda14

[PR] https://github.com/rpm-software-management/rpm/pull/33

Comment 1 Tomas Orsava 2016-05-31 08:04:12 UTC
Due to the `scripts/pythonrggs.py` file being renamed to `scripts/pythondistdeps.py` during the change, the file name also needs to be modified in `scripts/Makefile.am`. This is  contained in a commit [C3].

[C3] https://github.com/rpm-software-management/rpm/commit/05347154797478ed720eac424a398b59c8ac9802

That should be everything.

Comment 2 Miro Hrončok 2016-05-31 11:10:56 UTC
Some notes:

Adding this to F24 is quite pointless, because only packages that are rebuilt would get the virtual provide. Considering the state of F24 (final freeze today [1]), it would also be unwise to fiddle with RPM.

In rawhide/F25, it would be nice to do this before F25 mass rebuild, but according to [2] it's not happening, so we might need to rebuild all Python packages to make those virtual provides appear.

[1] https://fedoraproject.org/wiki/Releases/24/Schedule
[2] https://fedoraproject.org/wiki/Releases/25/Schedule

Comment 3 Florian Festi 2016-06-13 14:34:04 UTC
Added to rpm-4.13.0-0.rc1.38.fc25. Note that right now only the Provides: part is enabled. To fully make use of the dependency generator the Requires part also needs to be enabled. But this is better left to a point in time when the packages have all be build with the new Provides: already. Please reopen or open a new bug when you are ready to switch Requires on, too.

Comment 4 Tomas Orsava 2016-07-04 11:46:20 UTC
We have decided not to enable the dependency generator after all, not unless the implementation is improved sometime in the future.

For more information: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/Z5YWSXI5IRAOLR57TMQNPSVFGIPLGMDO/

Comment 5 Tomas Orsava 2016-07-14 17:00:08 UTC
Hi Florian!
There might be an issue with this backport. When we try to build a Python package that does have the .egg-info or .dist-info data, the automatic provides still does not seem to get generated.

For example this package was build 2 days ago and should by our count have the Provides generated: http://koji.fedoraproject.org/koji/buildinfo?buildID=779736

Might there be some issue with the backport not being turned on, for example?

Comment 6 Tomas Orsava 2016-07-14 17:15:21 UTC
You can try for example building this tiny package python3-pyPEG2:
https://admin.fedoraproject.org/pkgdb/package/rpms/python3-pyPEG2/

The resulting RPM does include an .egg-info file, but the provides is still not generated.

I checked that the new version of RPM is available in the rawhide mock and indeed it is 4.13.0-0.rc1.38.fc25

Comment 7 Tomas Orsava 2016-07-14 17:50:33 UTC
Interestingly, if I take the resulting RPM from python3-pyPEG2 [0], unpack it and run the pythondistdeps.py script (the one taken from dist-git of the RPM package in Fedora) on the unpacked files, it does generate the correct Provides tag:

$ find usr | ./pythondistdeps.py --provides 
python3.4dist(pypeg2) == 2.15.2

So it seems the problem could be in how the script is invoked?


[0] https://kojipkgs.fedoraproject.org//work/tasks/7626/14897626/python3-pyPEG2-2.15.2-1.fc25.noarch.rpm

Comment 8 Tomas Orsava 2016-07-15 12:03:00 UTC
So, the issue was caused by only .py/.pyc/.so files being passed to the new generator, whereas the data needed was in additional meta files.

The issue is fixed in rpm-4.13.0-0.rc1.39.fc25.src.rpm.

Comment 9 Tomas Orsava 2016-08-15 10:07:17 UTC
In the end it was decided to enable the --majorver-provides switch in Fedora 25 and rawhide (F26).

The updated RPM version is: rpm-4.13.0-0.rc1.42.fc26.src.rpm and rpm-4.13.0-0.rc1.41.fc25.src.rpm.


Note You need to log in before you can comment on or make changes to this bug.