Bug 1340885

Summary: rpm: Please backport new virtual provides generator for Python packages
Product: [Fedora] Fedora Reporter: Tomas Orsava <torsava>
Component: rpmAssignee: Packaging Maintenance Team <packaging-team-maint>
Status: CLOSED RAWHIDE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: rawhideCC: ffesti, jzeleny, lkardos, mcyprian, mhroncok, novyjindrich, packaging-team-maint, pknirsch, pviktori
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-06-13 14:34:04 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Tomas Orsava 2016-05-30 14:43:32 UTC
Upstream has made an important change to what virtual provides are generated for Python packages, and we would like to ask you to backport this functionality to Fedora. If you could put it in F24 it would be greatly appreciated, otherwise rawhide is acceptable.

The Python-maintenance and Copr teams are currently working on an automatic way to create Fedora packages from the Python Packaging Index (PyPI) [0][1]. There is a hard problem of matching PyPI dependencies to Fedora packages, which this upstream RPM change handily solves [2].

The necessary changes are concentrated in only one file (`scripts/pythonrggs.py`, renamed in the process to `scripts/pythondistdeps.py`) and done in two commits [C1,2]. The change was also discussed in a pull request [PR].

Note that these changes were done in November 2015/March 2016, and therefore are not yet contained in the latest upstream release (rpm-4.14.0-rc1) from September 2015.

Options: Please leave default values (i.e. no `--majorver-provides`).

Thank you!

[0] https://github.com/fedora-python/pyp2rpm
[1] https://copr.fedorainfracloud.org/coprs/msuchy/PyPi3-for-Fedora/builds/
[2] https://github.com/fedora-python/pyp2rpm/issues/22

[C1] https://github.com/rpm-software-management/rpm/commit/29abb07fbf6b9ea255bd26e492c104eac8d2370f
[C2] https://github.com/rpm-software-management/rpm/commit/f8a75ae6a206e2fcc53c7e91ccca2ccc306fda14

[PR] https://github.com/rpm-software-management/rpm/pull/33

Comment 1 Tomas Orsava 2016-05-31 08:04:12 UTC
Due to the `scripts/pythonrggs.py` file being renamed to `scripts/pythondistdeps.py` during the change, the file name also needs to be modified in `scripts/Makefile.am`. This is  contained in a commit [C3].

[C3] https://github.com/rpm-software-management/rpm/commit/05347154797478ed720eac424a398b59c8ac9802

That should be everything.

Comment 2 Miro HronĨok 2016-05-31 11:10:56 UTC
Some notes:

Adding this to F24 is quite pointless, because only packages that are rebuilt would get the virtual provide. Considering the state of F24 (final freeze today [1]), it would also be unwise to fiddle with RPM.

In rawhide/F25, it would be nice to do this before F25 mass rebuild, but according to [2] it's not happening, so we might need to rebuild all Python packages to make those virtual provides appear.

[1] https://fedoraproject.org/wiki/Releases/24/Schedule
[2] https://fedoraproject.org/wiki/Releases/25/Schedule

Comment 3 Florian Festi 2016-06-13 14:34:04 UTC
Added to rpm-4.13.0-0.rc1.38.fc25. Note that right now only the Provides: part is enabled. To fully make use of the dependency generator the Requires part also needs to be enabled. But this is better left to a point in time when the packages have all be build with the new Provides: already. Please reopen or open a new bug when you are ready to switch Requires on, too.

Comment 4 Tomas Orsava 2016-07-04 11:46:20 UTC
We have decided not to enable the dependency generator after all, not unless the implementation is improved sometime in the future.

For more information: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/Z5YWSXI5IRAOLR57TMQNPSVFGIPLGMDO/

Comment 5 Tomas Orsava 2016-07-14 17:00:08 UTC
Hi Florian!
There might be an issue with this backport. When we try to build a Python package that does have the .egg-info or .dist-info data, the automatic provides still does not seem to get generated.

For example this package was build 2 days ago and should by our count have the Provides generated: http://koji.fedoraproject.org/koji/buildinfo?buildID=779736

Might there be some issue with the backport not being turned on, for example?

Comment 6 Tomas Orsava 2016-07-14 17:15:21 UTC
You can try for example building this tiny package python3-pyPEG2:

The resulting RPM does include an .egg-info file, but the provides is still not generated.

I checked that the new version of RPM is available in the rawhide mock and indeed it is 4.13.0-0.rc1.38.fc25

Comment 7 Tomas Orsava 2016-07-14 17:50:33 UTC
Interestingly, if I take the resulting RPM from python3-pyPEG2 [0], unpack it and run the pythondistdeps.py script (the one taken from dist-git of the RPM package in Fedora) on the unpacked files, it does generate the correct Provides tag:

$ find usr | ./pythondistdeps.py --provides 
python3.4dist(pypeg2) == 2.15.2

So it seems the problem could be in how the script is invoked?

[0] https://kojipkgs.fedoraproject.org//work/tasks/7626/14897626/python3-pyPEG2-2.15.2-1.fc25.noarch.rpm

Comment 8 Tomas Orsava 2016-07-15 12:03:00 UTC
So, the issue was caused by only .py/.pyc/.so files being passed to the new generator, whereas the data needed was in additional meta files.

The issue is fixed in rpm-4.13.0-0.rc1.39.fc25.src.rpm.

Comment 9 Tomas Orsava 2016-08-15 10:07:17 UTC
In the end it was decided to enable the --majorver-provides switch in Fedora 25 and rawhide (F26).

The updated RPM version is: rpm-4.13.0-0.rc1.42.fc26.src.rpm and rpm-4.13.0-0.rc1.41.fc25.src.rpm.