Bug 1421776 - rpm: Python Provides generator doesn't work with Python wheels
Summary: rpm: Python Provides generator doesn't work with Python wheels
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Fedora
Classification: Fedora
Component: rpm
Version: 25
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Tomas Orsava
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-02-13 16:10 UTC by Tomas Orsava
Modified: 2017-02-20 11:18 UTC (History)
7 users (show)

Fixed In Version: rpm-4.13.0.1-2.fc26 rpm-4.13.0-7.fc25
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-02-16 14:02:54 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Tomas Orsava 2017-02-13 16:10:14 UTC
Description of problem:
The Python Provides generator (`scripts/pythondistdeps.py --provides --majorver-provides`) does not work with packages built and installed using a Python wheel (macros %py_build_wheel, %py_install_wheel). It doesn't generate the appropriate Provides tags and emits the following warning:

/usr/lib/rpm/pythondistdeps.py:112: RuntimeWarning: Version for six 1.10.0 (/builddir/build/BUILDROOT/python-six-1.10.0-7.fc26.noarch/usr/lib/python3.6/site-packages) has not been found
  warn("Version for {!r} has not been found".format(dist), RuntimeWarning)

However, when the same package is built using the standard %py_build, %py_install macros, the pythondistdeps.py script works fine.

I've investigated it and I've found that it fails because it can't detect the "py_version" (Python version, e.g. 2.7 or 3.5) of the wheel. Inside eggs (%py_build/install), the "py_version" is contained in the file name of the .egg-info directory (six-1.10.0-py3.6.egg-info), whereas in the wheel, the .dist-info directory does not contain py_version (six-1.10.0.dist-info). I've checked the [PEP] for Metadata 2.0 format and it doesn't allow for the possibility of encoding py_version in the file name.

[PEP] https://www.python.org/dev/peps/pep-0426/#metadata-files

Therefore, perhaps pythondistdeps.py needs to be modified to take py_version from elsewhere?


Version-Release number of selected component (if applicable):
rpm-4.13.0-11.fc26

How reproducible:
Every time

Steps to Reproduce:
1. Build a Python package that uses %py_build/install_wheel macros.
E.g. python-setuptools, or this python-six [specfile] where `build_wheel` is configurable.
[specfile] https://paste.fedoraproject.org/556832/raw/


Actual results:
...
+ cp -pr LICENSE /builddir/build/BUILDROOT/python-six-1.10.0-7.fc26.noarch/usr/share/licenses/python3-six
+ exit 0
/usr/lib/rpm/pythondistdeps.py:112: RuntimeWarning: Version for six 1.10.0 (/builddir/build/BUILDROOT/python-six-1.10.0-7.fc26.noarch/usr/lib/python3.6/site-packages) has not been found
  warn("Version for {!r} has not been found".format(dist), RuntimeWarning)
/usr/lib/rpm/pythondistdeps.py:112: RuntimeWarning: Version for six 1.10.0 (/builddir/build/BUILDROOT/python-six-1.10.0-7.fc26.noarch/usr/lib/python3.6/site-packages) has not been found
  warn("Version for {!r} has not been found".format(dist), RuntimeWarning)
/usr/lib/rpm/pythondistdeps.py:112: RuntimeWarning: Version for six 1.10.0 (/builddir/build/BUILDROOT/python-six-1.10.0-7.fc26.noarch/usr/lib/python3.6/site-packages) has not been found
  warn("Version for {!r} has not been found".format(dist), RuntimeWarning)
/usr/lib/rpm/pythondistdeps.py:112: RuntimeWarning: Version for six 1.10.0 (/builddir/build/BUILDROOT/python-six-1.10.0-7.fc26.noarch/usr/lib/python3.6/site-packages) has not been found
  warn("Version for {!r} has not been found".format(dist), RuntimeWarning)
/usr/lib/rpm/pythondistdeps.py:112: RuntimeWarning: Version for six 1.10.0 (/builddir/build/BUILDROOT/python-six-1.10.0-7.fc26.noarch/usr/lib/python3.6/site-packages) has not been found
  warn("Version for {!r} has not been found".format(dist), RuntimeWarning)
/usr/lib/rpm/pythondistdeps.py:112: RuntimeWarning: Version for six 1.10.0 (/builddir/build/BUILDROOT/python-six-1.10.0-7.fc26.noarch/usr/lib/python3.6/site-packages) has not been found
  warn("Version for {!r} has not been found".format(dist), RuntimeWarning)
/usr/lib/rpm/pythondistdeps.py:112: RuntimeWarning: Version for six 1.10.0 (/builddir/build/BUILDROOT/python-six-1.10.0-7.fc26.noarch/usr/lib/python3.6/site-packages) has not been found
  warn("Version for {!r} has not been found".format(dist), RuntimeWarning)
/usr/lib/rpm/pythondistdeps.py:112: RuntimeWarning: Version for six 1.10.0 (/builddir/build/BUILDROOT/python-six-1.10.0-7.fc26.noarch/usr/lib/python3.6/site-packages) has not been found
  warn("Version for {!r} has not been found".format(dist), RuntimeWarning)
Provides: python3-six = 1.10.0-7.fc26
Requires(rpmlib): rpmlib(CompressedFileNames) <= 3.0.4-1 rpmlib(FileDigests) <= 4.6.0-1 rpmlib(PayloadFilesHavePrefix) <= 4.0-1
Checking for unpackaged file(s): /usr/lib/rpm/check-files /builddir/build/BUILDROOT/python-six-1.10.0-7.fc26.noarch
...

Expected results:
...
+ cp -pr LICENSE /builddir/build/BUILDROOT/python-six-1.10.0-7.fc26.noarch/usr/share/licenses/python3-six
+ exit 0
Provides: python3-six = 1.10.0-7.fc26 python3.6dist(six) = 1.10.0 python3dist(six) = 1.10.0
Requires(rpmlib): rpmlib(CompressedFileNames) <= 3.0.4-1 rpmlib(FileDigests) <= 4.6.0-1 rpmlib(PayloadFilesHavePrefix) <= 4.0-1
Checking for unpackaged file(s): /usr/lib/rpm/check-files /builddir/build/BUILDROOT/python-six-1.10.0-7.fc26.noarch
...

Comment 1 Tomas Orsava 2017-02-13 16:20:01 UTC
python-setuptools build log containing the same warning and lacking the proper Provides tags:
https://kojipkgs.fedoraproject.org//packages/python-setuptools/34.1.1/1.fc26/data/logs/noarch/build.log

Comment 2 Neal Gompa 2017-02-13 17:17:14 UTC
Where would we get the Python version in the metadata then?

Comment 3 Igor Gnatenko 2017-02-13 17:43:16 UTC
There's no other way apart from directory name. We had 30 minutes discussion with Tomas. I will send PR.

Comment 4 Tomas Orsava 2017-02-14 16:29:11 UTC
I've wrote a patch and sent it upstream: https://github.com/rpm-software-management/rpm/pull/154

Comment 5 Tomas Orsava 2017-02-16 14:02:54 UTC
Upstream patch was merged and now it's deployed in Fedora 25 and rawhide.
Fedora 24 does not have the new Python provides generator (pythondistdeps.py) and thus doesn't suffer from this affliction.

Comment 6 Panu Matilainen 2017-02-18 09:25:33 UTC
(In reply to Tomas Orsava from comment #5)
> Upstream patch was merged and now it's deployed in Fedora 25 and rawhide.

Eh? Applying a patch in dist-git does not "deploy" it. You need to build it too and for released versions, follow the update procress to get it into the distro.

While an unapplied (because it's not even built!) patch is mostly harmless, this does make pushing rpm 4.13.0.1 to F25 and the git history way messier than need be, for no reason at all.

If you plan on touching a package maintained by somebody else, for TALK TO THE MAINTAINERS FIRST!

Comment 7 Panu Matilainen 2017-02-18 09:36:12 UTC
Oh and in case it wasn't clear between the lines - please do NOT create an F25 update for this because it'd just further complicate getting 4.13.0.1 out.

Comment 8 Tomas Orsava 2017-02-20 10:35:30 UTC
(In reply to Panu Matilainen from comment #6)
> (In reply to Tomas Orsava from comment #5)
> > Upstream patch was merged and now it's deployed in Fedora 25 and rawhide.
> 
> Eh? Applying a patch in dist-git does not "deploy" it. You need to build it
> too and for released versions, follow the update procress to get it into the
> distro.

Oh, my sincere apologies, I must have forgot to fire the official build at the end :/

> While an unapplied (because it's not even built!) patch is mostly harmless,
> this does make pushing rpm 4.13.0.1 to F25 and the git history way messier
> than need be, for no reason at all.
> 
> If you plan on touching a package maintained by somebody else, for TALK TO
> THE MAINTAINERS FIRST!

I've asked Florian if I can push the patch before doing so (though I didn't mention the F25 branch). I'm sorry if backporting it into F25 complicates the git history, but the problem needed to be fixed there as well.

Comment 9 Panu Matilainen 2017-02-20 11:18:15 UTC
(In reply to Tomas Orsava from comment #8)
> (In reply to Panu Matilainen from comment #6)
> > If you plan on touching a package maintained by somebody else, for TALK TO
> > THE MAINTAINERS FIRST!
> 
> I've asked Florian if I can push the patch before doing so (though I didn't
> mention the F25 branch). I'm sorry if backporting it into F25 complicates
> the git history, but the problem needed to be fixed there as well.

Oh, in that case apologies for yelling. What's needed in this case is the maintainers talking between each other :)


Note You need to log in before you can comment on or make changes to this bug.