Bug 1718398

Summary: Use RPM built wheels of setuptools and pip in Python's ensurepip
Product: Red Hat Enterprise Linux 8 Reporter: Charalampos Stratakis <cstratak>
Component: python27-2.7-moduleAssignee: Charalampos Stratakis <cstratak>
Status: CLOSED ERRATA QA Contact: Lukáš Zachar <lzachar>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 8.1CC: jkejda, pviktori
Target Milestone: rc   
Target Release: 8.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: python27-2.7-8010020190702154151.51c94b97 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-11-05 20:44:11 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Charalampos Stratakis 2019-06-07 16:14:59 UTC
This bug was initially created as a copy of Bug #1701286

I am copying this bug because: 



Upstream CPython bundles pre-built wheels of setuptools and pip. It installs them to the system when the ensurepip module is invoked, or to new virtual environments created via the venv module (the venv module transitively invokes the ensurepip module).

In Fedora, we remove those bundled wheels and we have a workaround.

In Fedora 28 (and earlier), we had a rather large custom patch in python3 (named "rewheel") that made the ensurepip module (and transitively venv) collect the files from the installed python2-setuptools and python2-pip packages, "rewheeled" them into an on demand wheel in /tmp and used that wheel instead of the originally bundled one.

Since Fedora 29, we do it differently. During the build of python-setuptools and python-pip RPMs a wheel is built from sources. The components gained a new subpackage python-setuptools-wheel / python-pip-wheel with the wheel:

$ rpm -ql python-pip-wheel
/usr/share/licenses/python-pip-wheel
/usr/share/licenses/python-pip-wheel/LICENSE.txt
/usr/share/python-wheels
/usr/share/python-wheels/pip-19.0.3-py2.py3-none-any.whl


Python is then patched simply to look for the wheels in different location and pick the highest available version instead of the hardcoded one.

The change can be seen in https://src.fedoraproject.org/rpms/python3/pull-request/49

It has several benefits (with footnotes):

 - the patch is trivial and doesn't need much maintenance [1], whereas this was not true for the previous "rewheel" patch
 - RPM built wheels can be used in multiple Python interpreters assuming they are compatible - in Fedora they are currently compatible with Python 2.7, 3.4-3.8, both PyPys and somehow also with Jython [2]
 - when python-wheel also gains a wheel package (python-wheel-wheel), together those RPM built wheels can also be used in the python-virtualenv package - previously it just bundled the wheels from upstream
 - the mechanism in ensurepip is as much upstream-similar as possible
 - when pip or setuptools is updated after python2 is built, the version reported by the ensurepip module remains current, whereas previously it was baked in on build time
 - python2 doesn't depend on python2-pip and  python2-setuptools packages, which might be appreciated by some system administrators [3]


To lower the maintenance burden, I suppose we should update python in RHEL 8.1 to be aligned with Fedora 29+ in this manner.

Python 2 in RHEL 6/7 doesn't have pip.

Python 3 in RHEL 7 has rewheel. Whether it should stay that way or not is out of scope here.


------------------------

[1] The patch needs to be rebased when new setuptools/pip releases are bundled within a new python release, however the rebasing only consists of hardcoded version bumps in the original - that serves as a reminder to also update the versions hardcoded in the specfile (used when this entire patch is bconded out)

[2] They are not compatible with Python 2.6 but that doesn't have neithr ensurepip nor venv

[3] Technically it still depends on pip and setuptools wheels, however those are not "installed" (from Python perspective) but only "available"

Comment 2 Lukáš Zachar 2019-09-26 15:27:32 UTC
strace has shown that whl from /usr/share/python2 was used to install pip and setuptools

Comment 4 errata-xmlrpc 2019-11-05 20:44:11 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2019:3335