RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1917971 - RFE: virtualenv to prefer wheels from /usr/share/pythonXY-wheels when creating environment for Python X.Y
Summary: RFE: virtualenv to prefer wheels from /usr/share/pythonXY-wheels when creatin...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 8
Classification: Red Hat
Component: python-virtualenv
Version: 8.4
Hardware: Unspecified
OS: Unspecified
low
unspecified
Target Milestone: rc
: 8.4
Assignee: Lumír Balhar
QA Contact: Lukáš Zachar
URL:
Whiteboard:
Depends On:
Blocks: 1945483
TreeView+ depends on / blocked
 
Reported: 2021-01-19 18:33 UTC by Miro Hrončok
Modified: 2021-11-09 19:26 UTC (History)
4 users (show)

Fixed In Version: python-virtualenv-15.1.0-20.module+el8.5.0+10529+3694ceb5
Doc Type: No Doc Update
Doc Text:
Clone Of:
: 1945483 (view as bug list)
Environment:
Last Closed: 2021-11-09 17:36:25 UTC
Type: Bug
Target Upstream Version:
Embargoed:
pm-rhel: mirror+

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2021:4150 0 None None None 2021-11-09 17:36:38 UTC

Description Miro Hrončok 2021-01-19 18:33:54 UTC 
In RHEL 8, we build wheels for pip/setuptools/wheel for each Python module. E.g. the python38 module has wheels in /usr/share/python38-wheels.

The venv (ensurepip) module uses the wheels from that specific directory. The wheels for setuptools and pip are guaranteed to be installed with Python.

The virtualenv tool, when used as:

    /usr/bin/virtualenv -p python3.8 --no-download

However always uses the wheels from the Platform Python stack (from /usr/share/python3-wheels). That means that they ale older than the ones used for venv.

I'd like to get rid of this difference, especially as a precaution for the future:

 - the wheels in /usr/share/python3-wheels are built and tested with Python 3.6 only
 - once they stop working with a newly introduced Python module, we will need to fix that anyway


The solution should be to:

 1) update https://src.osci.redhat.com/rpms/python-virtualenv/blob/stream-15.1.0-rhel-8.4.0/f/rpm-wheels.patch to prefer the /usr/share/pythonXY-wheels dirs if it is available (but fallback to /usr/share/python3-wheels for 3.6 and 3rd party Pythons)

 2) making sure pythonXY-wheel-wheel is installed when pythonXY and virtualenv are installed. That could mean we need to update virtualenv with each new Python module, as the RPM reverse dependencies are only week (AFAIK).

  Requires: (python38-wheel-wheel if python38)
  Requires: (python39-wheel-wheel if python39)
Comment 4 Lumír Balhar 2021-03-17 14:30:04 UTC 
To support all old and new Pythons, we have decided to rebase the python-vitrualenv package to the latest "legacy" version 16.7.10 https://github.com/pypa/virtualenv/commits/legacy

The reason is to keep support for Python 2 but also use new importlib module (where possible) and support all pip/setuptools/wheel wheels we currently ship in Python modules. The same changes would have to be done downstream so rebase is a reasonable way to go.
Comment 5 Lumír Balhar 2021-03-18 13:32:31 UTC 
Ok, back to the original plan because the mentioned version needs setuptools much newer than we have in our modules. So, I'm gonna backport only the necessary fixes.
Comment 6 Lumír Balhar 2021-03-18 14:16:05 UTC 
Module scratch builds:
- https://mbsweb.engineering.redhat.com/module/10394
- https://mbsweb.engineering.redhat.com/module/10395

PR: https://src.osci.redhat.com/rpms/python-virtualenv/pull-request/11

Verification:

python2-virtualenv-15.1.0-19.module+el8.1.0+3507+d69c168d.noarch
python3-virtualenv-15.1.0-19.module+el8.1.0+3514+14e79d98.noarch

With old virtualenv, you always have the same wheels from /usr/share/python{2,3}-wheels for all Python 2/3 virtual environments.

(the first number in venv folder is the number of virtualenv-X command, the second one is the version of Python the environment is created for)

Installed wheels:

Python 2

venv2-2.7/lib/python2.7/site-packages/pip-9.0.3.dist-info:
venv2-2.7/lib/python2.7/site-packages/setuptools-39.0.1.dist-info:
venv2-2.7/lib/python2.7/site-packages/wheel-0.31.1.dist-info:
venv2-2/lib/python2.7/site-packages/pip-9.0.3.dist-info:
venv2-2/lib/python2.7/site-packages/setuptools-39.0.1.dist-info:
venv2-2/lib/python2.7/site-packages/wheel-0.31.1.dist-info:
venv3-2.7/lib/python2.7/site-packages/pip-9.0.3.dist-info:
venv3-2.7/lib/python2.7/site-packages/setuptools-39.0.1.dist-info:
venv3-2.7/lib/python2.7/site-packages/wheel-0.31.1.dist-info:
venv3-2/lib/python2.7/site-packages/pip-9.0.3.dist-info:
venv3-2/lib/python2.7/site-packages/setuptools-39.0.1.dist-info:
venv3-2/lib/python2.7/site-packages/wheel-0.31.1.dist-info:

Python 3

venv2-3.6/lib/python3.6/site-packages/pip-9.0.3.dist-info:
venv2-3.6/lib/python3.6/site-packages/setuptools-39.2.0.dist-info:
venv2-3.6/lib/python3.6/site-packages/wheel-0.31.1.dist-info:
venv2-3.8/lib/python3.8/site-packages/pip-9.0.3.dist-info:
venv2-3.8/lib/python3.8/site-packages/setuptools-39.2.0.dist-info:
venv2-3.8/lib/python3.8/site-packages/wheel-0.31.1.dist-info:
venv2-3.9/lib/python3.9/site-packages/pip-9.0.3.dist-info:
venv2-3.9/lib/python3.9/site-packages/setuptools-39.2.0.dist-info:
venv2-3.9/lib/python3.9/site-packages/wheel-0.31.1.dist-info:
venv2-3/lib/python3.6/site-packages/pip-9.0.3.dist-info:
venv2-3/lib/python3.6/site-packages/setuptools-39.2.0.dist-info:
venv2-3/lib/python3.6/site-packages/wheel-0.31.1.dist-info:
venv3-3.6/lib/python3.6/site-packages/pip-9.0.3.dist-info:
venv3-3.6/lib/python3.6/site-packages/setuptools-39.2.0.dist-info:
venv3-3.6/lib/python3.6/site-packages/wheel-0.31.1.dist-info:
venv3-3.8/lib/python3.8/site-packages/pip-9.0.3.dist-info:
venv3-3.8/lib/python3.8/site-packages/setuptools-39.2.0.dist-info:
venv3-3.8/lib/python3.8/site-packages/wheel-0.31.1.dist-info:
venv3-3.9/lib/python3.9/site-packages/pip-9.0.3.dist-info:
venv3-3.9/lib/python3.9/site-packages/setuptools-39.2.0.dist-info:
venv3-3.9/lib/python3.9/site-packages/wheel-0.31.1.dist-info:
venv3-3/lib/python3.6/site-packages/pip-9.0.3.dist-info:
venv3-3/lib/python3.6/site-packages/setuptools-39.2.0.dist-info:
venv3-3/lib/python3.6/site-packages/wheel-0.31.1.dist-info:

For newer Pythons, virtualenv also produces:

/usr/lib/python3.6/site-packages/virtualenv.py:1043: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp

New packages:

- python3-virtualenv-15.1.0-20.scrmod+el8.5.0+10395+36be8d19.noarch
- python2-virtualenv-15.1.0-20.scrmod+el8.5.0+10394+d85a88a7.noarch

The DeprecationWarning is no longer there. Also, virtualenv prefers python-version-specific wheels to python3- python2- wheels. The version should also depend on python3{8,9}-wheel-wheel which is not implemented in the scratch build but is in the PR. These packages will be installed if you already have python38 or 39 installed to make the collection of wheels complete.

Content of wheels folders:

/usr/share/python2-wheels/:
-rwxr-xr-x.  1 root root 1106970 Aug 19  2020 pip-9.0.3-py2.py3-none-any.whl
-rwxr-xr-x.  1 root root  343592 Jan 13 12:52 setuptools-39.0.1-py2.py3-none-any.whl
-rwxr-xr-x.  1 root root   32929 Jul 22  2019 wheel-0.31.1-py2.py3-none-any.whl

/usr/share/python38-wheels/:
-rwxr-xr-x.  1 root root 1265487 Nov 30 07:45 pip-19.3.1-py2.py3-none-any.whl
-rwxr-xr-x.  1 root root  359541 Nov 30 07:47 setuptools-41.6.0-py2.py3-none-any.whl
-rwxr-xr-x.  1 root root   21726 Nov 30 07:51 wheel-0.33.6-py2.py3-none-any.whl

/usr/share/python39-wheels/:
-rwxr-xr-x.  1 root root 1353249 Feb  5 10:18 pip-20.2.4-py2.py3-none-any.whl
-rwxr-xr-x.  1 root root  561792 Feb  5 10:17 setuptools-50.3.2-py3-none-any.whl
-rwxr-xr-x.  1 root root   33174 Feb 11 06:58 wheel-0.35.1-py2.py3-none-any.whl

/usr/share/python3-wheels/:
-rwxr-xr-x.  1 root root 1108681 Jan  8 05:58 pip-9.0.3-py2.py3-none-any.whl
-rwxr-xr-x.  1 root root  346642 Mar 26  2020 setuptools-39.2.0-py2.py3-none-any.whl
-rwxr-xr-x.  1 root root   32929 Jul 22  2019 wheel-0.31.1-py2.py3-none-any.whl

Virtualenvs content:

Python 2

venv3-2/lib/python2.7/site-packages/pip-9.0.3.dist-info:
venv3-2/lib/python2.7/site-packages/setuptools-39.0.1.dist-info:
venv3-2/lib/python2.7/site-packages/wheel-0.31.1.dist-info:
venv2-2/lib/python2.7/site-packages/pip-9.0.3.dist-info:
venv2-2/lib/python2.7/site-packages/setuptools-39.0.1.dist-info:
venv2-2/lib/python2.7/site-packages/wheel-0.31.1.dist-info:

Python 2.7

venv2-2.7/lib/python2.7/site-packages/pip-9.0.3.dist-info:
venv2-2.7/lib/python2.7/site-packages/setuptools-39.0.1.dist-info:
venv2-2.7/lib/python2.7/site-packages/wheel-0.31.1.dist-info:
venv3-2.7/lib/python2.7/site-packages/pip-9.0.3.dist-info:
venv3-2.7/lib/python2.7/site-packages/setuptools-39.0.1.dist-info:
venv3-2.7/lib/python2.7/site-packages/wheel-0.31.1.dist-info:

Python 3.6

venv2-3.6/lib/python3.6/site-packages/pip-9.0.3.dist-info:
venv2-3.6/lib/python3.6/site-packages/setuptools-39.2.0.dist-info:
venv2-3.6/lib/python3.6/site-packages/wheel-0.31.1.dist-info:
venv3-3.6/lib/python3.6/site-packages/pip-9.0.3.dist-info:
venv3-3.6/lib/python3.6/site-packages/setuptools-39.2.0.dist-info:
venv3-3.6/lib/python3.6/site-packages/wheel-0.31.1.dist-info:

Python 3.8

venv2-3.8/lib/python3.8/site-packages/pip-19.3.1.dist-info:
venv2-3.8/lib/python3.8/site-packages/setuptools-41.6.0.dist-info:
venv2-3.8/lib/python3.8/site-packages/wheel-0.33.6.dist-info:
venv3-3.8/lib/python3.8/site-packages/pip-19.3.1.dist-info:
venv3-3.8/lib/python3.8/site-packages/setuptools-41.6.0.dist-info:
venv3-3.8/lib/python3.8/site-packages/wheel-0.33.6.dist-info:

Python 3.9

venv2-3.9/lib/python3.9/site-packages/pip-20.2.4.dist-info:
venv2-3.9/lib/python3.9/site-packages/setuptools-50.3.2.dist-info:
venv2-3.9/lib/python3.9/site-packages/wheel-0.35.1.dist-info:
venv3-3.9/lib/python3.9/site-packages/pip-20.2.4.dist-info:
venv3-3.9/lib/python3.9/site-packages/setuptools-50.3.2.dist-info:
venv3-3.9/lib/python3.9/site-packages/wheel-0.35.1.dist-info:

Python 3

venv2-3/lib/python3.6/site-packages/pip-9.0.3.dist-info:
venv2-3/lib/python3.6/site-packages/setuptools-39.2.0.dist-info:
venv2-3/lib/python3.6/site-packages/wheel-0.31.1.dist-info:
venv3-3/lib/python3.6/site-packages/pip-9.0.3.dist-info:
venv3-3/lib/python3.6/site-packages/setuptools-39.2.0.dist-info:
venv3-3/lib/python3.6/site-packages/wheel-0.31.1.dist-info:
Comment 7 Lumír Balhar 2021-03-30 10:36:51 UTC 
The fix will be actually delivered also in python27:2.7 module but the new code should have no effect there. Do we want to clone this bugzilla and make the new implementation official also for Python 2?
Comment 15 errata-xmlrpc 2021-11-09 17:36:25 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: python36:3.6 security and bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:4150
Note You need to log in before you can comment on or make changes to this bug.