Bug 1917971
| Summary: | RFE: virtualenv to prefer wheels from /usr/share/pythonXY-wheels when creating environment for Python X.Y | |||
|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 8 | Reporter: | Miro Hrončok <mhroncok> | |
| Component: | python-virtualenv | Assignee: | Lumír Balhar <lbalhar> | |
| Status: | CLOSED ERRATA | QA Contact: | Lukáš Zachar <lzachar> | |
| Severity: | unspecified | Docs Contact: | ||
| Priority: | low | |||
| Version: | 8.4 | CC: | cstratak, lbalhar, pviktori, torsava | |
| Target Milestone: | rc | Keywords: | FutureFeature, Triaged | |
| Target Release: | 8.4 | Flags: | pm-rhel:
mirror+
|
|
| Hardware: | Unspecified | |||
| OS: | Unspecified | |||
| Whiteboard: | ||||
| Fixed In Version: | python-virtualenv-15.1.0-20.module+el8.5.0+10529+3694ceb5 | Doc Type: | No Doc Update | |
| Doc Text: | Story Points: | --- | ||
| Clone Of: | ||||
| : | 1945483 (view as bug list) | Environment: | ||
| Last Closed: | 2021-11-09 17:36:25 UTC | Type: | Bug | |
| Regression: | --- | Mount Type: | --- | |
| Documentation: | --- | CRM: | ||
| Verified Versions: | Category: | --- | ||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
| Cloudforms Team: | --- | Target Upstream Version: | ||
| Embargoed: | ||||
| Bug Depends On: | ||||
| Bug Blocks: | 1945483 | |||
To support all old and new Pythons, we have decided to rebase the python-vitrualenv package to the latest "legacy" version 16.7.10 https://github.com/pypa/virtualenv/commits/legacy The reason is to keep support for Python 2 but also use new importlib module (where possible) and support all pip/setuptools/wheel wheels we currently ship in Python modules. The same changes would have to be done downstream so rebase is a reasonable way to go. Ok, back to the original plan because the mentioned version needs setuptools much newer than we have in our modules. So, I'm gonna backport only the necessary fixes. Module scratch builds: - https://mbsweb.engineering.redhat.com/module/10394 - https://mbsweb.engineering.redhat.com/module/10395 PR: https://src.osci.redhat.com/rpms/python-virtualenv/pull-request/11 Verification: python2-virtualenv-15.1.0-19.module+el8.1.0+3507+d69c168d.noarch python3-virtualenv-15.1.0-19.module+el8.1.0+3514+14e79d98.noarch With old virtualenv, you always have the same wheels from /usr/share/python{2,3}-wheels for all Python 2/3 virtual environments. (the first number in venv folder is the number of virtualenv-X command, the second one is the version of Python the environment is created for) Installed wheels: Python 2 venv2-2.7/lib/python2.7/site-packages/pip-9.0.3.dist-info: venv2-2.7/lib/python2.7/site-packages/setuptools-39.0.1.dist-info: venv2-2.7/lib/python2.7/site-packages/wheel-0.31.1.dist-info: venv2-2/lib/python2.7/site-packages/pip-9.0.3.dist-info: venv2-2/lib/python2.7/site-packages/setuptools-39.0.1.dist-info: venv2-2/lib/python2.7/site-packages/wheel-0.31.1.dist-info: venv3-2.7/lib/python2.7/site-packages/pip-9.0.3.dist-info: venv3-2.7/lib/python2.7/site-packages/setuptools-39.0.1.dist-info: venv3-2.7/lib/python2.7/site-packages/wheel-0.31.1.dist-info: venv3-2/lib/python2.7/site-packages/pip-9.0.3.dist-info: venv3-2/lib/python2.7/site-packages/setuptools-39.0.1.dist-info: venv3-2/lib/python2.7/site-packages/wheel-0.31.1.dist-info: Python 3 venv2-3.6/lib/python3.6/site-packages/pip-9.0.3.dist-info: venv2-3.6/lib/python3.6/site-packages/setuptools-39.2.0.dist-info: venv2-3.6/lib/python3.6/site-packages/wheel-0.31.1.dist-info: venv2-3.8/lib/python3.8/site-packages/pip-9.0.3.dist-info: venv2-3.8/lib/python3.8/site-packages/setuptools-39.2.0.dist-info: venv2-3.8/lib/python3.8/site-packages/wheel-0.31.1.dist-info: venv2-3.9/lib/python3.9/site-packages/pip-9.0.3.dist-info: venv2-3.9/lib/python3.9/site-packages/setuptools-39.2.0.dist-info: venv2-3.9/lib/python3.9/site-packages/wheel-0.31.1.dist-info: venv2-3/lib/python3.6/site-packages/pip-9.0.3.dist-info: venv2-3/lib/python3.6/site-packages/setuptools-39.2.0.dist-info: venv2-3/lib/python3.6/site-packages/wheel-0.31.1.dist-info: venv3-3.6/lib/python3.6/site-packages/pip-9.0.3.dist-info: venv3-3.6/lib/python3.6/site-packages/setuptools-39.2.0.dist-info: venv3-3.6/lib/python3.6/site-packages/wheel-0.31.1.dist-info: venv3-3.8/lib/python3.8/site-packages/pip-9.0.3.dist-info: venv3-3.8/lib/python3.8/site-packages/setuptools-39.2.0.dist-info: venv3-3.8/lib/python3.8/site-packages/wheel-0.31.1.dist-info: venv3-3.9/lib/python3.9/site-packages/pip-9.0.3.dist-info: venv3-3.9/lib/python3.9/site-packages/setuptools-39.2.0.dist-info: venv3-3.9/lib/python3.9/site-packages/wheel-0.31.1.dist-info: venv3-3/lib/python3.6/site-packages/pip-9.0.3.dist-info: venv3-3/lib/python3.6/site-packages/setuptools-39.2.0.dist-info: venv3-3/lib/python3.6/site-packages/wheel-0.31.1.dist-info: For newer Pythons, virtualenv also produces: /usr/lib/python3.6/site-packages/virtualenv.py:1043: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses import imp New packages: - python3-virtualenv-15.1.0-20.scrmod+el8.5.0+10395+36be8d19.noarch - python2-virtualenv-15.1.0-20.scrmod+el8.5.0+10394+d85a88a7.noarch The DeprecationWarning is no longer there. Also, virtualenv prefers python-version-specific wheels to python3- python2- wheels. The version should also depend on python3{8,9}-wheel-wheel which is not implemented in the scratch build but is in the PR. These packages will be installed if you already have python38 or 39 installed to make the collection of wheels complete. Content of wheels folders: /usr/share/python2-wheels/: -rwxr-xr-x. 1 root root 1106970 Aug 19 2020 pip-9.0.3-py2.py3-none-any.whl -rwxr-xr-x. 1 root root 343592 Jan 13 12:52 setuptools-39.0.1-py2.py3-none-any.whl -rwxr-xr-x. 1 root root 32929 Jul 22 2019 wheel-0.31.1-py2.py3-none-any.whl /usr/share/python38-wheels/: -rwxr-xr-x. 1 root root 1265487 Nov 30 07:45 pip-19.3.1-py2.py3-none-any.whl -rwxr-xr-x. 1 root root 359541 Nov 30 07:47 setuptools-41.6.0-py2.py3-none-any.whl -rwxr-xr-x. 1 root root 21726 Nov 30 07:51 wheel-0.33.6-py2.py3-none-any.whl /usr/share/python39-wheels/: -rwxr-xr-x. 1 root root 1353249 Feb 5 10:18 pip-20.2.4-py2.py3-none-any.whl -rwxr-xr-x. 1 root root 561792 Feb 5 10:17 setuptools-50.3.2-py3-none-any.whl -rwxr-xr-x. 1 root root 33174 Feb 11 06:58 wheel-0.35.1-py2.py3-none-any.whl /usr/share/python3-wheels/: -rwxr-xr-x. 1 root root 1108681 Jan 8 05:58 pip-9.0.3-py2.py3-none-any.whl -rwxr-xr-x. 1 root root 346642 Mar 26 2020 setuptools-39.2.0-py2.py3-none-any.whl -rwxr-xr-x. 1 root root 32929 Jul 22 2019 wheel-0.31.1-py2.py3-none-any.whl Virtualenvs content: Python 2 venv3-2/lib/python2.7/site-packages/pip-9.0.3.dist-info: venv3-2/lib/python2.7/site-packages/setuptools-39.0.1.dist-info: venv3-2/lib/python2.7/site-packages/wheel-0.31.1.dist-info: venv2-2/lib/python2.7/site-packages/pip-9.0.3.dist-info: venv2-2/lib/python2.7/site-packages/setuptools-39.0.1.dist-info: venv2-2/lib/python2.7/site-packages/wheel-0.31.1.dist-info: Python 2.7 venv2-2.7/lib/python2.7/site-packages/pip-9.0.3.dist-info: venv2-2.7/lib/python2.7/site-packages/setuptools-39.0.1.dist-info: venv2-2.7/lib/python2.7/site-packages/wheel-0.31.1.dist-info: venv3-2.7/lib/python2.7/site-packages/pip-9.0.3.dist-info: venv3-2.7/lib/python2.7/site-packages/setuptools-39.0.1.dist-info: venv3-2.7/lib/python2.7/site-packages/wheel-0.31.1.dist-info: Python 3.6 venv2-3.6/lib/python3.6/site-packages/pip-9.0.3.dist-info: venv2-3.6/lib/python3.6/site-packages/setuptools-39.2.0.dist-info: venv2-3.6/lib/python3.6/site-packages/wheel-0.31.1.dist-info: venv3-3.6/lib/python3.6/site-packages/pip-9.0.3.dist-info: venv3-3.6/lib/python3.6/site-packages/setuptools-39.2.0.dist-info: venv3-3.6/lib/python3.6/site-packages/wheel-0.31.1.dist-info: Python 3.8 venv2-3.8/lib/python3.8/site-packages/pip-19.3.1.dist-info: venv2-3.8/lib/python3.8/site-packages/setuptools-41.6.0.dist-info: venv2-3.8/lib/python3.8/site-packages/wheel-0.33.6.dist-info: venv3-3.8/lib/python3.8/site-packages/pip-19.3.1.dist-info: venv3-3.8/lib/python3.8/site-packages/setuptools-41.6.0.dist-info: venv3-3.8/lib/python3.8/site-packages/wheel-0.33.6.dist-info: Python 3.9 venv2-3.9/lib/python3.9/site-packages/pip-20.2.4.dist-info: venv2-3.9/lib/python3.9/site-packages/setuptools-50.3.2.dist-info: venv2-3.9/lib/python3.9/site-packages/wheel-0.35.1.dist-info: venv3-3.9/lib/python3.9/site-packages/pip-20.2.4.dist-info: venv3-3.9/lib/python3.9/site-packages/setuptools-50.3.2.dist-info: venv3-3.9/lib/python3.9/site-packages/wheel-0.35.1.dist-info: Python 3 venv2-3/lib/python3.6/site-packages/pip-9.0.3.dist-info: venv2-3/lib/python3.6/site-packages/setuptools-39.2.0.dist-info: venv2-3/lib/python3.6/site-packages/wheel-0.31.1.dist-info: venv3-3/lib/python3.6/site-packages/pip-9.0.3.dist-info: venv3-3/lib/python3.6/site-packages/setuptools-39.2.0.dist-info: venv3-3/lib/python3.6/site-packages/wheel-0.31.1.dist-info: The fix will be actually delivered also in python27:2.7 module but the new code should have no effect there. Do we want to clone this bugzilla and make the new implementation official also for Python 2? Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: python36:3.6 security and bug fix update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2021:4150 |
In RHEL 8, we build wheels for pip/setuptools/wheel for each Python module. E.g. the python38 module has wheels in /usr/share/python38-wheels. The venv (ensurepip) module uses the wheels from that specific directory. The wheels for setuptools and pip are guaranteed to be installed with Python. The virtualenv tool, when used as: /usr/bin/virtualenv -p python3.8 --no-download However always uses the wheels from the Platform Python stack (from /usr/share/python3-wheels). That means that they ale older than the ones used for venv. I'd like to get rid of this difference, especially as a precaution for the future: - the wheels in /usr/share/python3-wheels are built and tested with Python 3.6 only - once they stop working with a newly introduced Python module, we will need to fix that anyway The solution should be to: 1) update https://src.osci.redhat.com/rpms/python-virtualenv/blob/stream-15.1.0-rhel-8.4.0/f/rpm-wheels.patch to prefer the /usr/share/pythonXY-wheels dirs if it is available (but fallback to /usr/share/python3-wheels for 3.6 and 3rd party Pythons) 2) making sure pythonXY-wheel-wheel is installed when pythonXY and virtualenv are installed. That could mean we need to update virtualenv with each new Python module, as the RPM reverse dependencies are only week (AFAIK). Requires: (python38-wheel-wheel if python38) Requires: (python39-wheel-wheel if python39)