Bug 2011455 - Recent change in Python sysconfig install scheme broke virtualenv
Summary: Recent change in Python sysconfig install scheme broke virtualenv
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: python-virtualenv
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
Assignee: Miro Hrončok
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
: 2011475 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-10-06 15:48 UTC by Miro Hrončok
Modified: 2021-10-07 13:18 UTC (History)
7 users (show)

Fixed In Version: python-virtualenv-20.8.1-1.fc36
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-10-07 13:18:57 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github pypa virtualenv issues 2208 0 None open Custom distutils/sysconfig INSTALL_SCHEME leads to different paths in virtualenv, how to avoid this? 2021-10-06 17:49:53 UTC

Description Miro Hrončok 2021-10-06 15:48:12 UTC
See https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org/thread/AAGUFQZ4RZDU7KUN4HA43KQJCMSFR3GW/

This change apparently broke virtualenv:

Before:

$ virtualenv --python=python3.10 venv
created virtual environment CPython3.10.0.final.0-64 in 158ms
  creator CPython3Posix(dest=/builddir/venv, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(extra_search_dir=/usr/share/python-wheels,download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/builddir/.local/share/virtualenv)
    added seed packages: pip==21.2.3, setuptools==57.4.0, wheel==0.37.0
  activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator
$ ls venv/bin/
activate  activate.csh  activate.fish  activate.ps1  activate_this.py  pip  pip-3.10  pip3  pip3.10  python  python3  python3.10  wheel  wheel-3.10  wheel3  wheel3.10


After:

$ virtualenv --python=python3.10 venv
created virtual environment CPython3.10.0.final.0-64 in 200ms
  creator CPython3Posix(dest=/builddir/venv, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(extra_search_dir=/usr/share/python-wheels,download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/builddir/.local/share/virtualenv)
    added seed packages: pip==21.2.3, setuptools==57.4.0, wheel==0.37.0
  activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator

$ ls venv/
lib64  local  pyvenv.cfg


There is no error message, no bad exit code, nothing.


python-setuptools-wheel-57.4.0-1.fc35.noarch
python-wheel-wheel-0.37.0-1.fc36.noarch
python3-distlib-0.3.2-1.fc35.noarch
python3-filelock-3.0.12-12.fc35.noarch
python3-platformdirs-2.3.0-1.fc36.noarch
python3-six-1.16.0-4.fc35.noarch
python3-virtualenv-20.7.2-1.fc36.noarch
python-pip-wheel-21.2.3-3.fc36.noarch
python3-3.10.0-2.fc36.x86_64
python3-libs-3.10.0-2.fc36.x86_64

Comment 1 Miro Hrončok 2021-10-06 17:15:51 UTC
$ virtualenv --python=python3.10 venv2 -vvvv
55 setup logging to NOTSET [DEBUG report:43]
58 find interpreter for spec PythonSpec(implementation=CPython, major=3, minor=10) [INFO builtin:62]
58 proposed PythonInfo(spec=CPython3.10.0.final.0-64, exe=/usr/bin/python3, platform=linux, version='3.10.0 (default, Oct  5 2021, 00:00:00) [GCC 11.2.1 20210728 (Red Hat 11.2.1-1)]', encoding_fs_io=utf-8-utf-8) [INFO builtin:69]
58 accepted PythonInfo(spec=CPython3.10.0.final.0-64, exe=/usr/bin/python3, platform=linux, version='3.10.0 (default, Oct  5 2021, 00:00:00) [GCC 11.2.1 20210728 (Red Hat 11.2.1-1)]', encoding_fs_io=utf-8-utf-8) [DEBUG builtin:71]
59 filesystem is case-sensitive [DEBUG info:28]
92 create virtual environment via CPython3Posix(dest=/builddir/venv2, clear=False, no_vcs_ignore=False, global=False) [INFO session:52]
92 create folder /builddir/venv2/lib64/python3.10 [DEBUG _sync:25]
93 create folder /builddir/venv2/local/bin [DEBUG _sync:25]
93 create folder /builddir/venv2/local/lib/python3.10/site-packages [DEBUG _sync:25]
93 create folder /builddir/venv2/local/lib64/python3.10/site-packages [DEBUG _sync:25]
93 write /builddir/venv2/pyvenv.cfg [DEBUG pyenv_cfg:34]
93 	home = /usr [DEBUG pyenv_cfg:38]
93 	implementation = CPython [DEBUG pyenv_cfg:38]
93 	version_info = 3.10.0.final.0 [DEBUG pyenv_cfg:38]
93 	virtualenv = 20.7.2 [DEBUG pyenv_cfg:38]
93 	include-system-site-packages = false [DEBUG pyenv_cfg:38]
93 	base-prefix = /usr [DEBUG pyenv_cfg:38]
93 	base-exec-prefix = /usr [DEBUG pyenv_cfg:38]
93 	base-executable = /usr/bin/python3 [DEBUG pyenv_cfg:38]
93 symlink /usr/bin/python3 to /builddir/venv2/local/bin/python [DEBUG _sync:44]
94 create virtualenv import hook file /builddir/venv2/local/lib/python3.10/site-packages/_virtualenv.pth [DEBUG api:95]
94 create /builddir/venv2/local/lib/python3.10/site-packages/_virtualenv.py [DEBUG api:98]
94 ============================== target debug ============================== [DEBUG session:54]
94 debug via /builddir/venv2/local/bin/python /usr/lib/python3.10/site-packages/virtualenv/create/debug.py [DEBUG creator:224]
94 {
  "sys": {
    "executable": "/builddir/venv2/local/bin/python",
    "_base_executable": "/builddir/venv2/local/bin/python",
    "prefix": "/usr",
    "base_prefix": "/usr",
    "real_prefix": null,
    "exec_prefix": "/usr",
    "base_exec_prefix": "/usr",
    "path": [
      "/usr/lib64/python310.zip",
      "/usr/lib64/python3.10",
      "/usr/lib64/python3.10/lib-dynload",
      "/usr/lib64/python3.10/site-packages",
      "/usr/lib/python3.10/site-packages"
    ],
    "meta_path": [
      "<class '_frozen_importlib.BuiltinImporter'>",
      "<class '_frozen_importlib.FrozenImporter'>",
      "<class '_frozen_importlib_external.PathFinder'>"
    ],
    "fs_encoding": "utf-8",
    "io_encoding": "utf-8"
  },
  "version": "3.10.0 (default, Oct  5 2021, 00:00:00) [GCC 11.2.1 20210728 (Red Hat 11.2.1-1)]",
  "makefile_filename": "/usr/lib64/python3.10/config-3.10-x86_64-linux-gnu/Makefile",
  "os": "<module 'os' from '/usr/lib64/python3.10/os.py'>",
  "site": "<module 'site' from '/usr/lib64/python3.10/site.py'>",
  "datetime": "<module 'datetime' from '/usr/lib64/python3.10/datetime.py'>",
  "math": "<module 'math' from '/usr/lib64/python3.10/lib-dynload/math.cpython-310-x86_64-linux-gnu.so'>",
  "json": "<module 'json' from '/usr/lib64/python3.10/json/__init__.py'>"
} [DEBUG session:55]
145 add seed packages via FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/builddir/.local/share/virtualenv) [INFO session:59]
199 install setuptools from wheel /usr/share/python-wheels/setuptools-57.4.0-py3-none-any.whl via CopyPipInstall [DEBUG via_app_data:50]
199 install wheel from wheel /usr/share/python-wheels/wheel-0.37.0-py2.py3-none-any.whl via CopyPipInstall [DEBUG via_app_data:50]
200 Attempting to acquire lock 140433895328976 on /builddir/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/setuptools-57.4.0-py3-none-any.lock [DEBUG filelock:270]
200 Lock 140433895328976 acquired on /builddir/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/setuptools-57.4.0-py3-none-any.lock [INFO filelock:274]
200 Attempting to acquire lock 140433895326144 on /builddir/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/wheel-0.37.0-py2.py3-none-any.lock [DEBUG filelock:270]
200 install pip from wheel /usr/share/python-wheels/pip-21.2.3-py3-none-any.whl via CopyPipInstall [DEBUG via_app_data:50]
200 Lock 140433895326144 acquired on /builddir/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/wheel-0.37.0-py2.py3-none-any.lock [INFO filelock:274]
200 Attempting to release lock 140433895328976 on /builddir/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/setuptools-57.4.0-py3-none-any.lock [DEBUG filelock:315]
201 Lock 140433895328976 released on /builddir/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/setuptools-57.4.0-py3-none-any.lock [INFO filelock:318]
201 Attempting to release lock 140433895326144 on /builddir/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/wheel-0.37.0-py2.py3-none-any.lock [DEBUG filelock:315]
201 Lock 140433895326144 released on /builddir/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/wheel-0.37.0-py2.py3-none-any.lock [INFO filelock:318]
201 copy /builddir/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/setuptools-57.4.0-py3-none-any/distutils-precedence.pth to /builddir/venv2/local/lib/python3.10/site-packages/distutils-precedence.pth [DEBUG _sync:52]
201 copy /builddir/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/wheel-0.37.0-py2.py3-none-any/wheel-0.37.0.virtualenv to /builddir/venv2/local/lib/python3.10/site-packages/wheel-0.37.0.virtualenv [DEBUG _sync:52]
201 Attempting to acquire lock 140433895326144 on /builddir/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/pip-21.2.3-py3-none-any.lock [DEBUG filelock:270]
201 Lock 140433895326144 acquired on /builddir/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/pip-21.2.3-py3-none-any.lock [INFO filelock:274]
202 Attempting to release lock 140433895326144 on /builddir/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/pip-21.2.3-py3-none-any.lock [DEBUG filelock:315]
202 Lock 140433895326144 released on /builddir/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/pip-21.2.3-py3-none-any.lock [INFO filelock:318]
202 copy directory /builddir/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/setuptools-57.4.0-py3-none-any/setuptools to /builddir/venv2/local/lib/python3.10/site-packages/setuptools [DEBUG _sync:52]
202 copy directory /builddir/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/wheel-0.37.0-py2.py3-none-any/wheel-0.37.0.dist-info to /builddir/venv2/local/lib/python3.10/site-packages/wheel-0.37.0.dist-info [DEBUG _sync:52]
202 copy /builddir/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/pip-21.2.3-py3-none-any/pip-21.2.3.virtualenv to /builddir/venv2/local/lib/python3.10/site-packages/pip-21.2.3.virtualenv [DEBUG _sync:52]
203 copy directory /builddir/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/pip-21.2.3-py3-none-any/pip to /builddir/venv2/local/lib/python3.10/site-packages/pip [DEBUG _sync:52]
206 copy directory /builddir/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/wheel-0.37.0-py2.py3-none-any/wheel to /builddir/venv2/local/lib/python3.10/site-packages/wheel [DEBUG _sync:52]
215 generated console scripts wheel3 wheel3.10 wheel-3.10 wheel [DEBUG base:48]
237 copy directory /builddir/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/setuptools-57.4.0-py3-none-any/_distutils_hack to /builddir/venv2/local/lib/python3.10/site-packages/_distutils_hack [DEBUG _sync:52]
237 copy directory /builddir/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/setuptools-57.4.0-py3-none-any/pkg_resources to /builddir/venv2/local/lib/python3.10/site-packages/pkg_resources [DEBUG _sync:52]
242 copy /builddir/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/setuptools-57.4.0-py3-none-any/setuptools-57.4.0.virtualenv to /builddir/venv2/local/lib/python3.10/site-packages/setuptools-57.4.0.virtualenv [DEBUG _sync:52]
242 copy directory /builddir/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/setuptools-57.4.0-py3-none-any/setuptools-57.4.0.dist-info to /builddir/venv2/local/lib/python3.10/site-packages/setuptools-57.4.0.dist-info [DEBUG _sync:52]
244 generated console scripts  [DEBUG base:48]
267 copy directory /builddir/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/pip-21.2.3-py3-none-any/pip-21.2.3.dist-info to /builddir/venv2/local/lib/python3.10/site-packages/pip-21.2.3.dist-info [DEBUG _sync:52]
269 generated console scripts pip3 pip pip3.10 pip-3.10 [DEBUG base:48]
269 add activators for Bash, CShell, Fish, PowerShell, Python [INFO session:64]
270 write /builddir/venv2/pyvenv.cfg [DEBUG pyenv_cfg:34]
271 	home = /usr [DEBUG pyenv_cfg:38]
271 	implementation = CPython [DEBUG pyenv_cfg:38]
271 	version_info = 3.10.0.final.0 [DEBUG pyenv_cfg:38]
271 	virtualenv = 20.7.2 [DEBUG pyenv_cfg:38]
271 	include-system-site-packages = false [DEBUG pyenv_cfg:38]
271 	base-prefix = /usr [DEBUG pyenv_cfg:38]
271 	base-exec-prefix = /usr [DEBUG pyenv_cfg:38]
271 	base-executable = /usr/bin/python3 [DEBUG pyenv_cfg:38]
271 created virtual environment CPython3.10.0.final.0-64 in 217ms
  creator CPython3Posix(dest=/builddir/venv2, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(extra_search_dir=/usr/share/python-wheels,download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/builddir/.local/share/virtualenv)
    added seed packages: pip==21.2.3, setuptools==57.4.0, wheel==0.37.0
  activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator [WARNING __main__:19]

Comment 2 Miro Hrončok 2021-10-06 17:49:53 UTC
I've opened an upstream virtualenv issue https://github.com/pypa/virtualenv/issues/2208

However, I think this is on us :(

A hacky workaround is to hotpatch virtualenv/discovery/py_info.py PythonInfo.install_path():

        # A hack for https://github.com/pypa/virtualenv/issues/2208
        if result.startswith(u"local/"):
            return result[6:]
        return result

Comment 3 Miro Hrončok 2021-10-06 18:32:28 UTC
*** Bug 2011475 has been marked as a duplicate of this bug. ***

Comment 5 Fedora Update System 2021-10-07 13:16:25 UTC
FEDORA-2021-642854e1cc has been submitted as an update to Fedora 36. https://bodhi.fedoraproject.org/updates/FEDORA-2021-642854e1cc

Comment 6 Fedora Update System 2021-10-07 13:18:57 UTC
FEDORA-2021-642854e1cc has been pushed to the Fedora 36 stable repository.
If problem still persists, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.