Bug 2053948

Summary:	“virtualenv --pip embed …” produces an AttributeError
Product:	[Fedora] Fedora	Reporter:	Ben Beasley <code>
Component:	python-virtualenv	Assignee:	Lumír Balhar <lbalhar>
Status:	CLOSED ERRATA	QA Contact:	Fedora Extras Quality Assurance <extras-qa>
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	rawhide	CC:	cstratak, lbalhar, m.cyprian, mhroncok, mrunge, python-sig, smilner
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	python-virtualenv-20.13.3-2.fc37 python-virtualenv-20.13.3-2.el9	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2022-03-15 19:16:08 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Ben Beasley 2022-02-13 14:07:11 UTC

Description of problem:

When given “--pip embed”, virtualenv produces “AttributeError: 'NoneType' object has no attribute 'version_tuple'”

Version-Release number of selected component (if applicable): at least 20.8.1-1.fc35 or 20.13.1-1.fc36


How reproducible:


Steps to Reproduce:
1. virtualenv --pip embed foo


Actual results:

> AttributeError: 'NoneType' object has no attribute 'version_tuple'


Expected results:

Compare to:

> $ virtualenv --pip bundle bar
> created virtual environment CPython3.10.2.final.0-64 in 206ms
>   creator CPython3Posix(dest=/home/ben/tmp/bar, clear=False, no_vcs_ignore=False, global=False)
>   seeder FromAppData(extra_search_dir=/usr/share/python-wheels,download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/ben/.local/share/virtualenv)
>     added seed packages: pip==21.3.1, setuptools==59.6.0, wheel==0.37.0
>   activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator


Additional info:

I ran into this while trying to package a pre-release of the “hatch” Python project manager—something like poetry or flit—which uses a command of this form internally. This causes a number of tests to fail, and the things they are trying to test really do fail in the same way outside the RPM build environment.

It’s my understanding that this has to do with Fedora’s practice of unbundling the pip wheel. I’ve found a couple of bug reports on virtualenv that seem to be related:

https://github.com/pypa/virtualenv/issues/2283
https://github.com/pypa/virtualenv/issues/2247

and in both cases the upstream response could be paraphrased as “whatever is happening here, it’s due to a downstream patch, so please go away.”

Is it expected that this command will fail? Is there anything that can be done about it?

The difference between “bundle” and “embed” doesn’t seem to be documented anywhere, so I’m not sure what the consequence of patching the virtualenv command in hatch would be. (I could ask.)

Comment 1 Ben Beasley 2022-02-13 16:11:27 UTC

A couple of updates from the discussion at hatch’s upstream bug tracker (https://github.com/ofek/hatch/issues/120):

• “virtualenv --pip embed …” and “virtalenv --pip bundle …” both work in Debian

• hatch can probably remove “--pip embed”, which works around the problem in that particular case

Comment 2 Lumír Balhar 2022-02-14 08:40:43 UTC

Thanks for the report. I'm open to any improvements to our downstream patch. To prepare something, I need to find out what is the difference between --pip bundle (default, works fine) and --pip embed (does not work).

We can then either make them work in the same way with system wheels provided by other RPMs or provide some meaningful error message for --pip embed as we did for upgrade_embed_wheels functionality.

Comment 3 Lumír Balhar 2022-02-15 08:28:02 UTC

Let me describe the difference between Fedora and Debian. In Debian, it seems, that they have all the wheels in one location and they are all always available. Therefore their patch is simpler. Also, I don't know how many Python versions they support at the same time and how often they update the wheels.

On the other hand, in Fedora, the logic has to be dynamic for the following reason: We update pip/setuptools/wheel very often and we support all Pythons supported by upstream + 3.6 and 2.7 as of the time of writing this. The wheels in /usr/share/python-wheels usually cover all modern Pythons in the same way upstream projects support them and then there are exceptions like Python 2.7 and soon Python 3.6. For those, we let the main Python packages bundle their own wheels in ensurepip module and then use it in virtualenv. So, for example, if you create a virtual environment for Python X.Y and a path like /usr/lib64/pythonX.Y/ensurepip/_bundled exists, wheels from that path are available for the virtualenv to install.

From my point of view, `--pip bundle` works as expected. According to this explanation[0], virtualenv should install the best candidate available offline. This is true because the system path with wheels is provided as one extra search dir so users can specify more sources of wheels and virtualenv picks the best ones.

On the other hand, `--pip embed` does not really make sense in our environment because the embedded wheels are not available at all. What we can do here is either to issue a warning and switch back to `bundle` or we can raise an error and exit.

What do you think about it?

[0] https://github.com/ofek/hatch/issues/120#issuecomment-1038815230

Comment 4 Lumír Balhar 2022-03-02 13:57:39 UTC

Which way do you prefer?

Comment 5 Ben Beasley 2022-03-02 14:43:02 UTC

I think I lean gently toward raising an error and exiting, because I generally think it’s better for tools to refuse a request they can’t satisfy than helpfully substitute an alternative. In automated builds, the warning might not even be noticed.

On the other hand, it’s a pretty abstract choice for me, since I’m not someone with a workflow that would be affected by the decision either way.

Comment 6 Lumír Balhar 2022-03-14 14:10:21 UTC

Could you please take a look at my implementation in: https://src.fedoraproject.org/rpms/python-virtualenv/pull-request/63

Comment 7 Ben Beasley 2022-03-14 14:29:57 UTC

Commented on the PR. (Looks sane to me.)

Thanks for working on this.

Comment 8 Fedora Update System 2022-03-15 19:14:11 UTC

FEDORA-2022-84fcaa6d49 has been submitted as an update to Fedora 37. https://bodhi.fedoraproject.org/updates/FEDORA-2022-84fcaa6d49

Comment 9 Fedora Update System 2022-03-15 19:16:08 UTC

FEDORA-2022-84fcaa6d49 has been pushed to the Fedora 37 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 10 Fedora Update System 2022-03-19 10:25:27 UTC

FEDORA-EPEL-2022-2067169c1b has been submitted as an update to Fedora EPEL 9. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2022-2067169c1b

Comment 11 Fedora Update System 2022-03-19 23:46:40 UTC

FEDORA-EPEL-2022-2067169c1b has been pushed to the Fedora EPEL 9 testing repository.

You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2022-2067169c1b

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 12 Fedora Update System 2022-03-27 02:17:14 UTC

FEDORA-EPEL-2022-2067169c1b has been pushed to the Fedora EPEL 9 stable repository.
If problem still persists, please make note of it in this bug report.