Bug 1464570

This whole thing looks really flawed to me.

1. 'ensurepip' is called with the following args from 'venv':
https://github.com/python/cpython/blob/3.6/Lib/venv/__init__.py#L243-L244

2. The expected bundled versions are hardcoded in 'ensurepip':
https://github.com/python/cpython/blob/3.6/Lib/ensurepip/__init__.py#L11-L18

3. 'ensurepip' always tries to open the bundled wheels:
https://github.com/python/cpython/blob/3.6/Lib/ensurepip/__init__.py#L80-L87

4. Fedora requires different bundled versions than upstream:
http://pkgs.fedoraproject.org/cgit/rpms/python3.git/tree/00260-require-setuptools-dependencies.patch?h=f26#n9

This means we must pull the 'setuptools', 'pip', etc. into the _bundled directory during the package build.

This is an issue with setuptools and its dependencies (upstream unbundled them and then rebundled them again) [0] [1].

Already have a fix for it, however I was blocked on creating a new build due to http://bugs.python.org/issue30714 which has been fixed now. Will push a new build soon.

Our python3 version doesn't use the bundled wheels of setuptools and pip (And that's why we do not ship the _bundled directory [2]), instead it uses the system installed wheels, which are then installed inside the venvs (rewheel patch [3]).

[0] http://pkgs.fedoraproject.org/cgit/rpms/python-setuptools.git/diff/python-setuptools.spec?id=49b1915e5e251e00a6fc95e78c60e611f633e2cf

[1] http://pkgs.fedoraproject.org/cgit/rpms/python-setuptools.git/commit/?id=5e5ba5a5931b254843baaf45e6dbd1ebe2769b91

[2] http://pkgs.fedoraproject.org/cgit/rpms/python3.git/tree/python3.spec#n1304

[3] http://pkgs.fedoraproject.org/cgit/rpms/python3.git/tree/00189-add-rewheel-module.patch

python3-3.6.1-7.fc26 has been submitted as an update to Fedora 26. https://bodhi.fedoraproject.org/updates/FEDORA-2017-cffdee8d35

python3-3.6.1-7.fc26 has been pushed to the Fedora 26 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-cffdee8d35

python3-3.6.1-7.fc26 has been pushed to the Fedora 26 stable repository. If problems still persist, please make note of it in this bug report.

I'm still experiencing this issue with package python3-3.6.1-8.fc26.x86_64

Observe:

$ python3 -m venv venv
Error: Command '['/home/jon/venv/bin/python3', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 1.

I can't reproduce with the version you mentioned.

What does "/home/jon/venv/bin/python3 -Im ensurepip --upgrade --default-pip" return?

$ /home/jon/venv/bin/python3 -Im ensurepip --upgrade --default-pip
Traceback (most recent call last):
  File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/lib64/python3.6/ensurepip/__main__.py", line 4, in <module>
    ensurepip._main()
  File "/usr/lib64/python3.6/ensurepip/__init__.py", line 212, in _main
    default_pip=args.default_pip,
  File "/usr/lib64/python3.6/ensurepip/__init__.py", line 109, in bootstrap
    shutil.copy(whl, tmpdir)
  File "/usr/lib64/python3.6/shutil.py", line 241, in copy
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/usr/lib64/python3.6/shutil.py", line 120, in copyfile
    with open(src, 'rb') as fsrc:
FileNotFoundError: [Errno 2] No such file or directory: '/usr/lib64/python3.6/ensurepip/_bundled/setuptools-28.8.0-py2.py3-none-any.whl'


I installed Fedora 26 from scratch (not an upgrade) yesterday.

Should I reopen this as I can still reproduce? Sorry, I'm not sure what correct procedure is.

(In reply to Jon Dufresne from comment #9)
> Should I reopen this as I can still reproduce? Sorry, I'm not sure what
> correct procedure is.

Hello.

I can't reproduce it as well. Is this happening on a fresh F26 install? Did you update pip or setuptools somehow through pip?

> Is this happening on a fresh F26 install?

Yes. It was installed from scratch on July 8.

> Did you update pip or setuptools somehow through pip?

No. If there is some way I could verify that my system isn't unintentionally modified, perhaps through check sums, please let me know.

Try:

$ rpm -V $(rpm -qa | grep python3)

I reinstalled Fedora 26 from scratch today with the official release. I can no longer reproduce. Not sure what I was experiencing, but can't reproduce it anymore. Thanks for your time and help.

Found this on google and was surprised of the mixed results because I've been able to reproduce this for as long as I can remember using F26. At least.

$ python3 -V
Python 3.6.3
$ python3 -m venv .venv
Error: Command '['/home/stemid/Utveckling/test/.venv/bin/python3', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 1.
$ source .venv/bin/activate
-bash: .venv/bin/activate: No such file or directory

Here is my workaround that I found online.

$ python3 -m venv --without-pip .venv
$ source .venv/bin/activate
$ curl https://bootstrap.pypa.io/get-pip.py | python

Forgot to mention:

python3.x86_64                         3.6.3-2.fc26

Fedora 26 (Workstation Edition)

Try:

$ rpm -V $(rpm -qa | grep python3)

(In reply to Miro Hrončok from comment #16)
> Try:
> 
> $ rpm -V $(rpm -qa | grep python3)

$ rpm -V $(rpm -qa | grep python3)
S.5....T.    /usr/lib/python3.6/site-packages/urllib3/packages/__pycache__/six.cpython-36.pyc

What does that mean? A leftover compiled file that should be removed?

One pyc shouldn't be a problem.

How about:

LANG=C.utf8 find /usr/lib64/python3.6/site-packages/ -exec rpm -qf {} \; | grep "is not owned by any package"

and:

LANG=C.utf8 find /usr/lib/python3.6/site-packages/ -exec rpm -qf {} \; | grep "is not owned by any package"

$ LANG=C.utf8 find /usr/lib64/python3.6/site-packages/ -exec rpm -qf {} \; | grep "is not owned by any package"
file /usr/lib64/python3.6/site-packages/report is not owned by any package
file /usr/lib64/python3.6/site-packages/reportclient is not owned by any package

$ LANG=C.utf8 find /usr/lib/python3.6/site-packages/ -exec rpm -qf {} \; | grep "is not owned by any package"
file /usr/lib/python3.6/site-packages/isc is not owned by any package
file /usr/lib/python3.6/site-packages/isc/__pycache__ is not owned by any package
file /usr/lib/python3.6/site-packages/humanize/locale is not owned by any package
file /usr/lib/python3.6/site-packages/humanize/locale/ru_RU is not owned by any package
file /usr/lib/python3.6/site-packages/humanize/locale/ru_RU/LC_MESSAGES is not owned by any package
file /usr/lib/python3.6/site-packages/humanize/locale/ko_KR is not owned by any package
file /usr/lib/python3.6/site-packages/humanize/locale/ko_KR/LC_MESSAGES is not owned by any package
file /usr/lib/python3.6/site-packages/humanize/locale/fr_FR is not owned by any package
file /usr/lib/python3.6/site-packages/humanize/locale/fr_FR/LC_MESSAGES is not owned by any package
file /usr/lib/python3.6/site-packages/pykickstart is not owned by any package
file /usr/lib/python3.6/site-packages/pykickstart/handlers is not owned by any package
file /usr/lib/python3.6/site-packages/pykickstart/commands is not owned by any package
file /usr/lib/python3.6/site-packages/sepolicy is not owned by any package
file /usr/lib/python3.6/site-packages/nbconvert is not owned by any package

If it's noteworthy, I believe I've upgraded this Fedora since maybe 24 or 23 without a clean re-install.

But those leftover python packages seem to be no older than 2017. One had a couple of .py files from 2015 but that was the only exception.

None of those seems dangerous. Confusing.

What is the exact error if you do:

$ python3 -Im ensurpip

(In reply to Miro Hrončok from comment #20)
> None of those seems dangerous. Confusing.
> 
> What is the exact error if you do:
> 
> $ python3 -Im ensurpip

I assume you meant outside of virtualenv.

$ python3 -Im ensurpip
/usr/bin/python3: No module named ensurpip

Inside virtualenv is the same except the python path is different. This is the virtualenv where I've used curl to workaround the issue.

(In reply to Miro Hrončok from comment #20)
> None of those seems dangerous. Confusing.
> 
> What is the exact error if you do:
> 
> $ python3 -Im ensurpip

Eh typo;

$ python3 -Im ensurepip       
Traceback (most recent call last):                                    
  File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main                                                                     
    "__main__", mod_spec)                                             
  File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code         
    exec(code, run_globals)                                           
  File "/usr/lib64/python3.6/ensurepip/__main__.py", line 4, in <module>                                                                     
    ensurepip._main()                                                 
  File "/usr/lib64/python3.6/ensurepip/__init__.py", line 212, in _main                                                                      
    default_pip=args.default_pip,                                     
  File "/usr/lib64/python3.6/ensurepip/__init__.py", line 91, in bootstrap                                                                   
    new_whl = rewheel.rewheel_from_record(dr, rewheel_dir.name)       
  File "/usr/lib64/python3.6/ensurepip/rewheel/__init__.py", line 75, in rewheel_from_record                                                 
    new_wheel.write(os.path.join(site_dir, f), arcname=f)             
  File "/usr/lib64/python3.6/zipfile.py", line 1594, in write         
    zinfo = ZipInfo.from_file(filename, arcname)                      
  File "/usr/lib64/python3.6/zipfile.py", line 484, in from_file      
    st = os.stat(filename)                                            
FileNotFoundError: [Errno 2] No such file or directory: '/usr/lib/python3.6/site-packages/pip/_vendor/ipaddress.py'

Ha!

That was fixed in https://bodhi.fedoraproject.org/updates/FEDORA-2017-eac4d90de1 https://bugzilla.redhat.com/show_bug.cgi?id=1522684

Just dnf update python3.

(In reply to Miro Hrončok from comment #23)
> Ha!
> 
> That was fixed in
> https://bodhi.fedoraproject.org/updates/FEDORA-2017-eac4d90de1
> https://bugzilla.redhat.com/show_bug.cgi?id=1522684
> 
> Just dnf update python3.

There are no updates for python3 in dnf. 

I'm on F26. 

$ LANG=C sudo dnf update python3
Last metadata expiration check: 0:10:48 ago on Tue Jan 23 22:59:06 2018.
Dependencies resolved.
Nothing to do.
Complete!

oh sorry, sudo dnf update python3-pip

(In reply to Stefan Midjich from comment #24)
> (In reply to Miro Hrončok from comment #23)
> > Ha!
> > 
> > That was fixed in
> > https://bodhi.fedoraproject.org/updates/FEDORA-2017-eac4d90de1
> > https://bugzilla.redhat.com/show_bug.cgi?id=1522684
> > 
> > Just dnf update python3.
> 
> There are no updates for python3 in dnf. 
> 
> I'm on F26. 
> 
> $ LANG=C sudo dnf update python3
> Last metadata expiration check: 0:10:48 ago on Tue Jan 23 22:59:06 2018.
> Dependencies resolved.
> Nothing to do.
> Complete!

That worked better, now it's resolved. Thanks. :)