Bug 1651317

Summary: pip and pipenv are incompatible
Product: [Fedora] Fedora Reporter: Mohammed Arafa <bugzilla>
Component: pipenvAssignee: Miro Hrončok <mhroncok>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: high Docs Contact:
Priority: high    
Version: 29CC: bugzilla, mhroncok, moshegrey
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: pipenv-2018.11.26-1.fc29 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-12-02 08:27:24 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1652089, 1652091    
Bug Blocks:    

Description Mohammed Arafa 2018-11-19 16:46:19 UTC
Description of problem:


Version-Release number of selected component (if applicable):
python-pip-wheel-18.0-4.fc29.noarch
pipenv-2018.7.1-2.fc29.noarch
python2-pip-18.0-4.fc29.noarch
python3-pip-18.0-4.fc29.noarch

How reproducible:
everytime

Steps to Reproduce:
1. install pip and pipenv from rpm
2. run pipenv graph --json
3.

Actual results:
pipenv graph --json
Traceback (most recent call last):
  File "/usr/bin/pipenv", line 11, in <module>
    load_entry_point('pipenv==2018.7.1', 'console_scripts', 'pipenv')()
  File "/usr/lib/python3.7/site-packages/click/core.py", line 721, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/python3.7/site-packages/click/core.py", line 696, in main
    rv = self.invoke(ctx)
  File "/usr/lib/python3.7/site-packages/click/core.py", line 1065, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/lib/python3.7/site-packages/click/core.py", line 894, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib/python3.7/site-packages/click/core.py", line 534, in invoke
    return callback(*args, **kwargs)
  File "/usr/lib/python3.7/site-packages/pipenv/cli.py", line 922, in graph
    do_graph(bare=bare, json=json, json_tree=json_tree, reverse=reverse)
  File "/usr/lib/python3.7/site-packages/pipenv/core.py", line 2435, in do_graph
    for d in simplejson.loads(c.out):
  File "/usr/lib64/python3.7/json/__init__.py", line 348, in loads
    return _default_decoder.decode(s)
  File "/usr/lib64/python3.7/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib64/python3.7/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Expected results:
just works

Additional info:
since this is a core requirement for pycharm to install packages.. it affects pycharm

Comment 1 Miro Hrončok 2018-11-19 18:04:51 UTC
pipenv bundles it's own pip, so I wonder what is the problem here.

Comment 2 Miro Hrončok 2018-11-19 18:06:30 UTC
also, I cannot reproduce this in a random project, do you have an open source project where this happens that I can clone from somewhere and test?

Comment 3 Mohammed Arafa 2018-11-19 21:10:36 UTC
no open source project sorry. i am learning python on the side for a few months now and i see this in a couple of my projects. i will say that it was working on fc28 before i upgraded to fc29 .. maybe the upgrade changed something? i know i had to delete virtual env already

Comment 4 Mohammed Arafa 2018-11-20 05:09:26 UTC
https://github.com/pypa/pipenv/issues/732 i do believe this might be it

Comment 5 Miro Hrončok 2018-11-20 15:03:53 UTC
Does this help:

$ pipenv --rm
$ pipenv install

?

Maybe the problem was that you created the environment with python3 without specifying the version explicitly (3.6 or 3.7) and F28 to F29 updated python3 from 3.6 to 3.7?

Comment 6 Mohammed Arafa 2018-11-21 12:04:45 UTC
output of the above command

pipenv install
Creating a virtualenv for this project...
Pipfile: /home/marafa/Documents/echognumix/amazon_fba/arbitrage/site_list/Pipfile
Using /usr/bin/python3.7m (3.7.1) to create virtualenv...
⠋Running virtualenv with interpreter /usr/bin/python3.7m
Using base prefix '/usr'
/usr/lib/python3.7/site-packages/virtualenv.py:1053: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp
New python executable in /home/marafa/.local/share/virtualenvs/site_list-q5BAfYg_/bin/python3.7m
Also creating executable in /home/marafa/.local/share/virtualenvs/site_list-q5BAfYg_/bin/python
Installing setuptools, pip, wheel...done.
Setting project for site_list-q5BAfYg_ to /home/marafa/Documents/echognumix/amazon_fba/arbitrage/site_list

Virtualenv location: /home/marafa/.local/share/virtualenvs/site_list-q5BAfYg_
Pipfile.lock (dc7aa0) out of date, updating to (5aabbc)...
Locking [dev-packages] dependencies...
Locking [packages] dependencies...
ils.py", line 402, in resolve_deps
    req_dir=req_dir
  File "/usr/lib/python3.7/site-packages/pipenv/utils.py", line 250, in actually_resolve_deps
    req = Requirement.from_line(dep)
  File "/usr/lib/python3.7/site-packages/pipenv/vendor/requirementslib/models/requirements.py", line 704, in from_line
    line, extras = _strip_extras(line)
TypeError: 'module' object is not callable

Comment 7 Miro Hrončok 2018-11-21 14:44:06 UTC
That's https://github.com/pypa/pipenv/issues/2924

I think we just need to upgrade pipenv to a new version. 

Does the workaround of setting environment variable IP_SHIMS_BASE_MODULE=pipenv.patched.notpip work for you?

Comment 8 Miro Hrončok 2018-11-26 12:56:54 UTC
*** Bug 1650896 has been marked as a duplicate of this bug. ***

Comment 9 Mohammed Arafa 2018-11-28 00:17:43 UTC
Adding pymongo to Pipfile's [packages]...
Pipfile.lock (dc7aa0) out of date, updating to (5aabbc)...
Locking [dev-packages] dependencies...
Locking [packages] dependencies...
ils.py", line 402, in resolve_deps
    req_dir=req_dir
  File "/usr/lib/python3.7/site-packages/pipenv/utils.py", line 250, in actually_resolve_deps
    req = Requirement.from_line(dep)
  File "/usr/lib/python3.7/site-packages/pipenv/vendor/requirementslib/models/requirements.py", line 704, in from_line
    line, extras = _strip_extras(line)
TypeError: 'module' object is not callable


no it does not work. 

is there a timeline yet as to when it will hit the repos?

Comment 10 Mohammed Arafa 2018-11-28 00:18:52 UTC
fyi currently installed versions:
python3-pip-18.0-4.fc29.noarch
python2-pip-18.0-4.fc29.noarch
python-pip-wheel-18.0-4.fc29.noarch
pipenv-2018.7.1-2.fc29.noarch

Comment 11 Miro Hrončok 2018-11-28 06:41:55 UTC
new pip is ready in rawhide
new pipenv is ready for review in https://src.fedoraproject.org/rpms/pipenv/pull-request/3

This should hit F29 testing during this week.

Comment 12 Mohammed Arafa 2018-11-28 13:43:00 UTC
thank you

Comment 13 Mohammed Arafa 2018-11-30 02:45:00 UTC
the src.rpm is for fc27 .. will that be an issue?

Comment 14 Mohammed Arafa 2018-11-30 02:59:34 UTC
i mean shouldnt it be fc29?

Comment 15 Miro Hrončok 2018-11-30 09:57:18 UTC
what srpm?

Comment 16 Fedora Update System 2018-11-30 10:39:53 UTC
pipenv-2018.11.26-1.fc29 python-pip-18.1-1.fc29 python-shellingham-1.2.7-1.fc29 has been submitted as an update to Fedora 29. https://bodhi.fedoraproject.org/updates/FEDORA-2018-b5c855ceea

Comment 17 Fedora Update System 2018-12-01 02:42:52 UTC
pipenv-2018.11.26-1.fc29, python-pip-18.1-1.fc29, python-shellingham-1.2.7-1.fc29 has been pushed to the Fedora 29 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-b5c855ceea

Comment 18 Mohammed Arafa 2018-12-01 12:05:00 UTC
sudo yum update --enablerepo updates-testing pipenv-2018.11.26-1.fc29
Last metadata expiration check: 0:02:10 ago on Sat 01 Dec 2018 07:01:46 AM EST.
Dependencies resolved.

 Problem: cannot install the best update candidate for package pipenv-2018.7.1-2.fc29.noarch
  - nothing provides python3dist(toml) >= 0.10.0 needed by pipenv-2018.11.26-1.fc29.noarch
========================================================================================================================================================================
 Package                            Arch                               Version                                        Repository                                   Size
========================================================================================================================================================================
Skipping packages with broken dependencies:
 pipenv                             noarch                             2018.11.26-1.fc29                              updates-testing                             4.5 M

Transaction Summary
========================================================================================================================================================================
Skip  1 Package

Nothing to do.
Complete!

Comment 19 Miro Hrončok 2018-12-01 15:55:13 UTC
python3dist(toml) >= 0.10.0 is on it's way from testing to stable. maybe you got old metadata for stable but new metadat for testing and hence it cannot find it?

try:

sudo dnf --refresh update --enablerepo updates-testing pipenv

Comment 20 Mohammed Arafa 2018-12-01 18:50:52 UTC
sudo dnf --refresh update --enablerepo updates-testing pipenv
Copr repo for PyCharm owned by phracek                                                                                                  2.3 kB/s | 3.0 kB     00:01
Fedora Modular 29 - x86_64                                                                                                               13 kB/s |  17 kB     00:01
Fedora Modular 29 - x86_64 - Updates                                                                                                     11 kB/s |  17 kB     00:01
Fedora 29 - x86_64 - Test Updates                                                                                                       9.9 kB/s |  15 kB     00:01
Fedora 29 - x86_64 - Updates                                                                                                             13 kB/s |  17 kB     00:01
Fedora 29 - x86_64                                                                                                                       11 kB/s |  17 kB     00:01
google-chrome                                                                                                                           1.2 kB/s | 1.3 kB     00:01
google-talkplugin                                                                                                                       772  B/s | 951  B     00:01
RPM Fusion for Fedora 29 - Free - Updates                                                                                               8.4 kB/s |  13 kB     00:01
RPM Fusion for Fedora 29 - Free                                                                                                         8.6 kB/s |  14 kB     00:01
RPM Fusion for Fedora 29 - Nonfree - Updates                                                                                            8.4 kB/s |  13 kB     00:01
RPM Fusion for Fedora 29 - Nonfree                                                                                                      8.7 kB/s |  14 kB     00:01
skype (stable)                                                                                                                          2.0 kB/s | 2.9 kB     00:01
slack                                                                                                                                   366  B/s | 1.0 kB     00:02
Stable virtio-win builds                                                                                                                2.1 kB/s | 3.0 kB     00:01
Dependencies resolved.

 Problem: cannot install the best update candidate for package pipenv-2018.7.1-2.fc29.noarch
  - nothing provides python3dist(toml) >= 0.10.0 needed by pipenv-2018.11.26-1.fc29.noarch
========================================================================================================================================================================
 Package                            Arch                               Version                                        Repository                                   Size
========================================================================================================================================================================
Skipping packages with broken dependencies:
 pipenv                             noarch                             2018.11.26-1.fc29                              updates-testing                             4.5 M

Transaction Summary
========================================================================================================================================================================
Skip  1 Package

Nothing to do.
Complete!


and thank you for your patience

Comment 21 Miro Hrončok 2018-12-01 23:52:32 UTC
I can actually reproduce this.

https://bodhi.fedoraproject.org/updates/FEDORA-2018-1e6748cd4d

This was pushed to stable 3 hours ago. Let's give it a day for sync, maybe something wrong happened in the mirrors / repos.


If you need the fix fast, grab the new version from that update directly:

dnf update --enablerepo=updates-testing pipenv https://kojipkgs.fedoraproject.org//packages/python-toml/0.10.0/1.fc29/noarch/python3-toml-0.10.0-1.fc29.noarch.rpm

Comment 22 Fedora Update System 2018-12-02 08:27:24 UTC
pipenv-2018.11.26-1.fc29, python-pip-18.1-1.fc29, python-shellingham-1.2.7-1.fc29 has been pushed to the Fedora 29 stable repository. If problems still persist, please make note of it in this bug report.