Bug 1401120

Enterprise clone of this bug.

+++ This bug was initially created as a clone of Bug #1361755 +++

Description of problem:
Pip fails to install dependencies listed in the requirements.txt file due to mirror trust issues 

How reproducible:
Very

Steps to Reproduce:
1. rhc app-create py27 python-2.7
2. cd py27
3. echo click==6.6 > requirements.txt
4. git add .
5. git commit -a -m "click6.6"
6. git push

Actual results:
remote: Building git ref 'master', commit a6d4bea
remote: Activating virtenv
remote: Checking for pip dependency listed in requirements.txt file..
remote: The directory '/var/lib/openshift/579c36602d527102bf000088/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
remote: You are using pip version 7.1.0, however version 8.1.2 is available.
remote: You should consider upgrading via the 'pip install --upgrade pip' command.
remote: The directory '/var/lib/openshift/579c36602d527102bf000088/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
remote: Collecting click==6.6 (from -r /var/lib/openshift/579c36602d527102bf000088/app-root/runtime/repo/requirements.txt (line 1))
remote:   The repository located at mirror1.ops.rhcloud.com is not a trusted or secure host and is being ignored. If this repository is available via HTTPS it is recommended to use HTTPS instead, otherwise you may silence this warning and allow it anyways with '--trusted-host mirror1.ops.rhcloud.com'.
remote:   Could not find a version that satisfies the requirement click==6.6 (from -r /var/lib/openshift/579c36602d527102bf000088/app-root/runtime/repo/requirements.txt (line 1)) (from versions: )
remote: No matching distribution found for click==6.6 (from -r /var/lib/openshift/579c36602d527102bf000088/app-root/runtime/repo/requirements.txt (line 1))

Expected results:
Should install the dependency

Additional info:

--- Additional comment from  on 2016-07-30 23:07:26 EDT ---

I can get around the untrusted mirror by putting `-i https://pypi.python.org/simple` at the top of the file, but then there's a file permission problem.  So using these steps to duplicate:

1. rhc app-create py27 python-2.7
2. cd py27
3. echo "-i https://pypi.python.org/simple" > requirements.txt
4. echo click==6.6 >> requirements.txt
5. git add .
6. git commit -a -m "Installing stuff"
7. git push

It fails to install the dependencies (click in this case) because of this error:

remote: Building git ref 'master', commit da6184e
remote: Activating virtenv
remote: Checking for pip dependency listed in requirements.txt file..
remote: The directory '/var/lib/openshift/579d69cf2d52714129000171/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
remote: You are using pip version 7.1.0, however version 8.1.2 is available.
remote: You should consider upgrading via the 'pip install --upgrade pip' command.
remote: The directory '/var/lib/openshift/579d69cf2d52714129000171/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
remote: Collecting click==6.6 (from -r /var/lib/openshift/579d69cf2d52714129000171/app-root/runtime/repo/requirements.txt (line 2))
remote:   Downloading click-6.6.tar.gz (283kB)
remote: Building wheels for collected packages: click
remote: Exception:
remote: Traceback (most recent call last):
remote:   File "/var/lib/openshift/579d69cf2d52714129000171/app-root/runtime/dependencies/python/virtenv/lib/python2.7/site-packages/pip/basecommand.py", line 223, in main
remote:     status = self.run(options, args)
remote:   File "/var/lib/openshift/579d69cf2d52714129000171/app-root/runtime/dependencies/python/virtenv/lib/python2.7/site-packages/pip/commands/install.py", line 293, in run
remote:     wb.build(autobuilding=True)
remote:   File "/var/lib/openshift/579d69cf2d52714129000171/app-root/runtime/dependencies/python/virtenv/lib/python2.7/site-packages/pip/wheel.py", line 756, in build
remote:     ensure_dir(output_dir)
remote:   File "/var/lib/openshift/579d69cf2d52714129000171/app-root/runtime/dependencies/python/virtenv/lib/python2.7/site-packages/pip/utils/__init__.py", line 70, in ensure_dir
remote:     os.makedirs(path)
remote:   File "/var/lib/openshift/579d69cf2d52714129000171/python/virtenv/lib64/python2.7/os.py", line 150, in makedirs
remote:     makedirs(head, mode)
remote:   File "/var/lib/openshift/579d69cf2d52714129000171/python/virtenv/lib64/python2.7/os.py", line 150, in makedirs
remote:     makedirs(head, mode)
remote:   File "/var/lib/openshift/579d69cf2d52714129000171/python/virtenv/lib64/python2.7/os.py", line 150, in makedirs
remote:     makedirs(head, mode)
remote:   File "/var/lib/openshift/579d69cf2d52714129000171/python/virtenv/lib64/python2.7/os.py", line 150, in makedirs
remote:     makedirs(head, mode)
remote:   File "/var/lib/openshift/579d69cf2d52714129000171/python/virtenv/lib64/python2.7/os.py", line 150, in makedirs
remote:     makedirs(head, mode)
remote:   File "/var/lib/openshift/579d69cf2d52714129000171/python/virtenv/lib64/python2.7/os.py", line 150, in makedirs
remote:     makedirs(head, mode)
remote:   File "/var/lib/openshift/579d69cf2d52714129000171/python/virtenv/lib64/python2.7/os.py", line 157, in makedirs
remote:     mkdir(name, mode)
remote: OSError: [Errno 13] Permission denied: '/var/lib/openshift/579d69cf2d52714129000171/.cache'
remote: 
remote: Running setup.py script..

--- Additional comment from Jacob Lucky on 2016-08-04 10:54:25 EDT ---

It looks like PIP has been updated so and is now rejecting the insecure mirror. We'll need to update the pypi mirror to https://mirror.openshift.com/mirror/python/web/simple/

--- Additional comment from Jacob Lucky on 2016-08-04 11:13:32 EDT ---

Note: We should close out this related issue when this is resolved: https://github.com/openshift/origin-server/issues/6390

--- Additional comment from openshift-github-bot on 2016-08-04 19:49:39 EDT ---

Commit pushed to master at https://github.com/openshift/li

https://github.com/openshift/li/commit/76a5317388ef144d05ac07a58336964db0614792
Update the pypi mirror to use secure connection https

The new version of PIP requires a mirror link to use secure connection
(https) as the http link will be ignored as the pip install step will
fail. The http link is now replaced with https from Ops to address
the issue.

Bug 1361755
Link https://bugzilla.redhat.com/show_bug.cgi?id=1361755

Signed-off-by: Vu Dinh <vdinh>

--- Additional comment from openshift-github-bot on 2016-08-05 14:13:26 EDT ---

Commit pushed to master at https://github.com/openshift/origin-server

https://github.com/openshift/origin-server/commit/53619d51b37b07beb3084682f71fa0d23db7a7e6
Fix pip cache directory permission denied

For Python 2.7, a new version of PIP (7.1.0) attemps to write into .cache
directory in gear's home directory which will cause permission denied error.
Users do not have permission to create files or folders in gear's home directory.
The .cache directory needs to be created during the installation process. As
a result, a piece of code is added to "install" script to create .cache dir
before pip install step happens to address the issue.

Bug 1361755
Link https://bugzilla.redhat.com/show_bug.cgi?id=1361755

Signed-off-by: Vu Dinh <vdinh>

--- Additional comment from Gaoyun Pei on 2016-09-05 01:42:51 EDT ---

Verify this bug on devenv_5821.

Dependencies added in the requirements.txt could installed successfully now.

1. rhc app-create py27 python-2.7
2. cd py27
3. echo click==6.6 > requirements.txt
4. git add .
5. git commit -a -m "click6.6"
6. git push

Counting objects: 5, done.
Compressing objects: 100% (2/2), done.
Writing objects: 100% (3/3), 267 bytes, done.
Total 3 (delta 1), reused 0 (delta 0)
remote: Stopping Python 2.7 cartridge
remote: Waiting for stop to finish
remote: Waiting for stop to finish
remote: Waiting for stop to finish
remote: Building git ref 'master', commit 009a409
remote: Activating virtenv
remote: Checking for pip dependency listed in requirements.txt file..
remote: You are using pip version 7.1.0, however version 8.1.2 is available.
remote: You should consider upgrading via the 'pip install --upgrade pip' command.
remote: Collecting click==6.6 (from -r /var/lib/openshift/57cd044a5f90149af6000009/app-root/runtime/repo/requirements.txt (line 1))
remote:   Downloading https://mirror.openshift.com/mirror/python/web/packages/7a/00/c14926d8232b36b08218067bcd5853caefb4737cda3f0a47437151344792/click-6.6.tar.gz (283kB)
remote: Building wheels for collected packages: click
remote:   Running setup.py bdist_wheel for click
remote:   Stored in directory: /var/lib/openshift/57cd044a5f90149af6000009/.cache/pip/wheels/84/a2/38/664439dc79dbb324389bf945d6d2b7c0912129c7826056daad
remote: Successfully built click
remote: Installing collected packages: click
remote: Successfully installed click-6.6
remote: Checking pip install marker..
remote: Running setup.py script..
remote: running develop
remote: running egg_info
remote: creating YourAppName.egg-info
remote: writing YourAppName.egg-info/PKG-INFO
remote: writing top-level names to YourAppName.egg-info/top_level.txt
remote: writing dependency_links to YourAppName.egg-info/dependency_links.txt
remote: writing manifest file 'YourAppName.egg-info/SOURCES.txt'
remote: reading manifest file 'YourAppName.egg-info/SOURCES.txt'
remote: writing manifest file 'YourAppName.egg-info/SOURCES.txt'
remote: running build_ext
remote: Creating /var/lib/openshift/57cd044a5f90149af6000009/app-root/runtime/dependencies/python/virtenv/lib/python2.7/site-packages/YourAppName.egg-link (link to .)
remote: Adding YourAppName 1.0 to easy-install.pth file
remote: 
remote: Installed /var/lib/openshift/57cd044a5f90149af6000009/app-root/runtime/repo
remote: Processing dependencies for YourAppName==1.0
remote: Finished processing dependencies for YourAppName==1.0
remote: Preparing build for deployment
remote: Deployment id is 75f2313e
remote: Activating deployment
remote: Starting Python 2.7 cartridge (Apache+mod_wsgi)
remote: Application directory "/" selected as DocumentRoot
remote: Application "wsgi.py" selected as default WSGI entry point
remote: -------------------------
remote: Git Post-Receive Result: success
remote: Activation status: success
remote: Deployment completed with status: success
To ssh://57cd044a5f90149af6000009.rhcloud.com/~/git/py27.git/
   aa0ebd0..009a409  master -> master