Bug 1841001

Summary: RFE: Allow installation of pipenv from a specified PIP_INDEX_URL in Python S2I Image ‘ubi7/python-36’ and ‘rhscl/python-36-rhel7’
Product: Red Hat Software Collections Reporter: Abhinay Purty <apurty>
Component: rh-python36-containerAssignee: Lumír Balhar <lbalhar>
Status: CLOSED CURRENTRELEASE QA Contact: rhscl image testing <rhscl-container-qe>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: rh-python36CC: cstratak, hhorak, lbalhar, lkuprova, pkubat, torsava
Target Milestone: alphaKeywords: FutureFeature
Target Release: 3.6   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: No Doc Update
Doc Text:
Documented in Bug 1847416
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-10-14 09:02:03 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Abhinay Purty 2020-05-28 06:01:01 UTC
Description of problem:

While creating a build in OpneShift Container Platform 3.11 with the python S2I Image ‘ubi7/python-36’ and ‘rhscl/python-36-rhel7’ with a local pypi mirror and pipenv, it doesn't take the  env variables passed to it. For the installation of pipenv, pip is using the --isolated option which ignores all environment vars as per the code mentioned in https://github.com/sclorg/s2i-python-container/blob/master/3.6/s2i/bin/assemble#L26


Version-Release number of selected component (if applicable):

Python S2I Image ‘ubi7/python-36’ and ‘rhscl/python-36-rhel7’

How reproducible:

Create a build with the python S2I Image ‘ubi7/python-36’ and ‘rhscl/python-36-rhel7’ with a local pypi mirror and pipenv,

~~~~~~~~~

PIP_TRUSTED_HOST: pypi-mirror.mycompany.local
PIP_INDEX_URL: https://pypi-mirror.mycompany.local/artifactory/api/pypi/pypi.org/simple
ENABLE_PIPENV: true

~~~~~~~~~


Actual results:


The build stars and updates pip from the local mirror:

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Cloning "git.local:ZF-Cloud-Public/zf-custom-container-nx-file-uploader.git" ...
	Commit:	49b30154a4d24cf5c5b31961cefd32c601797382 (Update nx-file-uploader/Pipfile)
	Author:	Foo Bar <foo.bar>
	Date:	Tue May 5 11:52:44 2020 +0200
Using docker-registry.default.svc:5000/ocp010078/zf-python-36@sha256:593d67185b7f3bc264549e6ae91ca7a583d78b9da8f8f7010569db415c5bdba6 as the s2i builder image
---> Installing application source ...
---> Upgrading pip to version 19.3.1 ...
Collecting pip==19.3.1
Downloading https://https://pypi-mirror.mycompany.local/artifactory/api/pypi/pypi.org/packages/packages/00/b6/9cfa56b4081ad13874b0c6f96af8ce16cfbc1cb06bedf8e9164ce5551ec1/pip-19.3.1-py2.py3-none-any.whl  (1.4MB)
Installing collected packages: pip
Found existing installation: pip 9.0.1
Uninstalling pip-9.0.1:
Successfully uninstalled pip-9.0.1
Successfully installed pip-19.3.1
---> Installing pipenv packaging tool ...
Using base prefix '/opt/rh/rh-python36/root/usr'
New python executable in /opt/app-root/src/.local/venvs/pipenv/bin/python3
Also creating executable in /opt/app-root/src/.local/venvs/pipenv/bin/python
Installing setuptools, pip, wheel...done.
Collecting pipenv
  Retrying (Retry(total=4, connect=None, read=None, redirect=None)) after connection broken by 'NewConnectionError('<pip._vendor.requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x7fcb9a4fee10>: Failed to establish a new connection: [Errno 101] Network is unreachable',)': /simple/pipenv/
  Retrying (Retry(total=3, connect=None, read=None, redirect=None)) after connection broken by 'NewConnectionError('<pip._vendor.requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x7fcb9a4fee48>: Failed to establish a new connection: [Errno 101] Network is unreachable',)': /simple/pipenv/
  Retrying (Retry(total=2, connect=None, read=None, redirect=None)) after connection broken by 'NewConnectionError('<pip._vendor.requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x7fcb9a4feef0>: Failed to establish a new connection: [Errno 101] Network is unreachable',)': /simple/pipenv/
  Retrying (Retry(total=1, connect=None, read=None, redirect=None)) after connection broken by 'NewConnectionError('<pip._vendor.requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x7fcb9a4fe780>: Failed to establish a new connection: [Errno 101] Network is unreachable',)': /simple/pipenv/
  Retrying (Retry(total=0, connect=None, read=None, redirect=None)) after connection broken by 'NewConnectionError('<pip._vendor.requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x7fcb9a4fecf8>: Failed to establish a new connection: [Errno 101] Network is unreachable',)': /simple/pipenv/
  Could not find a version that satisfies the requirement pipenv (from versions: )
No matching distribution found for pipenv
error: build error: non-zero (13) exit code from docker-registry.default.svc:5000/ocp010078/zf-python-36@sha256:593d67185b7f3bc264549e6ae91ca7a583d78b9da8f8f701056
https://github.com/sclorg/s2i-python-container/blob/master/3.6/s2i/bin/assemble#L26
9db415c5bdba6

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Expected results:

The Python S2I Image ‘ubi7/python-36’ and ‘rhscl/python-36-rhel7’ should take the  environment variables passed to it.

Comment 2 Tomas Orsava 2020-05-28 09:29:30 UTC
Hi Abhinay,
thank you for filing the issue.

As we discussed in the email thread, it's not really a problem of the environment variable not being respected, it was not intended to be used for the installation of pipenv.
However, we can look into adding that functionality.

Could you please attach the customer ticket?

Comment 4 Tomas Orsava 2020-05-28 09:31:17 UTC
Oh I'm sorry, I see the customer ticket now :)

Comment 7 Lumír Balhar 2020-06-10 04:41:34 UTC
Implementation ready for review: https://github.com/sclorg/s2i-python-container/pull/387

I have a question. When we finish this implementation the updated images will be very quickly available for Fedora (as updates). Will you be able to test them?

Comment 10 Lumír Balhar 2020-06-29 05:10:39 UTC
Centos images with this feature are released. Could you please test them?

Comment 19 Lumír Balhar 2020-09-29 08:53:09 UTC
Unfortunately, this feature is still not released. A new version of ubi7/python-36 container image should be available this week.

Comment 20 Lumír Balhar 2020-10-05 07:44:55 UTC
This feature is now available in the latest release of ubi7 container images. Tested with ubi7/python-36 and ubi7/python-38.

If you specify `PIP_INDEX_URL` in .s2i/environment file, pip uses it during the build (in assemble script). If the installation from a custom index fails, the script tries to do it again with `--isolated` which ignores custom settings and installs packages from the official PyPI.

From the log:

---> Installing pipenv packaging tool ...
Looking in indexes: https://example.com/
ERROR: Could not find a version that satisfies the requirement pipenv (from versions: none)
ERROR: No matching distribution found for pipenv
WARNING: Installation of pipenv failed, trying again from official PyPI with pip --isolated install
Collecting pipenv
...
Installing collected packages: virtualenv-clone, certifi, filelock, distlib, six, appdirs, virtualenv, pipenv
Successfully installed appdirs-1.4.4 certifi-2020.6.20 distlib-0.3.1 filelock-3.0.12 pipenv-2020.8.13 six-1.15.0 virtualenv-20.0.33 virtualenv-clone-0.5.4

Comment 21 Petr Kubat 2020-10-14 09:02:03 UTC
As the feature is available in latest released images, closing the BZ with CURRENTRELEASE. Feel free to reopen if you run into any issues.