Description of problem: When deploying a simple python cartridge in OSE 1.2.1 into an environment where there is NO internet access there is no way to point the application to an internal mirror. So, when i create an app and then edit it the first time (any simple edit) commit and push i get: > remote: Stopping PYTHON cart > remote: httpd (no pid file) not running > remote: setup.py found. Setting up virtualenv > remote: 'import site' failed; use -v for traceback > remote: New python executable in > /var/lib/openshift/523becb91c95a4918a000004/python//virtenv/bin/python > remote: Installing setuptools...................... > remote: Complete output from command > /var/lib/openshift/5...n/virtenv/bin/python -c "#!python > remote: \"\"\"Bootstra...sys.argv[1:]) > remote: > remote: > remote: > remote: > remote: > remote: > remote: " /usr/lib/python2.6/s...ols-0.6c11-py2.6.egg: > remote: 'import site' failed; use -v for traceback > remote: Downloading > http://pypi.python.org/packages/2.6/s/setuptools/setuptools-0.6c11-py2.6.egg > remote: Traceback (most recent call last): > remote: File "<string>", line 279, in <module> > remote: File "<string>", line 211, in main > remote: File "<string>", line 159, in download_setuptools > remote: File "/usr/lib64/python2.6/urllib2.py", line 126, in urlopen > remote: return _opener.open(url, data, timeout) > remote: File "/usr/lib64/python2.6/urllib2.py", line 391, in open > remote: response = self._open(req, data) > remote: File "/usr/lib64/python2.6/urllib2.py", line 409, in _open > remote: '_open', req) > remote: File "/usr/lib64/python2.6/urllib2.py", line 369, in _call_chain > remote: result = func(*args) > remote: File "/usr/lib64/python2.6/urllib2.py", line 1190, in http_open > remote: return self.do_open(httplib.HTTPConnection, req) > remote: File "/usr/lib64/python2.6/urllib2.py", line 1165, in do_open > remote: raise URLError(err) > remote: urllib2.URLError: <urlopen error [Errno 113] No route to host> > remote: ---------------------------------------- > remote: ...Installing setuptools...done. > remote: Traceback (most recent call last): > remote: File "/usr/bin/virtualenv", line 3, in <module> > remote: virtualenv.main() > remote: File "/usr/lib/python2.6/site-packages/virtualenv.py", line 928, in > main > remote: never_download=options.never_download) > remote: File "/usr/lib/python2.6/site-packages/virtualenv.py", line 1040, > in create_environment > remote: search_dirs=search_dirs, never_download=never_download) > remote: File "/usr/lib/python2.6/site-packages/virtualenv.py", line 593, in > install_setuptools > remote: search_dirs=search_dirs, never_download=never_download) > remote: File "/usr/lib/python2.6/site-packages/virtualenv.py", line 567, in > _install_req > remote: cwd=cwd) > remote: File "/usr/lib/python2.6/site-packages/virtualenv.py", line 1006, > in call_subprocess > remote: % (cmd_desc, proc.returncode)) > remote: OSError: Command /var/lib/openshift/5...n/virtenv/bin/python -c > "#!python > remote: \"\"\"Bootstra...sys.argv[1:]) > remote: > remote: > remote: > remote: > remote: > remote: > remote: " /usr/lib/python2.6/s...ols-0.6c11-py2.6.egg failed with error > code 1 > remote: An error occurred executing 'gear postreceive' > remote: > remote: For more details about the problem, try running the command again > with the '--trace' option. I edited the projects setup.py to use my internal URL: setup(name='YourAppName', version='1.0', description='OpenShift App', author='Your Name', author_email='example', url='http://internalpypi.me.org/sigs', # install_requires=['Django>=1.3'], ) But on next push the same error occurs and: http://pypi.python.org/packages/2.6/s/setuptools/setuptools-0.6c11-py2.6.egg is still used. Version-Release number of selected component (if applicable): How reproducible: every time Steps to Reproduce: 1. rhc create-app python1 python-2.6 2. edit a file (any simple edit to trigger a rebuild and restart) 3. edit setup.py to point at an internal mirror 4. commit 5. push Actual results: requests go against http://pypi.python.org/packages/2.6/s/setuptools/setuptools-0.6c11-py2.6.egg Expected results: requests go against http://internalpypi.me.org/.../setuptools-0.6c11-py2.6.egg Additional info:
Can the internal server be a proxy? Setting the HTTP_PROXY and HTTPS_PROXY environment variables worked as expected with setup.py. You could set them for all gears by creating /etc/openshift/env/HTTP{,S}_PROXY files with the content being the company's internal proxy. Otherwise it could likely be set in a pre_build action hook for only that gear.
We set the python mirror in the build for python cartridge. We should allow deployments to customize the mirror by setting an enviroment variable under /etc/openshift/env and then make the python cartridge use that variable. We could name it as OPENSHIFT_PYPI_MIRROR_URL or something similar. Thanks, Mrunal
Created attachment 805214 [details] support for setting OPENSHIFT_PYPI_MIRROR_URL Here's how to apply this: cd /var/lib/openshift/.cartridge_repository/redhat-python/0.0.1/ patch -p3 < /path/to/BZ1012721.patch The expected output is: patching file versions/2.6/bin/control patching file versions/shared/bin/build To undo this change you can apply the patch in reverse: cd /var/lib/openshift/.cartridge_repository/redhat-python/0.0.1/ patch -R -p3 < /path/to/BZ1012721.patch Once that is complete you can set /etc/openshift/env/OPENSHIFT_PYPI_MIRROR_URL. The content of that file should be a url to a PyPi index.
Mrunal let me know what you think of the patch. I see no reason why we couldn't set OPENSHIFT_PYPI_MIRROR_URL for all Nodes in Online and avoid using the hardcoded rhcloud approach.
Hi, Brenton, Yes, the patch looks good and makes sense to use for Online as well. Thanks, Mrunal
August, let us know if this will solve the customer's problem and we'll upstream the patch and get it into OSE.
This works well in my tests and would be of GREAT benefit!
OK, I'll work to get this upstream and then backport to OSE. We should be able to release it with 1.2.4 in just a few weeks.
See the test cases for the upstream bug on how to test this.
I have verified it on puddle [1.2/2013-10-03.1]: 1.create a python app #rhc app create pythonapp python-2.6 2.set OPENSHIFT_PYPI_MIRROR_URL on Node #echo http://mirror1.ops.rhcloud.com/mirror/python/web/simple >>/etc/openshift/env/OPENSHIFT_PYPI_MIRROR_URL 3.configure setup.py file #setup.py from setuptools import setup setup(name='YourAppName', version='1.0', description='OpenShift App', author='Your Name', author_email='example', url='http://mirror1.ops.rhcloud.com/mirror/python/web/simple', install_requires=['Django==1.4.2'], ) 4.git push Output: remote: Reading http://mirror1.ops.rhcloud.com/mirror/python/web/simple/Django/ remote: Best match: Django 1.4.2 remote: Downloading http://mirror1.ops.rhcloud.com/mirror/python/web/packages/source/D/Django/Django-1.4.2.tar.gz#md5=6ffecdc01ad360e1abdca1015ae0893a remote: Processing Django-1.4.2.tar.gz remote: Running Django-1.4.2/setup.py -q bdist_egg --dist-dir /tmp/easy_install-UPFOML/Django-1.4.2/egg-dist-tmp-1ITGbx
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. http://rhn.redhat.com/errata/RHBA-2013-1435.html