Bug 972983 - Unable to Install MySQL-Python To Pyramid Quickstart Application In Setup.py or Using SSH
Unable to Install MySQL-Python To Pyramid Quickstart Application In Setup.py ...
Status: CLOSED CURRENTRELEASE
Product: OpenShift Origin
Classification: Red Hat
Component: Image (Show other bugs)
2.x
Unspecified Unspecified
medium Severity high
: ---
: ---
Assigned To: Mrunal Patel
libra bugs
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2013-06-10 22:02 EDT by Daniel Pronych
Modified: 2015-05-14 22:20 EDT (History)
8 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2014-04-24 17:34:14 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
Install error with the 80-character width limit obtained from the Git Bash prompt. (6.97 KB, text/plain)
2013-06-10 22:02 EDT, Daniel Pronych
no flags Details
Results of rebuild using MySQL-python 1.2.4 - Still Has Errors (23.67 KB, text/plain)
2013-06-26 19:26 EDT, Daniel Pronych
no flags Details
Results of rebuild using MySQL-python 1.2.3 - Still Has Errors (23.39 KB, text/plain)
2013-06-26 21:02 EDT, Daniel Pronych
no flags Details
Commit Log for Python 2.7 after executing ./active_virtenv and then a new commit (12.75 KB, text/plain)
2013-07-04 00:20 EDT, Daniel Pronych
no flags Details
Appserver.log after executing ./activate_virtenv, recommit and reload website. (3.42 KB, text/plain)
2013-07-04 00:21 EDT, Daniel Pronych
no flags Details
app (106.37 KB, image/png)
2013-07-11 00:48 EDT, Wei Sun
no flags Details

  None (edit)
Description Daniel Pronych 2013-06-10 22:02:36 EDT
Created attachment 759397 [details]
Install error with the 80-character width limit obtained from the Git Bash prompt.

Description of problem: Cannot install MySQL-python module to work with MySQL with the Python web framework, Pyramid, application.  SQLite works properly.  Also looks like setuptools will not install properly.


Version-Release number of selected component (if applicable): Python 2.7


How reproducible:
Install Python Pyramid web framework using quickstarter code such as from https://github.com/lmacken/openshift-quickstarter


Steps to Reproduce:
1. Log in to application using ssh.
2. Use easy_install or pip to install MySQL-python, i.e. "pip MySQL-python"
3. Note the encountered error message.

Actual results:
* See attachment including encountered errors.

Expected results:
* MySQL-python should have been installed.

Additional info:
* application hexidecimal code has been replaced with !appcode.
* application name has been replaced with app-dom.rhcloud.com.


  Thanks and looking forward to your response!
Comment 1 Clayton Coleman 2013-06-12 10:55:14 EDT
May be related to the virtualenv problem previously reported.
Comment 2 Daniel Pronych 2013-06-26 14:55:08 EDT
Please provide an update or if further information is required.  Thanks and looking forward to an update!
Comment 3 Mrunal Patel 2013-06-26 15:20:10 EDT
Could you try to install MySQL-Python=1.2.3 by adding it to your setup.py? You should add the force_clean_build marker to force your virtualenv to be recreated. (It will have to be since it must be in a broken state).
Comment 4 Daniel Pronych 2013-06-26 19:26:40 EDT
Created attachment 765836 [details]
Results of rebuild using MySQL-python 1.2.4 - Still Has Errors

Thanks, please see the attached file of the results, including the requested force_clean_build marker.  Also, I compiled first specifying MySQL-python=1.2.3 and setup.py did not accept that setting.  The attached log file shows the build of MySQL-python 1.2.4.  After uploading this code the application returns error code 503:

"Service Temporarily Unavailable

The server is temporarily unable to service your request due to maintenance downtime or capacity problems. Please try again later."

Thanks and looking forward to your response.
Comment 5 Mrunal Patel 2013-06-26 20:36:09 EDT
Sorry, about the confusion, it needs to be specified as "MySQL-Python==1.2.3"

e.g.

--- a/setup.py                                                                                                        
+++ b/setup.py                                                                                                        
@@ -7,8 +7,7 @@ setup(name='YourAppName', version='1.0',                                                              

       #  Uncomment one or more lines below in the install_requires section
       #  for the specific client drivers/modules your application needs.
-      install_requires=['greenlet', 'gevent',                                                                        
-                        #  'MySQL-python',                                                                           
+      install_requires=['greenlet', 'gevent', 'MySQL-python==1.2.3',                                                 
                         #  'pymongo',
                         #  'psycopg2',
       ],
Comment 6 Daniel Pronych 2013-06-26 21:02:01 EDT
Created attachment 765861 [details]
Results of rebuild using MySQL-python 1.2.3 - Still Has Errors

No problem, thanks for the clarification.  Please find attached the message log when deploying with MySQL-python 1.2.3, even with the other version the system still encounters the same 503 error code after redeployment.  Thanks!
Comment 7 Luke Macken 2013-07-03 13:47:00 EDT
/usr/bin/ld: cannot find -lpython2.7

It looks like the 2.7 version of the python-libs package needs to be installed system-wide on OpenShift. /usr/lib64/libpython2.6.so* exists, but not 2.7.
Comment 8 Luke Macken 2013-07-03 14:10:49 EDT
(In reply to Luke Macken from comment #7)
> /usr/bin/ld: cannot find -lpython2.7
> 
> It looks like the 2.7 version of the python-libs package needs to be
> installed system-wide on OpenShift. /usr/lib64/libpython2.6.so* exists, but
> not 2.7.

Okay, after talking to some people on IRC it looks like you need to `python ~/python/bin/activate_virtenv`, which is a wrapper that updates the LD_LIBRARY_PATH to point to libpython2.7.so
Comment 9 Daniel Pronych 2013-07-04 00:20:40 EDT
Created attachment 768584 [details]
Commit Log for Python 2.7 after executing ./active_virtenv and then a new commit
Comment 10 Daniel Pronych 2013-07-04 00:21:57 EDT
Created attachment 768585 [details]
Appserver.log after executing ./activate_virtenv, recommit and reload website.
Comment 11 Daniel Pronych 2013-07-04 00:24:17 EDT
Please see the following results.  The above attachments have been obtained after completing the activate_virtenv commands below:

--

[!domain!.rhcloud.com bin]\> `python ./activate_virtenv`
  File "./activate_virtenv", line 2
    export LD_LIBRARY_PATH="${OPENSHIFT_PYTHON_DIR}/opt/lib:${LD_LIBRARY_PATH}"

                         ^
SyntaxError: invalid syntax

[!domain!.rhcloud.com bin]\> python ./activate_virtenv
  File "./activate_virtenv", line 2
    export LD_LIBRARY_PATH="${OPENSHIFT_PYTHON_DIR}/opt/lib:${LD_LIBRARY_PATH}"

                         ^
SyntaxError: invalid syntax

[!domain!.rhcloud.com bin]\> cat activate_virtenv
#  Set the library path so that the python shared library can be found.
export LD_LIBRARY_PATH="${OPENSHIFT_PYTHON_DIR}/opt/lib:${LD_LIBRARY_PATH}"
export LIBRARY_PATH="${OPENSHIFT_PYTHON_DIR}/opt/lib:${LIBRARY_PATH}"
source ${OPENSHIFT_PYTHON_DIR}/virtenv/bin/activate
[!domain!.rhcloud.com bin]\> echo $LD_LIBRARY_PATH
/var/lib/openshift/!appcode!/python//opt/lib
[!domain!.rhcloud.com bin]\> echo $LIBRARY_PATH

[!domain!.rhcloud.com !appcode!\> echo $OPENSHIFT_PYTHON_DIR
/var/lib/openshift/!appcode!/python/
[!domain!.rhcloud.com bin]\> echo $OPENSHIFT_PYTHON

[!domain!.rhcloud.com bin]\> ./activate_virtenv

--

  The application also still reports 503 Service Temporarily Unavailable.  Thanks.
Comment 12 Wei Sun 2013-07-11 00:47:40 EDT
Test this bug on INT(devenv_3481). Accroding to #Comment 3 and #Comment 8 ,pip to install MySQL-python, install MySQL-Python=1.2.3 by adding it to the setup.py and run  `python ~/python/bin/activate_virtenv`,all the results are same with above results.
But the application can be accessed.
Comment 13 Wei Sun 2013-07-11 00:48:23 EDT
Created attachment 771972 [details]
app
Comment 14 Luke Macken 2013-07-15 13:27:51 EDT
(In reply to Daniel Pronych from comment #11)
> Please see the following results.  The above attachments have been obtained
> after completing the activate_virtenv commands below:
> 
> --
> 
> [!domain!.rhcloud.com bin]\> `python ./activate_virtenv`
>   File "./activate_virtenv", line 2
>     export
> LD_LIBRARY_PATH="${OPENSHIFT_PYTHON_DIR}/opt/lib:${LD_LIBRARY_PATH}"

It looks like you need to `source ~/python/bin/activate_virtenv` instead.
Comment 15 Daniel Pronych 2013-07-21 11:20:29 EDT
Good day,

  I was able to run the above command with no output; however, the application, through the web browser, is still reporting 503 Service Temporarily Unavailable.  Also, from the command, I executed easy_install mysql-python and obtained the following results:

(virtenv)[!domain!.rhcloud.com !appcode!]\> easy_install mysql-py
thon
Searching for mysql-python
Best match: MySQL-python 1.2.3
Processing MySQL_python-1.2.3-py2.7-linux-x86_64.egg
MySQL-python 1.2.3 is already the active version in easy-install.pth

Using /var/lib/openshift/!appcode!/python/virtenv/lib/python2.7/s
ite-packages/MySQL_python-1.2.3-py2.7-linux-x86_64.egg
Processing dependencies for mysql-python
Finished processing dependencies for mysql-python

  Thanks!
Comment 16 Daniel Pronych 2013-07-21 11:29:44 EDT
Also, here is further information from appserver.log and the environment variable's value which also indicates possible permissions issues:

--
Location of appserver.log
--

[!domain!.rhcloud.com logs]\> pwd
/var/lib/openshift/!appcode!/python/logs

--
Ending portion of appserver.log:
--

...
  File "build/bdist.linux-x86_64/egg/pkg_resources.py", line 914, in resource_fi
lename
  File "build/bdist.linux-x86_64/egg/pkg_resources.py", line 1601, in get_resour
ce_filename
  File "build/bdist.linux-x86_64/egg/pkg_resources.py", line 1657, in _extract_r
esource
  File "build/bdist.linux-x86_64/egg/pkg_resources.py", line 960, in extraction_
error
pkg_resources.ExtractionError: Can't extract file(s) to egg cache

The following error occurred while trying to extract file(s) to the Python egg
cache:

  [Errno 2] No such file or directory: '~//virtenv/lib/python2.7/site-packages'

The Python egg cache directory is currently set to:

  ~//virtenv/lib/python2.7/site-packages

Perhaps your account does not have write access to this directory?  You can
change the cache directory by setting the PYTHON_EGG_CACHE environment
variable to point to an accessible directory.

--
Value of PYTHON_EGG_CACHE environment variable:
--

[!domain!.rhcloud.com logs]\> echo $PYTHON_EGG_CACHE
/var/lib/openshift/!domain!/python//virtenv/.python-eggs


  Thanks!
Comment 17 Mrunal Patel 2013-07-22 11:09:02 EDT
Hi Daniel,
Instead of doing easy_install by hand, could you retry by adding MySQL-python==1.2.3 to your setup.py and git push with force_clean_build marker?

Thanks,
Mrunal
Comment 18 Daniel Pronych 2013-08-01 23:15:20 EDT
Mrunal,

  Thanks, the only change I made was adding this marker as I already that exact code in my setup.py file, here are some highlights from the code submission:

...
  - Found a
.openshift/markers/force_clean_build marker - recreating virtenv ...
...
Reading
http://mirror1.ops.rhcloud.com/mirror/python/web/simple/MySQL-python/
Best
match: MySQL-python 1.2.3
Downloading
http://mirror1.ops.rhcloud.com/mirror/python/web/packages/source/M/MySQL-python/MySQL-python-1.2.3.tar.gz#md5=215eddb6d853f6f4be5b4afc4154292f
...

  The main website page still returns the following:

503 Service Temporarily Unavailable

  Please advise.  Thanks.
Comment 19 chunchen 2013-08-19 01:46:45 EDT
Test this bug on devenv_3672 with new python-2.7, there did not meet this errors, please refer to the following results:

1. Install MySQL-python by setup.py
1) create a python-2.7 app
rhc app create cpy27 python-2.7
2) Add MySQL-python to the setup.py
3) git push
<------------snip------------>
remote: running build_ext
remote: Creating /var/lib/openshift/93ef8bfe087d11e3befd22000ab311b3/python/virtenv/lib/python2.7/site-packages/YourAppName.egg-link (link to .)
remote: Adding YourAppName 1.0 to easy-install.pth file
remote: 
remote: Installed /var/lib/openshift/93ef8bfe087d11e3befd22000ab311b3/python/virtenv
remote: Processing dependencies for YourAppName==1.0
remote: Searching for MySQL-python==1.2.3
remote: Best match: MySQL-python 1.2.3
remote: Adding MySQL-python 1.2.3 to easy-install.pth file
remote: 
remote: Using /opt/rh/python27/root/usr/lib64/python2.7/site-packages
remote: Finished processing dependencies for YourAppName==1.0
remote: Script /var/lib/openshift/93ef8bfe087d11e3befd22000ab311b3/python//virtenv/bin/activate.csh cannot be made relative (it's not a normal script that starts with #!/var/lib/openshift/93ef8bfe087d11e3befd22000ab311b3/python/virtenv/bin/python)
remote: Script /var/lib/openshift/93ef8bfe087d11e3befd22000ab311b3/python//virtenv/bin/activate.fish cannot be made relative (it's not a normal script that starts with #!/var/lib/openshift/93ef8bfe087d11e3befd22000ab311b3/python/virtenv/bin/python)
remote: Starting application cpy27
remote: Script /var/lib/openshift/93ef8bfe087d11e3befd22000ab311b3/python//virtenv/bin/activate.csh cannot be made relative (it's not a normal script that starts with #!/var/lib/openshift/93ef8bfe087d11e3befd22000ab311b3/python/virtenv/bin/python)
remote: Script /var/lib/openshift/93ef8bfe087d11e3befd22000ab311b3/python//virtenv/bin/activate.fish cannot be made relative (it's not a normal script that starts with #!/var/lib/openshift/93ef8bfe087d11e3befd22000ab311b3/python/virtenv/bin/python)
remote: Starting PYTHON cart
<------------snip------------>


2. Install MySQL-python by setup.py with force_clean_build marker
1) create a python-2.7 app
rhc app create cpy272 python-2.7
2) Add MySQL-python to the setup.py and add force_clean_build marker
3) git push
remote: .openshift/markers/force_clean_build found!
remote: Recreating virtenv
remote: Script /var/lib/openshift/848253975478715312766976/python//virtenv/bin/activate.csh cannot be made relative (it's not a normal script that starts with #!/var/lib/openshift/848253975478715312766976/python/virtenv/bin/python)
remote: Script /var/lib/openshift/848253975478715312766976/python//virtenv/bin/activate.fish cannot be made relative (it's not a normal script that starts with #!/var/lib/openshift/848253975478715312766976/python/virtenv/bin/python)
remote: running develop
remote: running egg_info
remote: creating YourAppName.egg-info
remote: writing requirements to YourAppName.egg-info/requires.txt
remote: writing YourAppName.egg-info/PKG-INFO
remote: writing top-level names to YourAppName.egg-info/top_level.txt
remote: writing dependency_links to YourAppName.egg-info/dependency_links.txt
remote: writing requirements to YourAppName.egg-info/requires.txt
remote: writing YourAppName.egg-info/PKG-INFO
remote: writing top-level names to YourAppName.egg-info/top_level.txt
remote: writing dependency_links to YourAppName.egg-info/dependency_links.txt
remote: writing manifest file 'YourAppName.egg-info/SOURCES.txt'
remote: warning: manifest_maker: standard file 'setup.py' not found
remote: 
remote: reading manifest file 'YourAppName.egg-info/SOURCES.txt'
remote: writing manifest file 'YourAppName.egg-info/SOURCES.txt'
remote: running build_ext
remote: Creating /var/lib/openshift/848253975478715312766976/python/virtenv/lib/python2.7/site-packages/YourAppName.egg-link (link to .)
remote: Adding YourAppName 1.0 to easy-install.pth file
remote: 
remote: Installed /var/lib/openshift/848253975478715312766976/python/virtenv
remote: Processing dependencies for YourAppName==1.0
remote: Searching for gevent
remote: Reading http://mirror1.ops.rhcloud.com/mirror/python/web/simple/gevent/
remote: Best match: gevent 0.13.8
remote: Downloading http://mirror1.ops.rhcloud.com/mirror/python/web/packages/source/g/gevent/gevent-0.13.8.tar.gz#md5=ca9dcaa7880762d8ebbc266b11252960
remote: Processing gevent-0.13.8.tar.gz
remote: Running gevent-0.13.8/setup.py -q bdist_egg --dist-dir /tmp/easy_install-UqAz3k/gevent-0.13.8/egg-dist-tmp-MmhIP8
remote: zip_safe flag not set; analyzing archive contents...
remote: Linking /tmp/easy_install-UqAz3k/gevent-0.13.8/build/lib.linux-x86_64-2.7/gevent/core.so to /tmp/easy_install-UqAz3k/gevent-0.13.8/gevent/core.so
remote: Adding gevent 0.13.8 to easy-install.pth file
remote: 
remote: Installed /var/lib/openshift/848253975478715312766976/python/virtenv/lib/python2.7/site-packages/gevent-0.13.8-py2.7-linux-x86_64.egg
remote: Searching for greenlet
remote: Reading http://mirror1.ops.rhcloud.com/mirror/python/web/simple/greenlet/
remote: Best match: greenlet 0.4.1
remote: Downloading http://mirror1.ops.rhcloud.com/mirror/python/web/packages/source/g/greenlet/greenlet-0.4.1.zip#md5=c2deda75bdda59c38cae12a77cc53adc
remote: Processing greenlet-0.4.1.zip
remote: Running greenlet-0.4.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-MA27tA/greenlet-0.4.1/egg-dist-tmp-txDIgf
remote: In file included from slp_platformselect.h:12,
remote:                  from greenlet.c:416:
remote: platform/switch_amd64_unix.h:40: warning: 'noclone' attribute directive ignored
remote: platform/switch_amd64_unix.h:43: warning: 'noclone' attribute directive ignored
remote: greenlet.c: In function 'g_switch':
remote: greenlet.c:543: warning: 'err' may be used uninitialized in this function
remote: Adding greenlet 0.4.1 to easy-install.pth file
remote: 
remote: Installed /var/lib/openshift/848253975478715312766976/python/virtenv/lib/python2.7/site-packages/greenlet-0.4.1-py2.7-linux-x86_64.egg
remote: Searching for MySQL-python==1.2.3
remote: Best match: MySQL-python 1.2.3
remote: Adding MySQL-python 1.2.3 to easy-install.pth file
remote: 
remote: Using /opt/rh/python27/root/usr/lib64/python2.7/site-packages
remote: Finished processing dependencies for YourAppName==1.0
remote: Script /var/lib/openshift/848253975478715312766976/python//virtenv/bin/activate.csh cannot be made relative (it's not a normal script that starts with #!/var/lib/openshift/848253975478715312766976/python/virtenv/bin/python)
remote: Script /var/lib/openshift/848253975478715312766976/python//virtenv/bin/activate.fish cannot be made relative (it's not a normal script that starts with #!/var/lib/openshift/848253975478715312766976/python/virtenv/bin/python)
remote: Starting application cpy272
remote: Script /var/lib/openshift/848253975478715312766976/python//virtenv/bin/activate.csh cannot be made relative (it's not a normal script that starts with #!/var/lib/openshift/848253975478715312766976/python/virtenv/bin/python)
remote: Script /var/lib/openshift/848253975478715312766976/python//virtenv/bin/activate.fish cannot be made relative (it's not a normal script that starts with #!/var/lib/openshift/848253975478715312766976/python/virtenv/bin/python)

3. Install MySQL-python by pip in rhcsh
1) create a python-2.7 app
rhc app create cpy271 python-2.7
2) SSH into this app
rhc ssh cpy271
3) \> source ${OPENSHIFT_PYTHON_DIR}/virtenv/bin/activate
4) \> pip install MySQL-python
Requirement already satisfied (use --upgrade to upgrade): MySQL-python in /opt/rh/python27/root/usr/lib64/python2.7/site-packages
Cleaning up...
5) \> pip install MySQL-python --upgrade
Downloading/unpacking MySQL-python from https://pypi.python.org/packages/source/M/MySQL-python/MySQL-python-1.2.4.zip#md5=ddf2386daf10a97af115ffad2ed4a9a0
  Downloading MySQL-python-1.2.4.zip (113kB): 113kB downloaded
  Running setup.py egg_info for package MySQL-python
    Downloading http://pypi.python.org/packages/source/d/distribute/distribute-0.6.28.tar.gz
    Extracting in /tmp/tmpHKQeXo
    Now working in /tmp/tmpHKQeXo/distribute-0.6.28
    Building a Distribute egg in /var/lib/openshift/662418933441223156826112/python/virtenv/build/MySQL-python
    /var/lib/openshift/662418933441223156826112/python/virtenv/build/MySQL-python/distribute-0.6.28-py2.7.egg
    
Installing collected packages: MySQL-python
  Found existing installation: MySQL-python 1.2.3
    Not uninstalling MySQL-python at /opt/rh/python27/root/usr/lib64/python2.7/site-packages, outside environment /var/lib/openshift/662418933441223156826112/python/virtenv
  Running setup.py install for MySQL-python
    building '_mysql' extension
    gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -Dversion_info=(1,2,4,'final',1) -D__version__=1.2.4 -I/usr/include/mysql -I/opt/rh/python27/root/usr/include/python2.7 -c _mysql.c -o build/temp.linux-x86_64-2.7/_mysql.o -g -pipe -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -fno-strict-aliasing -fwrapv -fPIC -DUNIV_LINUX -DUNIV_LINUX
    In file included from /usr/include/mysql/my_config.h:14,
                     from _mysql.c:44:
    /usr/include/mysql/my_config_x86_64.h:1082:1: warning: "HAVE_WCSCOLL" redefined
    In file included from /opt/rh/python27/root/usr/include/python2.7/pyconfig.h:6,
                     from /opt/rh/python27/root/usr/include/python2.7/Python.h:8,
                     from _mysql.c:29:
    /opt/rh/python27/root/usr/include/python2.7/pyconfig-64.h:908:1: warning: this is the location of the previous definition
    gcc -pthread -shared build/temp.linux-x86_64-2.7/_mysql.o -L/usr/lib64/mysql -L/opt/rh/python27/root/usr/lib64 -lmysqlclient_r -lz -lpthread -lcrypt -lnsl -lm -lpthread -lssl -lcrypto -lpython2.7 -o build/lib.linux-x86_64-2.7/_mysql.so
    
Successfully installed MySQL-python
Cleaning up...
Comment 21 Ben Parees 2014-04-10 14:56:55 EDT
This appears to have been resolved by the mysql-python libraries being site-installed.
Comment 22 Wenjing Zheng 2014-04-11 02:05:21 EDT
Verified on INT(devenv_4654), can get the same results like comment #19: MySQL-python can be installed successfully.

Note You need to log in before you can comment on or make changes to this bug.