Description of problem: If try to do jenkins build for python-2.7 or python-3.3 apps, it would be failed. For python-2.7, no jenkins slave created for connection. For python-3.3, it would display this error: rsync: change_dir "/var/lib/openshift/510f90c08ec7260e070003ba/python-3.3/venv" failed: No such file or directory (2) rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1505) [receiver=3.0.6] Build step 'Execute shell' marked build as failure Archiving artifacts Finished: FAILURE Version-Release number of selected component (if applicable): devenv_2765 rhc-1.4.4.gem How reproducible: always Steps to Reproduce: 1.Create a python-2.7/python-3.3 app 2.Add jenkins client to the app 3.Make some changes in local repo and git push. Actual results: It's failed for both cartridges. For python-2.7 app, the jenkins slave couldn't create and here's the log: Feb 04, 2013 6:15:34 AM hudson.plugins.openshift.OpenShiftCloud reloadConfig INFO: Retrieving config XML from http://127.1.5.1:8080/job/app7-build/config.xml Feb 04, 2013 6:15:34 AM hudson.plugins.openshift.OpenShiftCloud createConnection INFO: Using credentials system_builder:krxGHGyWUtbT Feb 04, 2013 6:15:34 AM hudson.plugins.openshift.OpenShiftCloud provision WARNING: Caught java.lang.RuntimeException: Received empty config XML from API call to http://127.1.5.1:8080/job/app7-build/config.xml. Will retry 4 more times before cancelling build. Feb 04, 2013 6:15:39 AM hudson.plugins.openshift.OpenShiftCloud hasCapacity INFO: Checking capacity Feb 04, 2013 6:15:39 AM hudson.plugins.openshift.OpenShiftCloud hasCapacity INFO: User has consumed 2 of 3 gears. Feb 04, 2013 6:15:39 AM hudson.plugins.openshift.OpenShiftCloud reloadConfig INFO: Reloading configuration for app7-build... Feb 04, 2013 6:15:39 AM hudson.plugins.openshift.OpenShiftCloud reloadConfig INFO: Retrieving config XML from http://127.1.5.1:8080/job/app7-build/config.xml Feb 04, 2013 6:15:39 AM hudson.plugins.openshift.OpenShiftCloud createConnection INFO: Using credentials system_builder:krxGHGyWUtbT Feb 04, 2013 6:15:39 AM hudson.plugins.openshift.OpenShiftCloud provision WARNING: Caught java.lang.RuntimeException: Received empty config XML from API call to http://127.1.5.1:8080/job/app7-build/config.xml. Will retry 3 more times before cancelling build. Feb 04, 2013 6:15:44 AM hudson.plugins.openshift.OpenShiftCloud hasCapacity INFO: Checking capacity Feb 04, 2013 6:15:44 AM hudson.plugins.openshift.OpenShiftCloud hasCapacity INFO: User has consumed 2 of 3 gears. Feb 04, 2013 6:15:44 AM hudson.plugins.openshift.OpenShiftCloud reloadConfig INFO: Reloading configuration for app7-build... Feb 04, 2013 6:15:44 AM hudson.plugins.openshift.OpenShiftCloud reloadConfig INFO: Retrieving config XML from http://127.1.5.1:8080/job/app7-build/config.xml Feb 04, 2013 6:15:44 AM hudson.plugins.openshift.OpenShiftCloud createConnection INFO: Using credentials system_builder:krxGHGyWUtbT Feb 04, 2013 6:15:44 AM hudson.plugins.openshift.OpenShiftCloud provision WARNING: Caught java.lang.RuntimeException: Received empty config XML from API call to http://127.1.5.1:8080/job/app7-build/config.xml. Will retry 2 more times before cancelling build. Feb 04, 2013 6:15:49 AM hudson.plugins.openshift.OpenShiftCloud hasCapacity INFO: Checking capacity Feb 04, 2013 6:15:49 AM hudson.plugins.openshift.OpenShiftCloud hasCapacity INFO: User has consumed 2 of 3 gears. Feb 04, 2013 6:15:49 AM hudson.plugins.openshift.OpenShiftCloud reloadConfig INFO: Reloading configuration for app7-build... Feb 04, 2013 6:15:49 AM hudson.plugins.openshift.OpenShiftCloud reloadConfig INFO: Retrieving config XML from http://127.1.5.1:8080/job/app7-build/config.xml Feb 04, 2013 6:15:49 AM hudson.plugins.openshift.OpenShiftCloud createConnection INFO: Using credentials system_builder:krxGHGyWUtbT Feb 04, 2013 6:15:49 AM hudson.plugins.openshift.OpenShiftCloud provision WARNING: Caught java.lang.RuntimeException: Received empty config XML from API call to http://127.1.5.1:8080/job/app7-build/config.xml. Will retry 1 more times before cancelling build. Feb 04, 2013 6:15:54 AM hudson.plugins.openshift.OpenShiftCloud hasCapacity INFO: Checking capacity Feb 04, 2013 6:15:54 AM hudson.plugins.openshift.OpenShiftCloud hasCapacity INFO: User has consumed 2 of 3 gears. Feb 04, 2013 6:15:54 AM hudson.plugins.openshift.OpenShiftCloud reloadConfig INFO: Reloading configuration for app7-build... Feb 04, 2013 6:15:54 AM hudson.plugins.openshift.OpenShiftCloud reloadConfig INFO: Retrieving config XML from http://127.1.5.1:8080/job/app7-build/config.xml Feb 04, 2013 6:15:54 AM hudson.plugins.openshift.OpenShiftCloud createConnection INFO: Using credentials system_builder:krxGHGyWUtbT Feb 04, 2013 6:15:54 AM hudson.plugins.openshift.OpenShiftCloud provision WARNING: Caught java.lang.RuntimeException: Received empty config XML from API call to http://127.1.5.1:8080/job/app7-build/config.xml. Will retry 0 more times before cancelling build. Feb 04, 2013 6:15:59 AM hudson.plugins.openshift.OpenShiftCloud provision WARNING: Cancelling build due to earlier exceptions Feb 04, 2013 6:15:59 AM hudson.plugins.openshift.OpenShiftCloud cancelItem INFO: Cancelling Item Feb 04, 2013 6:15:59 AM hudson.plugins.openshift.OpenShiftCloud cancelItem WARNING: Build app7-build app7bldr has been cancelled For python-3.3, it would display following error: Checkout:py33-build / python-3.3/ci/jenkins/workspace/py33-build - hudson.remoting.Channel@1f0beab:py33bldr Using strategy: Default Checkout:py33-build / python-3.3/ci/jenkins/workspace/py33-build - hudson.remoting.LocalChannel@1c900b6 Cloning the remote Git repository Cloning repository origin Fetching upstream changes from ssh://510f524b8ec726080a000035.rhcloud.com/~/git/py33.git/ Seen branch in repository origin/HEAD Seen branch in repository origin/master Commencing build of Revision ca31d32b08b154be7ba88d5311cdeb3f14e1cd0c (origin/HEAD, origin/master) Checking out Revision ca31d32b08b154be7ba88d5311cdeb3f14e1cd0c (origin/HEAD, origin/master) No change to record in branch origin/HEAD No change to record in branch origin/master [py33-build] $ /bin/sh -xe /tmp/hudson7668693307331999973.sh + source /usr/libexec/openshift/cartridges/abstract/info/lib/jenkins_util + venv_excludes='--exclude=bin/activate --exclude=pyvenv.cfg' + alias 'rsync=rsync --exclude=bin/activate --exclude=pyvenv.cfg --delete-after -az -e '\''/usr/libexec/openshift/cartridges/jenkins-1.4/info/bin/git_ssh_wrapper.sh'\''' + rsync --exclude=bin/activate --exclude=pyvenv.cfg --delete-after -az -e /usr/libexec/openshift/cartridges/jenkins-1.4/info/bin/git_ssh_wrapper.sh '510f524b8ec726080a000035.rhcloud.com:~/python-3.3/venv/' /var/lib/openshift/510f8e868ec7260e07000361/python-3.3/venv/ rsync: change_dir "/var/lib/openshift/510f524b8ec726080a000035/python-3.3/venv" failed: No such file or directory (2) rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1505) [receiver=3.0.6] Build step 'Execute shell' marked build as failure Archiving artifacts Finished: FAILURE Expected results: Jenkins build succeed. Additional info:
The problem seems to be only with the python-3.3 cartridge, and is due to a typo in the Jenkins configuration (referencing 'venv' rather than 'virtenv'). Pull request forthcoming.
Created attachment 692971 [details] New build failure After correcting the virtualenv path in the build template, a new error with the build script was revealed. I've attached the Jenkins build output demonstrating the new failure.
Need to check with Jimmy and Steve to see if we can wait one more sprint before pushing this to prod.
Fixed with pull request: https://github.com/openshift/origin-server/pull/1306 waiting on merge ...
Tried on latest devenv_2772 and found python-3.3 still had same behavior as on devenv_2765. However, python-2.7 behaved as different as before. For python-2.7 app, it could create jenkins slave for collection. But it's failed to build as below: Started by user Jenkins System Builder Building remotely on app1bldr in workspace python-2.7/ci/jenkins/workspace/app1-build Checkout:app1-build / python-2.7/ci/jenkins/workspace/app1-build - hudson.remoting.Channel@128492b:app1bldr Using strategy: Default Checkout:app1-build / python-2.7/ci/jenkins/workspace/app1-build - hudson.remoting.LocalChannel@721257 Cloning the remote Git repository Cloning repository origin Fetching upstream changes from ssh://511093ecb778086621000009.rhcloud.com/~/git/app1.git/ Seen branch in repository origin/HEAD Seen branch in repository origin/master Commencing build of Revision 80d19161bbf363e0e7a1c35867b4604b9dbb472c (origin/HEAD, origin/master) Checking out Revision 80d19161bbf363e0e7a1c35867b4604b9dbb472c (origin/HEAD, origin/master) No change to record in branch origin/HEAD No change to record in branch origin/master [app1-build] $ /bin/sh -xe /tmp/hudson6388535557779083426.sh + source /usr/libexec/openshift/cartridges/abstract/info/lib/jenkins_util + alias 'rsync=rsync --delete-after -az -e '\''/usr/libexec/openshift/cartridges/jenkins-1.4/info/bin/git_ssh_wrapper.sh'\''' + rsync --delete-after -az -e /usr/libexec/openshift/cartridges/jenkins-1.4/info/bin/git_ssh_wrapper.sh '511093ecb778086621000009.rhcloud.com:~/python-2.7/virtenv/' /var/lib/openshift/5110989ab778085fa500002d/python-2.7/virtenv/ + . ci_build.sh ++ set +x Running .openshift/action_hooks/pre_build Traceback (most recent call last): File "/usr/lib/python2.6/site-packages/virtualenv.py", line 2098, in <module> main() File "/usr/lib/python2.6/site-packages/virtualenv.py", line 928, in main never_download=options.never_download) File "/usr/lib/python2.6/site-packages/virtualenv.py", line 1029, in create_environment site_packages=site_packages, clear=clear)) File "/usr/lib/python2.6/site-packages/virtualenv.py", line 1164, in install_python copyfile(join(stdlib_dir, fn), join(lib_dir, fn)) File "/usr/lib/python2.6/site-packages/virtualenv.py", line 430, in copyfile copyfileordir(src, dest) File "/usr/lib/python2.6/site-packages/virtualenv.py", line 405, in copyfileordir shutil.copytree(src, dest, True) File "/var/lib/openshift/5110989ab778085fa500002d/python-2.7/opt/lib/python2.7/shutil.py", line 175, in copytree os.makedirs(dst) File "/var/lib/openshift/5110989ab778085fa500002d//python-2.7/opt/lib/python2.7/os.py", line 157, in makedirs mkdir(name, mode) OSError: [Errno 17] File exists: '/var/lib/openshift/5110989ab778085fa500002d//python-2.7/virtenv/lib/python2.7/lib-dynload' Running virtualenv with interpreter /var/lib/openshift/5110989ab778085fa500002d//python-2.7/opt/bin/python Build step 'Execute shell' marked build as failure Archiving artifacts Finished: FAILURE
Code's not yet in devenv/master - still waiting on the merge -- the fix is in the pull request mentioned above : https://github.com/openshift/origin-server/pull/1306 Still waiting on it to merge in. Will be up for a bit - if it does will put the bug on QA or you can do it if the merge happens. Thanks.
Commit pushed to master at https://github.com/openshift/origin-server https://github.com/openshift/origin-server/commit/a01c9aa5482a89e470c808a9d4a8cf4ca2a3c422 Bug 907426 - [US3327]Jenkins build failed for python-2.7/python-3.3 apps
This bug will be verified as soon as we get the latest build(might be devenv_2774) with fix checked in.
This bug is fixed on devenv_2778. Jenkins build can be succeed for python-2.7 and python-3.3 apps as below: Counting objects: 7, done. Delta compression using up to 8 threads. Compressing objects: 100% (4/4), done. Writing objects: 100% (4/4), 362 bytes, done. Total 4 (delta 2), reused 0 (delta 0) remote: restart_on_add=false remote: Executing Jenkins build. remote: remote: You can track your build at https://jenk2-dev2788dmt3.dev.rhcloud.com/job/py33-build remote: remote: Waiting for build to schedule.........................................................Done remote: Waiting for job to complete.................................Done remote: SUCCESS remote: New build has been deployed. To ssh://5111d030bfe8acb793000175.rhcloud.com/~/git/py33.git/ 96254c7..c197f6c master -> master