Red Hat Bugzilla – Bug 802387
Some default settings in stickshift-node.conf are improper to be used in devenv instance
Last modified: 2014-06-18 03:24:26 EDT
Description of problem: Trigger jenkins build and failed. $ git commit -a -m "xx"; git push [master 71b5e8f] xx 1 files changed, 1 insertions(+), 1 deletions(-) Failed to add the host to the list of known hosts (/dev/nul). Counting objects: 5, done. Delta compression using up to 4 threads. Compressing objects: 100% (3/3), done. Writing objects: 100% (3/3), 272 bytes, done. Total 3 (delta 2), reused 0 (delta 0) remote: Executing Jenkins build. remote: remote: You can track your build at https://jenkins-jialiu.dev.rhcloud.com/job/phptest-build remote: remote: Waiting for build to schedule......... remote: **BUILD FAILED/CANCELLED** remote: Please see the Jenkins log for more details via rhc-tail-files remote: !!!!!!!! remote: Deployment Halted! remote: If the build failed before the deploy step, your previous remote: build is still running. Otherwise, your application may be remote: partially deployed or inaccessible. remote: Fix the build and try again. remote: !!!!!!!! To ssh://c1e92db8235d4005987009bf831a4c13@phptest-jialiu.dev.rhcloud.com/~/git/phptest.git/ 0afaf33..71b5e8f master -> master Version-Release number of selected component (if applicable): rhc-cartridge-jenkins-1.4-0.88.3-1.el6_2.noarch rhc-cartridge-jenkins-client-1.4-0.22.2-1.el6_2.noarch jenkins-plugin-openshift-0.4.28-1.el6_2.x86_64 rubygem-stickshift-controller-0.6.5-1.el6_2.noarch stickshift-abstract-0.6.5-1.el6_2.noarch rubygem-stickshift-node-0.6.6-1.el6_2.noarch rubygem-stickshift-common-0.6.4-1.el6_2.noarch rhc-node-0.88.4-1.el6_2.x86_64 How reproducible: Always Steps to Reproduce: 1. Create an app with jenkins enabled. 2. Do some change, then trigger jenkins build. 3. Actual results: jenkins build failed. jenkins log message: <--snip--> Mar 12, 2012 8:30:44 AM hudson.plugins.openshift.OpenShiftCloud provision INFO: Provisioning new node for workload = 1 and label = phptest-build com.openshift.express.client.OpenShiftEndpointException: Could not get user info for user "" at "https://openshift.redhat.com/broker/userinfo" at com.openshift.express.client.OpenShiftService.sendRequest(OpenShiftService.java:529) at com.openshift.express.client.OpenShiftService.getUserInfo(OpenShiftService.java:126) at hudson.plugins.openshift.OpenShiftCloud.getUserInfo(OpenShiftCloud.java:401) at hudson.plugins.openshift.OpenShiftCloud.getSlaves(OpenShiftCloud.java:618) at hudson.plugins.openshift.OpenShiftCloud.provision(OpenShiftCloud.java:439) at hudson.slaves.NodeProvisioner.update(NodeProvisioner.java:203) at hudson.slaves.NodeProvisioner.access$000(NodeProvisioner.java:54) at hudson.slaves.NodeProvisioner$NodeProvisionerInvoker.doRun(NodeProvisioner.java:286) at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:54) at java.util.TimerThread.mainLoop(Timer.java:534) at java.util.TimerThread.run(Timer.java:484) Caused by: com.openshift.express.internal.client.httpclient.InternalServerErrorException: {"debug":"padding check failed/var/www/libra/broker/lib/express/broker/auth_service.rb:109:in `private_decrypt'\n/var/www/libra/broker/lib/express/broker/auth_service.rb:109:in `check_broker_key'\n/var/www/libra/broker/lib/express/broker/auth_service.rb:50:in `login'\n/usr/lib/ruby/gems/1.8/gems/cloud-sdk-controller-0.5.10/lib/cloud-sdk-controller/app/controllers/legacy_broker_controller.rb:349:in `authenticate'\n/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/callbacks.rb:447:in `_run__688730835__process_action__927155398__callbacks'\n/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/callbacks.rb:410:in `send'","api":"1.1.2","data":"","api_c":["placeholder"],"broker_c":["namespace","rhlogin","ssh","app_uuid","debug","alter","cartridge","cart_type","action","app_name","api"],"exit_code":1,"messages":"","result":"padding check failed"} at com.openshift.express.internal.client.httpclient.UrlConnectionHttpClient.createException(UrlConnectionHttpClient.java:107) at com.openshift.express.internal.client.httpclient.UrlConnectionHttpClient.post(UrlConnectionHttpClient.java:75) at com.openshift.express.client.OpenShiftService.sendRequest(OpenShiftService.java:518) ... 10 more Caused by: java.io.IOException: Server returned HTTP response code: 500 for URL: https://openshift.redhat.com/broker/userinfo at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1403) at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) at com.openshift.express.internal.client.httpclient.UrlConnectionHttpClient.post(UrlConnectionHttpClient.java:70) ... 11 more <--snip--> Expected results: jenkins build should be succesfull. Additional info: Go to jenkins UI - > Manage Jenkins -> "OpenShift Broker Hostname", found jenkins job is using "openshift.redhat.com" Check /etc/libra/node.conf: # Libra server libra_server='localhost' Looks like jenkins server does not read libra_server value from /etc/libra/node.conf,
Refer to https://bugzilla.redhat.com/show_bug.cgi?id=802258#c1 for workaround about app creation failure.
Now all setting is located in /etc/stickshift/stickshift-node.conf, the root cause of this bug is improper default settings in stickshift-node.conf. # cat /etc/stickshift/stickshift-node.conf <--snip--> CLOUD_DOMAIN="dev.rhcloud.com" BROKER_HOST="openshift.redhat.com" CREATE_APP_SYMLINKS=0 <--snip--> BROKER_HOST setting is used by jenkins app, but for devenv environment, "localhost" is more proper here. BTW, "CREATE_APP_SYMLINKS" should be set to "1" in devenv environment, that will be more convenient for debugging. I know user can modify this config file manually, but tester must modify this file every time after launch a new instance. If these two default settings are modified to be more proper for devenv environment, that will bring much convenience for QE's testing. Because the root cause of this issue is incorrect setting in config file, low priority to "medium".
Verified this bug with devenv_1661, and PASS. # cat /etc/stickshift/stickshift-node.conf <--snip--> BROKER_HOST="localhost" CREATE_APP_SYMLINKS=1 <--snip-->