Bug 802686 - Jenkins hang there forever when slave app can not be created due to no node available.
Jenkins hang there forever when slave app can not be created due to no node a...
Status: CLOSED CURRENTRELEASE
Product: OpenShift Origin
Classification: Red Hat
Component: Containers (Show other bugs)
1.x
Unspecified Unspecified
high Severity low
: ---
: ---
Assigned To: Bill DeCoste
libra bugs
: Triaged
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2012-03-13 05:43 EDT by Johnny Liu
Modified: 2012-04-13 14:33 EDT (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2012-04-13 14:33:12 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Johnny Liu 2012-03-13 05:43:34 EDT
Description of problem:
jenkins will hang there for ever when fail to create slave app due to some reasons, e.g: slave app DNS fail to be resolved, no available node.


Version-Release number of selected component (if applicable):
jenkins-plugin-openshift-0.5.0-1.el6_2.x86_64

How reproducible:
Always

Steps to Reproduce:
1. Create a app with jenkins client enabled.
2. Change node profile to make "No nodes available" come true.
e.g:
Here I create a jboss app with jenkins client enabled, jboss slave app need find out a "medium" node to create slave app on it. But my node is "small".
3. Do some change to trigger jenkins build.
  
Actual results:
jenkins build will hang there for ever.

$ git commit -a -mx; git push
[master 349ad4e] x
 1 files changed, 1 insertions(+), 1 deletions(-)
Counting objects: 5, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 271 bytes, done.
Total 3 (delta 2), reused 0 (delta 0)
remote: Executing Jenkins build.
remote: 
remote: You can track your build at https://jenkins-jialiu.dev.rhcloud.com/job/jbosstest-build
remote: 
remote: Waiting for build to schedule.........................................................................

Expected results:
Any slave app creation failure can cause this issue, so "timeout" is
necessary to avoid this issue.

Additional info:
Comment 1 Johnny Liu 2012-03-13 05:45:37 EDT
Jenkins log:
com.openshift.express.client.OpenShiftEndpointException: Could not configure application "jbosstestbldr" at "https://domU-12-31-39-0B-01-21/broker/cartridge"
	at com.openshift.express.client.OpenShiftService.sendRequest(OpenShiftService.java:529)
	at com.openshift.express.client.OpenShiftService.requestApplicationAction(OpenShiftService.java:435)
	at com.openshift.express.client.OpenShiftService.createApplication(OpenShiftService.java:290)
	at hudson.plugins.openshift.OpenShiftSlave.createApp(OpenShiftSlave.java:225)
	at hudson.plugins.openshift.OpenShiftSlave.provision(OpenShiftSlave.java:194)
	at hudson.plugins.openshift.OpenShiftCloud$2.call(OpenShiftCloud.java:494)
	at hudson.plugins.openshift.OpenShiftCloud$2.call(OpenShiftCloud.java:488)
	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
	at java.util.concurrent.FutureTask.run(FutureTask.java:166)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
	at java.lang.Thread.run(Thread.java:679)
Caused by: com.openshift.express.internal.client.httpclient.InternalServerErrorException: {"broker_c":["namespace","rhlogin","ssh","app_uuid","debug","alter","cartridge","cart_type","action","app_name","api"],"api":"1.1.3","exit_code":1,"debug":"","result":"No nodes available.  If the problem persists please contact Red Hat support.","data":"","messages":"","api_c":["placeholder"]}
	at com.openshift.express.internal.client.httpclient.UrlConnectionHttpClient.createException(UrlConnectionHttpClient.java:107)
	at com.openshift.express.internal.client.httpclient.UrlConnectionHttpClient.post(UrlConnectionHttpClient.java:75)
	at com.openshift.express.client.OpenShiftService.sendRequest(OpenShiftService.java:518)
	... 11 more
Caused by: java.io.IOException: Server returned HTTP response code: 500 for URL: https://domU-12-31-39-0B-01-21/broker/cartridge
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1403)
	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254)
	at com.openshift.express.internal.client.httpclient.UrlConnectionHttpClient.post(UrlConnectionHttpClient.java:70)
	... 12 more
Comment 2 Bill DeCoste 2012-03-24 10:21:00 EDT
fixed in jenkins plugin 5.0.4
Comment 3 Johnny Liu 2012-03-27 07:33:11 EDT
Verification is blocked by Bug 807236, so leave this bug until 807236 is fixed.
Comment 4 Johnny Liu 2012-03-28 23:20:30 EDT
Re-test this bug with jenkins-plugin-openshift-0.5.5-1.el6_2.x86_64, this issue still reproduced.
Comment 5 Bill DeCoste 2012-04-02 15:09:03 EDT
Updated fix in jenkins-plugin-openshift 0.5.6
Comment 6 Johnny Liu 2012-04-05 08:21:37 EDT
Verified this bug with devenv_1696, and PASS.

Note You need to log in before you can comment on or make changes to this bug.