Bug 802686 - Jenkins hang there forever when slave app can not be created due to no node available.
Summary: Jenkins hang there forever when slave app can not be created due to no node a...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: OKD
Classification: Red Hat
Component: Containers
Version: 1.x
Hardware: Unspecified
OS: Unspecified
high
low
Target Milestone: ---
: ---
Assignee: Bill DeCoste
QA Contact: libra bugs
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2012-03-13 09:43 UTC by Johnny Liu
Modified: 2012-04-13 18:33 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2012-04-13 18:33:12 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Johnny Liu 2012-03-13 09:43:34 UTC
Description of problem:
jenkins will hang there for ever when fail to create slave app due to some reasons, e.g: slave app DNS fail to be resolved, no available node.


Version-Release number of selected component (if applicable):
jenkins-plugin-openshift-0.5.0-1.el6_2.x86_64

How reproducible:
Always

Steps to Reproduce:
1. Create a app with jenkins client enabled.
2. Change node profile to make "No nodes available" come true.
e.g:
Here I create a jboss app with jenkins client enabled, jboss slave app need find out a "medium" node to create slave app on it. But my node is "small".
3. Do some change to trigger jenkins build.
  
Actual results:
jenkins build will hang there for ever.

$ git commit -a -mx; git push
[master 349ad4e] x
 1 files changed, 1 insertions(+), 1 deletions(-)
Counting objects: 5, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 271 bytes, done.
Total 3 (delta 2), reused 0 (delta 0)
remote: Executing Jenkins build.
remote: 
remote: You can track your build at https://jenkins-jialiu.dev.rhcloud.com/job/jbosstest-build
remote: 
remote: Waiting for build to schedule.........................................................................

Expected results:
Any slave app creation failure can cause this issue, so "timeout" is
necessary to avoid this issue.

Additional info:

Comment 1 Johnny Liu 2012-03-13 09:45:37 UTC
Jenkins log:
com.openshift.express.client.OpenShiftEndpointException: Could not configure application "jbosstestbldr" at "https://domU-12-31-39-0B-01-21/broker/cartridge"
	at com.openshift.express.client.OpenShiftService.sendRequest(OpenShiftService.java:529)
	at com.openshift.express.client.OpenShiftService.requestApplicationAction(OpenShiftService.java:435)
	at com.openshift.express.client.OpenShiftService.createApplication(OpenShiftService.java:290)
	at hudson.plugins.openshift.OpenShiftSlave.createApp(OpenShiftSlave.java:225)
	at hudson.plugins.openshift.OpenShiftSlave.provision(OpenShiftSlave.java:194)
	at hudson.plugins.openshift.OpenShiftCloud$2.call(OpenShiftCloud.java:494)
	at hudson.plugins.openshift.OpenShiftCloud$2.call(OpenShiftCloud.java:488)
	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
	at java.util.concurrent.FutureTask.run(FutureTask.java:166)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
	at java.lang.Thread.run(Thread.java:679)
Caused by: com.openshift.express.internal.client.httpclient.InternalServerErrorException: {"broker_c":["namespace","rhlogin","ssh","app_uuid","debug","alter","cartridge","cart_type","action","app_name","api"],"api":"1.1.3","exit_code":1,"debug":"","result":"No nodes available.  If the problem persists please contact Red Hat support.","data":"","messages":"","api_c":["placeholder"]}
	at com.openshift.express.internal.client.httpclient.UrlConnectionHttpClient.createException(UrlConnectionHttpClient.java:107)
	at com.openshift.express.internal.client.httpclient.UrlConnectionHttpClient.post(UrlConnectionHttpClient.java:75)
	at com.openshift.express.client.OpenShiftService.sendRequest(OpenShiftService.java:518)
	... 11 more
Caused by: java.io.IOException: Server returned HTTP response code: 500 for URL: https://domU-12-31-39-0B-01-21/broker/cartridge
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1403)
	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254)
	at com.openshift.express.internal.client.httpclient.UrlConnectionHttpClient.post(UrlConnectionHttpClient.java:70)
	... 12 more

Comment 2 Bill DeCoste 2012-03-24 14:21:00 UTC
fixed in jenkins plugin 5.0.4

Comment 3 Johnny Liu 2012-03-27 11:33:11 UTC
Verification is blocked by Bug 807236, so leave this bug until 807236 is fixed.

Comment 4 Johnny Liu 2012-03-29 03:20:30 UTC
Re-test this bug with jenkins-plugin-openshift-0.5.5-1.el6_2.x86_64, this issue still reproduced.

Comment 5 Bill DeCoste 2012-04-02 19:09:03 UTC
Updated fix in jenkins-plugin-openshift 0.5.6

Comment 6 Johnny Liu 2012-04-05 12:21:37 UTC
Verified this bug with devenv_1696, and PASS.


Note You need to log in before you can comment on or make changes to this bug.