Bug 802686

Summary:	Jenkins hang there forever when slave app can not be created due to no node available.
Product:	OKD	Reporter:	Johnny Liu <jialiu>
Component:	Containers	Assignee:	Bill DeCoste <wdecoste>
Status:	CLOSED CURRENTRELEASE	QA Contact:	libra bugs <libra-bugs>
Severity:	low	Docs Contact:
Priority:	high
Version:	1.x	CC:	dmcphers, mhicks, rmillner
Target Milestone:	---	Keywords:	Triaged
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2012-04-13 18:33:12 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:

Description Johnny Liu 2012-03-13 09:43:34 UTC

Description of problem:
jenkins will hang there for ever when fail to create slave app due to some reasons, e.g: slave app DNS fail to be resolved, no available node.


Version-Release number of selected component (if applicable):
jenkins-plugin-openshift-0.5.0-1.el6_2.x86_64

How reproducible:
Always

Steps to Reproduce:
1. Create a app with jenkins client enabled.
2. Change node profile to make "No nodes available" come true.
e.g:
Here I create a jboss app with jenkins client enabled, jboss slave app need find out a "medium" node to create slave app on it. But my node is "small".
3. Do some change to trigger jenkins build.
  
Actual results:
jenkins build will hang there for ever.

$ git commit -a -mx; git push
[master 349ad4e] x
 1 files changed, 1 insertions(+), 1 deletions(-)
Counting objects: 5, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 271 bytes, done.
Total 3 (delta 2), reused 0 (delta 0)
remote: Executing Jenkins build.
remote: 
remote: You can track your build at https://jenkins-jialiu.dev.rhcloud.com/job/jbosstest-build
remote: 
remote: Waiting for build to schedule.........................................................................

Expected results:
Any slave app creation failure can cause this issue, so "timeout" is
necessary to avoid this issue.

Additional info:

Comment 1 Johnny Liu 2012-03-13 09:45:37 UTC

Jenkins log:
com.openshift.express.client.OpenShiftEndpointException: Could not configure application "jbosstestbldr" at "https://domU-12-31-39-0B-01-21/broker/cartridge"
	at com.openshift.express.client.OpenShiftService.sendRequest(OpenShiftService.java:529)
	at com.openshift.express.client.OpenShiftService.requestApplicationAction(OpenShiftService.java:435)
	at com.openshift.express.client.OpenShiftService.createApplication(OpenShiftService.java:290)
	at hudson.plugins.openshift.OpenShiftSlave.createApp(OpenShiftSlave.java:225)
	at hudson.plugins.openshift.OpenShiftSlave.provision(OpenShiftSlave.java:194)
	at hudson.plugins.openshift.OpenShiftCloud$2.call(OpenShiftCloud.java:494)
	at hudson.plugins.openshift.OpenShiftCloud$2.call(OpenShiftCloud.java:488)
	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
	at java.util.concurrent.FutureTask.run(FutureTask.java:166)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
	at java.lang.Thread.run(Thread.java:679)
Caused by: com.openshift.express.internal.client.httpclient.InternalServerErrorException: {"broker_c":["namespace","rhlogin","ssh","app_uuid","debug","alter","cartridge","cart_type","action","app_name","api"],"api":"1.1.3","exit_code":1,"debug":"","result":"No nodes available.  If the problem persists please contact Red Hat support.","data":"","messages":"","api_c":["placeholder"]}
	at com.openshift.express.internal.client.httpclient.UrlConnectionHttpClient.createException(UrlConnectionHttpClient.java:107)
	at com.openshift.express.internal.client.httpclient.UrlConnectionHttpClient.post(UrlConnectionHttpClient.java:75)
	at com.openshift.express.client.OpenShiftService.sendRequest(OpenShiftService.java:518)
	... 11 more
Caused by: java.io.IOException: Server returned HTTP response code: 500 for URL: https://domU-12-31-39-0B-01-21/broker/cartridge
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1403)
	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254)
	at com.openshift.express.internal.client.httpclient.UrlConnectionHttpClient.post(UrlConnectionHttpClient.java:70)
	... 12 more

Comment 2 Bill DeCoste 2012-03-24 14:21:00 UTC

fixed in jenkins plugin 5.0.4

Comment 3 Johnny Liu 2012-03-27 11:33:11 UTC

Verification is blocked by Bug 807236, so leave this bug until 807236 is fixed.

Comment 4 Johnny Liu 2012-03-29 03:20:30 UTC

Re-test this bug with jenkins-plugin-openshift-0.5.5-1.el6_2.x86_64, this issue still reproduced.

Comment 5 Bill DeCoste 2012-04-02 19:09:03 UTC

Updated fix in jenkins-plugin-openshift 0.5.6

Comment 6 Johnny Liu 2012-04-05 12:21:37 UTC

Verified this bug with devenv_1696, and PASS.