Bug 794544 - jenkins build for existing wsgi/rack app hang there upon git commit
Summary: jenkins build for existing wsgi/rack app hang there upon git commit
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: OKD
Classification: Red Hat
Component: Pod
Version: 1.x
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: ---
Assignee: Thomas Wiest
QA Contact: libra bugs
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2012-02-17 04:38 UTC by Johnny Liu
Modified: 2012-03-02 21:56 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2012-03-02 21:56:39 UTC
Target Upstream Version:


Attachments (Terms of Use)

Description Johnny Liu 2012-02-17 04:38:16 UTC
Description of problem: 
Make sure you have an existing wsgi/rack app with jenkins client embedded, that was created a long time ago (at least before sprint 4 release). In that time, its type was wsgi-3.2/rack-1.1.
After sprint 4 release, cartridge name was changed to python-2.6/ruby-1.8, and  app type was indeed changed seen from the output of rhc-admin-info, but the jenkins build xml was NOT updated, builderType is still "wsgi-3.2/rack-1.1" as following:
(Log into jenkins server app)
$ cat jenkins/data/jobs/wsgitest-build/config.xml
<--snip-->
  <properties>
    <hudson.plugins.openshift.OpenShiftBuilderSizeJobProperty>
      <builderSize>std</builderSize>
    </hudson.plugins.openshift.OpenShiftBuilderSizeJobProperty>
    <hudson.plugins.openshift.OpenShiftBuilderTypeJobProperty>
      <builderType>wsgi-3.2</builderType>
    </hudson.plugins.openshift.OpenShiftBuilderTypeJobProperty>
  </properties>
<--snip-->

So when you do git commit, trigger jenkins build, it will hang there for ever, because broker only support "python-2.6", not "wsgi-3.2", the following is the jenkins log:
<--snip-->
INFO: Started provisioning wsgitest-build from OpenShift Cloud with 1 executors. Remaining excess workload:-0.8297812
com.openshift.express.client.OpenShiftEndpointException: Could not configure application "wsgitestbldr" at "https://stg.openshift.redhat.com/broker/cartridge"
	at com.openshift.express.client.OpenShiftService.sendRequest(OpenShiftService.java:370)
	at com.openshift.express.client.OpenShiftService.requestApplicationAction(OpenShiftService.java:299)
	at com.openshift.express.client.OpenShiftService.createApplication(OpenShiftService.java:180)
	at hudson.plugins.openshift.OpenShiftSlave.createApp(OpenShiftSlave.java:216)
	at hudson.plugins.openshift.OpenShiftSlave.provision(OpenShiftSlave.java:185)
	at hudson.plugins.openshift.OpenShiftCloud$2.call(OpenShiftCloud.java:463)
	at hudson.plugins.openshift.OpenShiftCloud$2.call(OpenShiftCloud.java:457)
	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
	at java.util.concurrent.FutureTask.run(FutureTask.java:166)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
	at java.lang.Thread.run(Thread.java:679)
Caused by: com.openshift.express.internal.client.httpclient.BadRequestException: {"broker_c":["namespace","rhlogin","ssh","app_uuid","debug","alter","cartridge","cart_type","action","app_name","api"],"messages":"","result":"Invalid application type (-t|--type) specified: ''.  Valid application types are (raw-0.1, php-5.3, jbossas-7.0, jenkins-1.4, python-2.6, perl-5.10, ruby-1.8).","debug":"","data":"","exit_code":1,"api":"1.1.2","api_c":["placeholder"]}
	at com.openshift.express.internal.client.httpclient.UrlConnectionHttpClient.createException(UrlConnectionHttpClient.java:104)
	at com.openshift.express.internal.client.httpclient.UrlConnectionHttpClient.post(UrlConnectionHttpClient.java:71)
	at com.openshift.express.client.OpenShiftService.sendRequest(OpenShiftService.java:361)
	... 11 more
Caused by: java.io.IOException: Server returned HTTP response code: 400 for URL: https://stg.openshift.redhat.com/broker/cartridge
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1403)
	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254)
	at com.openshift.express.internal.client.httpclient.UrlConnectionHttpClient.post(UrlConnectionHttpClient.java:66)
	... 12 more
<--snip-->


Work around:
1. update jenkins/data/jobs/wsgitest-build/config.xml, change "wsgi-3.2" to "python-2.6"
2. update ./data/plugins/openshift/WEB-INF/classes/hudson/plugins/openshift/OpenShiftBuilderTypeJobProperty/config.jelly, change "wsgi-3.2" to "python-2.6".
3. Restart jenkins server app.
Now jenkins build is finished successfully


Actually this bug should be found in the testing for sprint 4 release, but miss it, sorry for being late to find this issue till now.

The above testing is against stage env.


Version-Release number of selected component (if applicable):


How reproducible:
Always

Steps to Reproduce:
1. 
2.
3.
  
Actual results:


Expected results:


Additional info:

Comment 1 Rob Millner 2012-02-17 16:10:02 UTC
Does this affect apps in production or were they fixed as part of the prod update at the end of sprint 4?

Comment 2 Bill DeCoste 2012-02-17 18:04:26 UTC
jenkins plugin build 0.4.19 should resolve

Comment 3 Bill DeCoste 2012-02-17 18:05:15 UTC
This was probably a bug in the last sprint when we switched the cartridge names

Comment 4 Johnny Liu 2012-02-18 09:54:41 UTC
(In reply to comment #1)
> Does this affect apps in production or were they fixed as part of the prod
> update at the end of sprint 4?

Unfortunately I does not have wsgi/rack app with embedded jenkins on production, But I am almost sure this also affects apps in production.

Just as what Bill said, this indeed a bug in the last print (print 4), but QE did not find it in sprint 4 test round.

Comment 5 Johnny Liu 2012-02-18 10:02:24 UTC
(In reply to comment #2)
> jenkins plugin build 0.4.19 should resolve

I guess this only fix the issue in config.jelly, but for the existing wsgi/rack app, this issue still occurred because already existing build's config files did not be updated, they are still using "wsgi-3.2/rack-1.1".

I think we need a migration script update to fix this issue, right?

CC Dan for my suggestion.

Comment 6 Johnny Liu 2012-02-18 10:06:30 UTC
I think after migrate script is updated, involve administrators of production and stage to migrate that existing wsgi/rack app with embedded jenkins to fix this issue.

Comment 7 Bill DeCoste 2012-02-18 14:27:51 UTC
With the recent change to the jenkins plugin, we will now handle all 4 cartridge types. There is no update necessary for existing wsgi/rack applications. The jenkins plugins will convert them to the newer types. The problem is easy to recreate given the steps above without an older application.

Comment 8 Dan McPherson 2012-02-19 17:39:29 UTC
The plan was to do the migration manually in production.

Comment 9 Johnny Liu 2012-02-20 07:58:18 UTC
Re-test this bug with jenkins-plugin-openshift-0.4.19-1.el6_2.x86_64 on devenv_1621.

Modified "builderType" to "rack-1.1/wsgi-3.2" on purpose in jenkins/data/jobs/<appname>-build/config.xml, then do git push to trigger jenkins build, it works fine.

But I found a minor issue, access http://jenkins-<domain>.dev.rhcloud.com/job/<app_name>-build/configure, check "Builder Type" drop-down list, find that jenkins is still using old cartridge name, and data/plugins/openshift/WEB-INF/classes/hudson/plugins/openshift/OpenShiftBuilderTypeJobProperty/config.jelly does not change to show new cartridge name.


So assign back this bug, once data/plugins/openshift/WEB-INF/classes/hudson/plugins/openshift/OpenShiftBuilderTypeJobProperty/config.jelly is updated, I will re-test this bug.

Comment 10 Johnny Liu 2012-02-20 12:19:45 UTC
Because Dan will do the migration manually in production, low priority to "medium"

Comment 11 Dan McPherson 2012-02-23 14:39:14 UTC
Hi Thomas,

  Was this migration completed?

Comment 12 Thomas Wiest 2012-02-23 15:54:46 UTC
I have done this migration in STG and PROD.

Comment 13 Johnny Liu 2012-02-27 05:26:52 UTC
Verified this bug with devenv_1642, and PASS.
data/plugins/openshift/WEB-INF/classes/hudson/plugins/openshift/OpenShiftBuilderTypeJobProperty/config.jelly is updated and jenkins build is completed successfully.


Note You need to log in before you can comment on or make changes to this bug.