Description of problem: after migration, try to push build for previous app with jenkins embedded, jenkins build will hang there. Version-Release number of selected component (if applicable): devenv_stage-144 How reproducible: always Steps to Reproduce: 1.start denenv_stage-143 2.create an app with jenkins embedded 3.do upgrade and migration 4.try to push changes for the exist app via jenkins. Actual results: jenkins build will hang there. Jenkins Log: INFO: Provisioning new node for workload = 1 and label = perl1-build com.openshift.express.client.OpenShiftEndpointException: Could not configure application "pyp1bldr" at "https://localhost/broker/cartridge" at com.openshift.express.client.OpenShiftService.sendRequest(OpenShiftService.java:529) at com.openshift.express.client.OpenShiftService.requestApplicationAction(OpenShiftService.java:435) at com.openshift.express.client.OpenShiftService.createApplication(OpenShiftService.java:290) at hudson.plugins.openshift.OpenShiftSlave.createApp(OpenShiftSlave.java:225) at hudson.plugins.openshift.OpenShiftSlave.provision(OpenShiftSlave.java:194) at hudson.plugins.openshift.OpenShiftCloud$2.call(OpenShiftCloud.java:494) at hudson.plugins.openshift.OpenShiftCloud$2.call(OpenShiftCloud.java:488) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:679) Caused by: com.openshift.express.internal.client.httpclient.BadRequestException: {"data":"","debug":"","api":"1.1.3","broker_c":["namespace","rhlogin","ssh","app_uuid","debug","alter","cartridge","cart_type","action","app_name","api"],"messages":"","api_c":["placeholder"],"result":"Invalid Profile: std. Must be: (jumbo|exlarge|large|micro|std)","exit_code":1} at com.openshift.express.internal.client.httpclient.UrlConnectionHttpClient.createException(UrlConnectionHttpClient.java:109) at com.openshift.express.internal.client.httpclient.UrlConnectionHttpClient.post(UrlConnectionHttpClient.java:75) at com.openshift.express.client.OpenShiftService.sendRequest(OpenShiftService.java:518) Expected results: push build should successful. Additional info: during the migration period, jenkins app will got 'Node execution failure' first time, and will pass the migration after another try. <---------> Migrating app 'jenkins' with uuid '54076892d9844998ae57312ef99d6da4' on node 'ip-10-64-29-128' for user: bmeng Failed to migrate with cmd: './migrate-2.0.7 --rhlogin 'bmeng' --migrate-app 'jenkins'' after 2 tries with exception: Node execution failure (invalid exit code from node). If the problem persists please contact Red Hat support. /var/www/stickshift/broker/lib/express/broker/application_container_proxy.rb:894:in `run_cartridge_command'./migrate-2.0.7:159:in `send'./migrate-2.0.7:159:in `redeploy_httpd_proxy'./migrate-2.0.7:49:in `migrate_app'/usr/lib/ruby/site_ruby/1.8/mcollective/rpc/client.rb:258:in `call'/usr/lib/ruby/site_ruby/1.8/mcollective/rpc/client.rb:258:in `custom_request'/usr/lib/ruby/site_ruby/1.8/mcollective/rpc/client.rb:511:in `call'/usr/lib/ruby/site_ruby/1.8/mcollective/rpc/client.rb:511:in `process_results_with_block'/usr/lib/ruby/site_ruby/1.8/mcollective/rpc/client.rb:450:in `call_agent'/usr/lib/ruby/site_ruby/1.8/mcollective/client.rb:129:in `req'/usr/lib/ruby/site_ruby/1.8/mcollective/client.rb:124:in `loop'/usr/lib/ruby/site_ruby/1.8/mcollective/client.rb:124:in `req'/usr/lib/ruby/1.8/timeout.rb:67:in `timeout'/usr/lib/ruby/site_ruby/1.8/mcollective/client.rb:123:in `req'/usr/lib/ruby/site_ruby/1.8/mcollective/rpc/client.rb:446:in `call_agent'/usr/lib/ruby/site_ruby/1.8/mcollective/rpc/client.rb:257:in `custom_request'/usr/lib/ruby/site_ruby/1.8/mcollective/rpc/client.rb:193:in `method_missing'./migrate-2.0.7:34:in `migrate_app'/var/www/stickshift/broker/lib/express/broker/application_container_proxy.rb:611:in `rpc_exec'./migrate-2.0.7:33:in `migrate_app'/usr/lib/ruby/1.8/timeout.rb:67:in `timeout'./migrate-2.0.7:32:in `migrate_app'./migrate-2.0.7:463:in `migrate_from_file'./migrate-2.0.7:461:in `each'./migrate-2.0.7:461:in `migrate_from_file'./migrate-2.0.7:573 Output: Migrating app on node with: ./migrate-2.0.7 --rhlogin 'bmeng' --migrate-app 'jenkins' <---------> !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Failures: ./migrate-2.0.7 --rhlogin 'bmeng' --migrate-app 'jenkins' ./migrate-2.0.7 --rhlogin 'bmeng+1' --migrate-app 'jk' !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! <---------> [root@ip-10-64-29-128 bin]# ./migrate-2.0.7 --rhlogin 'bmeng' --migrate-app 'jenkins' Migrating app on node with: ./migrate-2.0.7 --rhlogin 'bmeng' --migrate-app 'jenkins' Redeploying httpd proxy for 'jenkins' with uuid '54076892d9844998ae57312ef99d6da4' on node ip-10-64-29-128 Leaving app stopped 'jenkins' with uuid '54076892d9844998ae57312ef99d6da4' on node 'ip-10-64-29-128' [root@ip-10-64-29-128 bin]# ./migrate-2.0.7 --rhlogin 'bmeng+1' --migrate-app 'jk' Migrating app on node with: ./migrate-2.0.7 --rhlogin 'bmeng+1' --migrate-app 'jk' Redeploying httpd proxy for 'jk' with uuid '30c1e14c15cd44ea9f134b82b2e761ae' on node ip-10-64-29-128 Leaving app stopped 'jk' with uuid '30c1e14c15cd44ea9f134b82b2e761ae' on node 'ip-10-64-29-128'
Rob, can you take a look. Looks like there are 2 issues. 1 the builder sizes aren't being migrated. And 2 there is still an issue with redeploy httpd proxy after the folder renames.
Checkins 5ff56e99 and 5b8651e5 update builder sizes in the jenkins and build configurations respectively.
Correction, 5b8651e5 just updates a comment and is safe to either include or exclude from the release.
The following checkins address the (and related) issue: 5b8651e5 cleanup error message. 5ff56e993 Update jenkins config.xml and per-job config.xml to reflect new gear sizes. 252b2465 Dont need verbose output from the file edit. fd3ad80bd Had missed the existing jenkins block 2aa3651 missing variable migration 2a1570e Add APP_NAME to the list of retained variables. e87d1af PERL has an environment variable screen ... Migration works. Jenkins builds do not get stuck. The first time after migration Jenkins builds may fail. After a few tries re-running the slave is rebuilt and they run properly. Bug 804276 was created to track that issue.
Verified this bug with devenv_stage_146 updated from devenv_stage_143, and PASS