Bug 813229

Summary: Return "Node execution failure" while reloading an jenkins server
Product: OKD Reporter: bzhao
Component: ContainersAssignee: Abhishek Gupta <abhgupta>
Status: CLOSED CURRENTRELEASE QA Contact: libra bugs <libra-bugs>
Severity: medium Docs Contact:
Priority: medium    
Version: 2.xCC: bmeng, pruan, rmillner
Target Milestone: ---Keywords: Triaged
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: cartridge-jenkins-1.4-0.91.5-1+ Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-04-27 20:46:07 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Attachments:
Description Flags
development_log
none
jenkins_log none

Description bzhao 2012-04-17 09:29:55 UTC
Description of problem:
It will return "Node execution failure (invalid exit code from node).  If the problem persists please contact Red Hat support." while reloading an jenkins server

Version-Release number of selected component (if applicable):
rhc-0.91.1-1.git.9.713b065.el6.noarch
jenkins-plugin-openshift-0.5.13-0.el6_2.x86_64
jenkins-1.457-1.2.noarch
cartridge-jenkins-1.4-0.91.2-1.git.135.9f29adc.el6.noarch
cartridge-jenkins-client-1.4-0.25.2-1.git.0.7c72729.el6.noarch


How reproducible:
Always

Steps to Reproduce:
1.Create a jenkins server
#rhc-create-app -t jenkins-1.4 -p xx -a jenkinstest
2.Reload the jenkins server
# rhc-ctl-app  -a jenkinstest -p xx -e reload-jenkins-client-1.4
3.
  
Actual results:
Return "Could not reload Jenkins server 'jenkinstest' configuration:
   000
Cartridge return code: 1"
Node execution failure (invalid exit code from node).  If the problem persists please contact Red Hat support.

Expected results:
Reload the jenkins server successfully without any error

Additional info:
Part of development.log==>
Exception rescued in /broker/cartridge:
Node execution failure (invalid exit code from node).  If the problem persists please contact Red Hat support.
/usr/lib/ruby/gems/1.8/gems/gearchanger-m-collective-plugin-0.8.2/lib/gearchanger-m-collective-plugin/gearchanger/mcollective_application_container_proxy.rb:939:in `run_cartridge_command'
/usr/lib/ruby/gems/1.8/gems/gearchanger-m-collective-plugin-0.8.2/lib/gearchanger-m-collective-plugin/gearchanger/mcollective_application_container_proxy.rb:280:in `reload'
/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.9.2/lib/stickshift-controller/app/models/gear.rb:96:in `reload'
/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.9.2/lib/stickshift-controller/app/models/application.rb:678:in `reload'
/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.9.2/lib/stickshift-controller/app/models/application.rb:1378:in `call'
/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.9.2/lib/stickshift-controller/app/models/application.rb:1378:in `run_on_gears'
--DEBUG--
Could not reload Jenkins server 'jenkinstest' configuration:
   000
Cartridge return code: 1

Comment 1 bzhao 2012-04-17 09:30:53 UTC
Created attachment 577966 [details]
development_log

Comment 2 bzhao 2012-04-17 09:31:37 UTC
Created attachment 577967 [details]
jenkins_log

Comment 3 Rob Millner 2012-04-18 02:10:13 UTC
Fixed in commit d793aeb.

[rmillner@splat li]$ rhc app reload -p none -a jenkinstest -dContacting https://107.20.73.47
Response from server:
DEBUG:


Exit Code: 0
api_c: placeholder
broker_c: namespacerhloginsshapp_uuiddebugaltercartridgecart_typeactionapp_nameapi
API version:    1.1.3

RESULT:
Success

Comment 4 bzhao 2012-04-19 07:51:42 UTC
Passed verification against devenv 1730

Comment 5 Meng Bo 2012-04-24 07:19:27 UTC
reproduce this issue on devenv_1738 with cartridge-jenkins-1.4-0.91.6-1, reopen this bug.

<-------client log------->DEBUG:
Could not reload Jenkins server 'jenkins' configuration:
   /usr/libexec/stickshift/cartridges/jenkins-1.4/info/bin/app_ctl.sh: line 101: jenkins_reload: command not found
Cartridge return code: 1

Exit Code: 1
api_c: placeholder
broker_c: namespacerhloginsshapp_uuiddebugaltercartridgecart_typeactionapp_nameapi
API version:    1.1.3

RESULT:
Node execution failure (invalid exit code from node).  If the problem persists please contact Red Hat support.



<-------broker log------->
DEBUG: rpc_exec_direct: rpc_client=#<MCollective::RPC::Client:0x7f115b637158>
DEBUG: rpc_client.custom_request('cartridge_do', {:action=>"reload", :args=>"'jenkins' 'bmengauto1' '86c377dcfb864ac5a8e89ef2174d3ca3'", :cartridge=>"jenkins-1.4"}, @id, {'identity' => @id})
DEBUG: [#<MCollective::RPC::Result:0x7f115b6572f0 @agent="libra", @results={:sender=>"ip-10-117-58-249", :statusmsg=>"cartridge_do_action failed 1.  Output Could not reload Jenkins server 'jenkins' configuration:\n   /usr/libexec/stickshift/cartridges/jenkins-1.4/info/bin/app_ctl.sh: line 101: jenkins_reload: command not found\n", :statuscode=>1, :data=>{:exitcode=>1, :output=>"Could not reload Jenkins server 'jenkins' configuration:\n   /usr/libexec/stickshift/cartridges/jenkins-1.4/info/bin/app_ctl.sh: line 101: jenkins_reload: command not found\n"}}, @action="cartridge_do">]
DEBUG: server results: Could not reload Jenkins server 'jenkins' configuration:
DEBUG: server results:    /usr/libexec/stickshift/cartridges/jenkins-1.4/info/bin/app_ctl.sh: line 101: jenkins_reload: command not found
Node execution failure (invalid exit code from node).  If the problem persists please contact Red Hat support.
#<StickShift::NodeException: Node execution failure (invalid exit code from node).  If the problem persists please contact Red Hat support.>
.
.
.

Exception rescued in /broker/cartridge:
Node execution failure (invalid exit code from node).  If the problem persists please contact Red Hat support.
/usr/lib/ruby/gems/1.8/gems/gearchanger-m-collective-plugin-0.8.7/lib/gearchanger-m-collective-plugin/gearchanger/mcollective_application_container_proxy.rb:1135:in `run_cartridge_command'
/usr/lib/ruby/gems/1.8/gems/gearchanger-m-collective-plugin-0.8.7/lib/gearchanger-m-collective-plugin/gearchanger/mcollective_application_container_proxy.rb:280:in `reload'
/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.9.10/lib/stickshift-controller/app/models/gear.rb:96:in `reload'
/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.9.10/lib/stickshift-controller/app/models/application.rb:644:in `reload'
/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.9.10/lib/stickshift-controller/app/models/application.rb:1340:in `call'
/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.9.10/lib/stickshift-controller/app/models/application.rb:1340:in `run_on_gears'
--DEBUG--
Could not reload Jenkins server 'jenkins' configuration:
   /usr/libexec/stickshift/cartridges/jenkins-1.4/info/bin/app_ctl.sh: line 101: jenkins_reload: command not found
Cartridge return code: 1

Comment 6 Rob Millner 2012-04-24 18:39:40 UTC
li commit 318d8c83 removed the jenkins_reload command and its still referenced by app_ctl.sh.

$ grep jenkins_reload cartridges/jenkins-1.4/info/bin/app_ctl.sh
            if ! out=$(jenkins_reload "${JENKINS_USERNAME}" "${JENKINS_PASSWORD}" "${JENKINS_URL}" 2>&1)

Passing to the commiter to figure out what they want to do.

Comment 7 Abhishek Gupta 2012-04-24 23:23:34 UTC
There are a couple problems with the way we were doing it. 

#1 that URL failed most of the time and 
#2 when it succeeds, it reloads EVEYTHING and kills any active build

The reload now happens automatically through the jenkins plugin before every build.

Comment 8 Peter Ruan 2012-04-25 19:09:01 UTC
verified with ami-16964c7f