Red Hat Bugzilla – Bug 865282
[US2513][fork]Node execution failure when doing rhc threaddump for jbossEWS app
Last modified: 2015-05-14 19:00:41 EDT
Description of problem: Node execution failure when doing rhc threaddump for jbossews app as below: # rhc threaddump app4 Password: * Node execution failure (invalid exit code from node). If the problem persists please contact Red Hat support. Version-Release number of selected component (if applicable): rhc client built from fork_ami_US2597_US2599_US2813_US2817_US2872_US2816_US2814_226 fork_ami_US2513-6_224 How reproducible: always Steps to Reproduce: 1.Create an jbossews app #rhc app create app jbossews-1.0 2.Do threaddump for this app #rhc threaddump app Actual results: It's failed with node execution failure error. Expected results: The threaddump file should be available. Additional info: It can work well with jbosseap-6.0 app.
I can't recreate this from AMI US2513-6. Please see my logs below. Can you attach the errors you are seeing? [root@ip-10-100-253-248 ~]# rhc app create -l bdecoste42@gmail.com -p 123456 -a ews1 -t jbossews-1.0 -d Submitting form: rhlogin: bdecoste42@gmail.com debug: true Contacting https://localhost Creating application: ews1 in bdecoste42b Contacting https://localhost Response from server: DEBUG: Exit Code: 0 api_c: placeholder broker_c: namespacerhloginsshapp_uuiddebugaltercartridgecart_typeactionapp_nameapi API version: 1.1.3 DEBUG: 'ews1' creation returned success. Now your new domain name is being propagated worldwide (this might take a minute)... Pulling new repo down git clone ssh://a3fea95f3d76415eaadadd14b11de6e6@ews1-bdecoste42b.dev.rhcloud.com/~/git/ews1.git/ ews1 The authenticity of host 'ews1-bdecoste42b.dev.rhcloud.com (10.100.253.248)' can't be established. RSA key fingerprint is 23:9d:5c:a9:83:bb:9e:fc:f6:4c:0b:c1:0e:e8:bc:7a. Are you sure you want to continue connecting (yes/no)? yes Checking if the application is available #1 Application ews1 is available at: http://ews1-bdecoste42b.dev.rhcloud.com/ Git URL: ssh://a3fea95f3d76415eaadadd14b11de6e6@ews1-bdecoste42b.dev.rhcloud.com/~/git/ews1.git/ To make changes to 'ews1', commit to ews1/. Successfully created application: ews1 [root@ip-10-100-253-248 ~]# rhc app tail -l bdecoste42@gmail.com -p 123456 -a ews1 -d Submitting form: debug: true rhlogin: bdecoste42@gmail.com Contacting https://localhost Attempting to tail files: */log*/* Use ctl + c to stop ssh -t a3fea95f3d76415eaadadd14b11de6e6@ews1-bdecoste42b.dev.rhcloud.com 'tail */log*/*' ==> jbossews-1.0/logs/localhost.2012-10-11.log <== ==> jbossews-1.0/logs/manager.2012-10-11.log <== ==> jbossews-1.0/logs/catalina.out <== Oct 11, 2012 1:29:37 PM org.apache.catalina.core.StandardService start INFO: Starting service Catalina Oct 11, 2012 1:29:37 PM org.apache.catalina.core.StandardEngine start INFO: Starting Servlet Engine: Apache Tomcat/6.0.32 Oct 11, 2012 1:29:37 PM org.apache.catalina.startup.HostConfig deployWAR INFO: Deploying web application archive ROOT.war Oct 11, 2012 1:29:38 PM org.apache.coyote.http11.Http11Protocol start INFO: Starting Coyote HTTP/1.1 on http-127.0.250.129-8080 Oct 11, 2012 1:29:38 PM org.apache.catalina.startup.Catalina start INFO: Server startup in 1011 ms ==> jbossews-1.0/logs/catalina.2012-10-11.log <== Oct 11, 2012 1:29:37 PM org.apache.catalina.core.StandardService start INFO: Starting service Catalina Oct 11, 2012 1:29:37 PM org.apache.catalina.core.StandardEngine start INFO: Starting Servlet Engine: Apache Tomcat/6.0.32 Oct 11, 2012 1:29:37 PM org.apache.catalina.startup.HostConfig deployWAR INFO: Deploying web application archive ROOT.war Oct 11, 2012 1:29:38 PM org.apache.coyote.http11.Http11Protocol start INFO: Starting Coyote HTTP/1.1 on http-127.0.250.129-8080 Oct 11, 2012 1:29:38 PM org.apache.catalina.startup.Catalina start INFO: Server startup in 1011 ms ==> jbossews-1.0/logs/host-manager.2012-10-11.log <==
Created attachment 625790 [details] development log from boker
This problem is caused by rhc threaddump, it works fine with rhc tail separately. Please refer to debug infor here from command line: # rhc threaddump app1 -d Password: * D, [2012-10-12T10:53:58.865557 #2332] DEBUG -- : Connecting to https://ec2-54-242-155-252.compute-1.amazonaws.com/broker/rest/api D, [2012-10-12T10:54:00.056587 #2332] DEBUG -- : Getting all domains D, [2012-10-12T10:54:00.056827 #2332] DEBUG -- : Request: #<RestClient::Request:0x0000000236b788 @method="GET", @headers={:accept=>:json, "Authorization"=>"Basic amluemhhbmcrMUByZWRoYXQuY29tOmE=", "User-Agent"=>"rhc/0.99.5 (ruby 1.9.3; x86_64-linux)"}, @url="https://ec2-54-242-155-252.compute-1.amazonaws.com/broker/rest/domains", @cookies={}, @payload="", @user=nil, @password=nil, @timeout=nil, @open_timeout=nil, @block_response=nil, @raw_response=false, @verify_ssl=false, @ssl_client_cert=nil, @ssl_client_key=nil, @ssl_ca_file=nil, @tf=nil, @max_redirects=10, @processed_headers={"Accept"=>"application/json", "Accept-Encoding"=>"gzip, deflate", "Authorization"=>"Basic amluemhhbmcrMUByZWRoYXQuY29tOmE=", "User-Agent"=>"rhc/0.99.5 (ruby 1.9.3; x86_64-linux)", "Content-Length"=>"0", "Content-Type"=>"application/x-www-form-urlencoded"}, @args={:url=>"https://ec2-54-242-155-252.compute-1.amazonaws.com/broker/rest/domains", :method=>"GET", :headers=>{:accept=>:json, "Authorization"=>"Basic amluemhhbmcrMUByZWRoYXQuY29tOmE=", "User-Agent"=>"rhc/0.99.5 (ruby 1.9.3; x86_64-linux)"}, :payload=>{}, :timeout=>nil}> D, [2012-10-12T10:54:01.299825 #2332] DEBUG -- : Finding domain joycedev3 D, [2012-10-12T10:54:01.299906 #2332] DEBUG -- : Getting all domains D, [2012-10-12T10:54:01.300083 #2332] DEBUG -- : Request: #<RestClient::Request:0x00000001849828 @method="GET", @headers={:accept=>:json, "Authorization"=>"Basic amluemhhbmcrMUByZWRoYXQuY29tOmE=", "User-Agent"=>"rhc/0.99.5 (ruby 1.9.3; x86_64-linux)"}, @url="https://ec2-54-242-155-252.compute-1.amazonaws.com/broker/rest/domains", @cookies={}, @payload="", @user=nil, @password=nil, @timeout=nil, @open_timeout=nil, @block_response=nil, @raw_response=false, @verify_ssl=false, @ssl_client_cert=nil, @ssl_client_key=nil, @ssl_ca_file=nil, @tf=nil, @max_redirects=10, @processed_headers={"Accept"=>"application/json", "Accept-Encoding"=>"gzip, deflate", "Authorization"=>"Basic amluemhhbmcrMUByZWRoYXQuY29tOmE=", "User-Agent"=>"rhc/0.99.5 (ruby 1.9.3; x86_64-linux)", "Content-Length"=>"0", "Content-Type"=>"application/x-www-form-urlencoded"}, @args={:url=>"https://ec2-54-242-155-252.compute-1.amazonaws.com/broker/rest/domains", :method=>"GET", :headers=>{:accept=>:json, "Authorization"=>"Basic amluemhhbmcrMUByZWRoYXQuY29tOmE=", "User-Agent"=>"rhc/0.99.5 (ruby 1.9.3; x86_64-linux)"}, :payload=>{}, :timeout=>nil}> D, [2012-10-12T10:54:02.637625 #2332] DEBUG -- : Finding application :name => app1, :framework => D, [2012-10-12T10:54:02.637717 #2332] DEBUG -- : Getting all applications for domain joycedev3 D, [2012-10-12T10:54:02.637883 #2332] DEBUG -- : Request: #<RestClient::Request:0x000000022b48d0 @method="GET", @headers={:accept=>:json, "Authorization"=>"Basic amluemhhbmcrMUByZWRoYXQuY29tOmE=", "User-Agent"=>"rhc/0.99.5 (ruby 1.9.3; x86_64-linux)"}, @url="https://ec2-54-242-155-252.compute-1.amazonaws.com/broker/rest/domains/joycedev3/applications", @cookies={}, @payload="", @user=nil, @password=nil, @timeout=nil, @open_timeout=nil, @block_response=nil, @raw_response=false, @verify_ssl=false, @ssl_client_cert=nil, @ssl_client_key=nil, @ssl_ca_file=nil, @tf=nil, @max_redirects=10, @processed_headers={"Accept"=>"application/json", "Accept-Encoding"=>"gzip, deflate", "Authorization"=>"Basic amluemhhbmcrMUByZWRoYXQuY29tOmE=", "User-Agent"=>"rhc/0.99.5 (ruby 1.9.3; x86_64-linux)", "Content-Length"=>"0", "Content-Type"=>"application/x-www-form-urlencoded"}, @args={:url=>"https://ec2-54-242-155-252.compute-1.amazonaws.com/broker/rest/domains/joycedev3/applications", :method=>"GET", :headers=>{:accept=>:json, "Authorization"=>"Basic amluemhhbmcrMUByZWRoYXQuY29tOmE=", "User-Agent"=>"rhc/0.99.5 (ruby 1.9.3; x86_64-linux)"}, :payload=>{}, :timeout=>nil}> D, [2012-10-12T10:54:04.010336 #2332] DEBUG -- : Running thread dump for app1 D, [2012-10-12T10:54:04.010613 #2332] DEBUG -- : Request: #<RestClient::Request:0x00000002332848 @method="POST", @headers={:accept=>:json, "Authorization"=>"Basic amluemhhbmcrMUByZWRoYXQuY29tOmE=", "User-Agent"=>"rhc/0.99.5 (ruby 1.9.3; x86_64-linux)"}, @url="https://ec2-54-242-155-252.compute-1.amazonaws.com/broker/rest/domains/joycedev3/applications/app1/events", @cookies={}, @payload="event=thread-dump", @user=nil, @password=nil, @timeout=nil, @open_timeout=nil, @block_response=nil, @raw_response=false, @verify_ssl=false, @ssl_client_cert=nil, @ssl_client_key=nil, @ssl_ca_file=nil, @tf=nil, @max_redirects=10, @processed_headers={"Accept"=>"application/json", "Accept-Encoding"=>"gzip, deflate", "Authorization"=>"Basic amluemhhbmcrMUByZWRoYXQuY29tOmE=", "User-Agent"=>"rhc/0.99.5 (ruby 1.9.3; x86_64-linux)", "Content-Length"=>"17", "Content-Type"=>"application/x-www-form-urlencoded"}, @args={:url=>"https://ec2-54-242-155-252.compute-1.amazonaws.com/broker/rest/domains/joycedev3/applications/app1/events", :method=>"POST", :headers=>{:accept=>:json, "Authorization"=>"Basic amluemhhbmcrMUByZWRoYXQuY29tOmE=", "User-Agent"=>"rhc/0.99.5 (ruby 1.9.3; x86_64-linux)"}, :payload=>{:event=>"thread-dump"}, :timeout=>nil}> Node execution failure (invalid exit code from node). If the problem persists please contact Red Hat support.
Sorry about that, misread the bug. threaddump is now fixed.
This bug has been fixed after verifying on devenv_2327. The threaddump file is available while using rhc threaddump as below: # rhc threaddump app2 Password: * Success The thread dump file will be available via: rhc app tail -a app2 -f jbossews-1.0/jbossews-1.0/logs/catalina.out -o '-n 250'