Description of problem: rhc tail throws Net::SSH::AuthenticationFailed error Version-Release number of selected component (if applicable): rubygem-openshift-origin-msg-broker-mcollective-1.10.1-1.git.52.0e2b1c0 rhc-broker-1.10.1-1.git.300.b5cfc17 openshift-origin-broker-util-1.10.1-1.git.118.fedbc2c How reproducible: every time Steps to Reproduce: 1. spin up a new devenv EC2 instance 2. do rhc setup and create an application named "myapp" 3. do rhc tail myapp Actual results: [ofayans@dhcp129-148 ostest]$ rhc tail njkzq4csrj -d DEBUG: Authenticating with RHC::Auth::Token DEBUG: Connecting to https://ec2-23-22-248-158.compute-1.amazonaws.com/broker/rest/api DEBUG: Getting all domains DEBUG: Request GET https://ec2-23-22-248-158.compute-1.amazonaws.com/broker/rest/api DEBUG: SSL Verification failed -- Using self signed cert DEBUG: code 200 599 ms DEBUG: Request GET https://ec2-23-22-248-158.compute-1.amazonaws.com/broker/rest/api DEBUG: SSL Verification failed -- Using self signed cert DEBUG: code 200 593 ms DEBUG: Request GET https://ec2-23-22-248-158.compute-1.amazonaws.com/broker/rest/domains DEBUG: SSL Verification failed -- Using self signed cert DEBUG: code 200 927 ms DEBUG: Getting all domains DEBUG: Request GET https://ec2-23-22-248-158.compute-1.amazonaws.com/broker/rest/domains/ofayans/applications/njkzq4csrj?include=cartridges DEBUG: code 200 166 ms DEBUG: Tail in progress for cartridge * DEBUG: ssh -t 51b1ade81cb9aa0a83000002.rhcloud.com 'tail */log*/*' DEBUG: Opening Net::SSH connection to njkzq4csrj-ofayans.dev.rhcloud.com, 51b1ade81cb9aa0a83000002, tail */log*/* You can tail this application directly with: ssh -t 51b1ade81cb9aa0a83000002.rhcloud.com 'tail */log*/*' /usr/local/share/gems/gems/net-ssh-2.6.6/lib/net/ssh.rb:201:in `start': 51b1ade81cb9aa0a83000002 (Net::SSH::AuthenticationFailed) from /home/ofayans/.gem/ruby/1.9.1/gems/rhc-1.9.1/lib/rhc/ssh_helpers.rb:39:in `ssh_ruby' from /home/ofayans/.gem/ruby/1.9.1/gems/rhc-1.9.1/lib/rhc/commands/tail.rb:41:in `tail' from /home/ofayans/.gem/ruby/1.9.1/gems/rhc-1.9.1/lib/rhc/commands/tail.rb:22:in `run' from /home/ofayans/.gem/ruby/1.9.1/gems/rhc-1.9.1/lib/rhc/commands.rb:240:in `execute' from /home/ofayans/.gem/ruby/1.9.1/gems/rhc-1.9.1/lib/rhc/commands.rb:231:in `block (3 levels) in to_commander' from /usr/local/share/gems/gems/commander-4.1.3/lib/commander/command.rb:180:in `call' from /usr/local/share/gems/gems/commander-4.1.3/lib/commander/command.rb:180:in `call' from /usr/local/share/gems/gems/commander-4.1.3/lib/commander/command.rb:155:in `run' from /usr/local/share/gems/gems/commander-4.1.3/lib/commander/runner.rb:402:in `run_active_command' from /home/ofayans/.gem/ruby/1.9.1/gems/rhc-1.9.1/lib/rhc/command_runner.rb:70:in `run!' from /usr/local/share/gems/gems/commander-4.1.3/lib/commander/delegates.rb:11:in `run!' from /home/ofayans/.gem/ruby/1.9.1/gems/rhc-1.9.1/lib/rhc/cli.rb:36:in `start' from /home/ofayans/.gem/ruby/1.9.1/gems/rhc-1.9.1/bin/rhc:18:in `<top (required)>' from /usr/local/bin/rhc:23:in `load' from /usr/local/bin/rhc:23:in `<main>' Expected results: log tail Additional info: when I manually do ssh -t 51b1ade81cb9aa0a83000002.rhcloud.com 'tail */log*/*' it works fine for me
This issue is not reproduced on devenv_3335, please refer to the following results: 1. spin up a new devenv_3335 EC2 instance 2. create an application named "cphp0s0" rhc app create cphp0s0 php-5.3 -s --no-git 3. do rhc tail cphp0s0 [root test]# rhc tail cphp0s0 -d DEBUG: Using config file /root/.openshift/express.conf DEBUG: Authenticating with RHC::Auth::Token DEBUG: Connecting to https://ec2-184-73-116-168.compute-1.amazonaws.com/broker/rest/api DEBUG: Getting all domains DEBUG: Client supports API versions 1.1, 1.2, 1.3, 1.4 DEBUG: Using token authentication DEBUG: Request GET https://ec2-184-73-116-168.compute-1.amazonaws.com/broker/rest/api DEBUG: SSL Verification failed -- Using self signed cert DEBUG: code 200 1118 ms DEBUG: Server supports API versions 1.0, 1.1, 1.2, 1.3, 1.4, 1.5 DEBUG: Client API version 1.4 is not current. Refetching API DEBUG: Using token authentication DEBUG: Request GET https://ec2-184-73-116-168.compute-1.amazonaws.com/broker/rest/api DEBUG: SSL Verification failed -- Using self signed cert DEBUG: code 200 1447 ms DEBUG: Using token authentication DEBUG: Request GET https://ec2-184-73-116-168.compute-1.amazonaws.com/broker/rest/domains DEBUG: SSL Verification failed -- Using self signed cert DEBUG: code 200 1286 ms DEBUG: Getting all domains DEBUG: Using token authentication DEBUG: Request GET https://ec2-184-73-116-168.compute-1.amazonaws.com/broker/rest/domains/cdm/applications/cphp0s0?include=cartridges DEBUG: code 200 851 ms DEBUG: Tail in progress for cartridge * DEBUG: ssh -t 51b2c5a507f64e0a3d000021.rhcloud.com 'tail */log*/*' DEBUG: Opening Net::SSH connection to cphp0s0-cdm.dev.rhcloud.com, 51b2c5a507f64e0a3d000021, tail */log*/* ==> php/logs/access_log-20130608-000000-EST <== - - - [08/Jun/2013:02:58:12 -0400] "GET / HTTP/1.0" 200 5235 "-" "-" - - - [08/Jun/2013:02:58:14 -0400] "GET / HTTP/1.0" 200 5235 "-" "-" - - - [08/Jun/2013:02:58:16 -0400] "GET / HTTP/1.0" 200 5235 "-" "-" - - - [08/Jun/2013:02:58:18 -0400] "GET / HTTP/1.0" 200 5235 "-" "-" - - - [08/Jun/2013:02:58:20 -0400] "GET / HTTP/1.0" 200 5235 "-" "-" - - - [08/Jun/2013:02:58:22 -0400] "GET / HTTP/1.0" 200 5235 "-" "-" - - - [08/Jun/2013:02:58:24 -0400] "GET / HTTP/1.0" 200 5235 "-" "-" - - - [08/Jun/2013:02:58:26 -0400] "GET / HTTP/1.0" 200 5235 "-" "-" - - - [08/Jun/2013:02:58:28 -0400] "GET / HTTP/1.0" 200 5235 "-" "-" - - - [08/Jun/2013:02:58:30 -0400] "GET / HTTP/1.0" 200 5235 "-" "-" ==> php/logs/error_log-20130608-000000-EST <== [Sat Jun 08 01:49:39 2013] [notice] SELinux policy enabled; httpd running as context unconfined_u:system_r:openshift_t:s0:c0,c529 [Sat Jun 08 01:49:39 2013] [notice] Digest: generating secret for digest authentication ... [Sat Jun 08 01:49:39 2013] [notice] Digest: done [Sat Jun 08 01:49:39 2013] [notice] Apache/2.2.15 (Unix) configured -- resuming normal operations [Sat Jun 08 02:17:58 2013] [error] [client 127.1.8.129] File does not exist: /var/lib/openshift/51b2c5a507f64e0a3d000021/app-root/runtime/repo/php/favicon.ico [Sat Jun 08 02:17:59 2013] [error] [client 127.1.8.129] File does not exist: /var/lib/openshift/51b2c5a507f64e0a3d000021/app-root/runtime/repo/php/favicon.ico ==> haproxy/logs/scale_events.log <== D, [2013-06-08T02:12:00.522958 #27535] DEBUG -- : GEAR_INFO - capacity: 0.0% gear_count: 1 sessions: 0 up/remove_thresh: 90.0%/1.0% sec_left_til_remove: 120 gear_remove_thresh: 0/20 I, [2013-06-08T02:14:29.137728 #31429] INFO -- : Starting haproxy_ctld D, [2013-06-08T02:14:29.145074 #31429] DEBUG -- : GEAR_INFO - capacity: 0.0% gear_count: 2 sessions: 0 up/remove_thresh: 90.0%/40.5% sec_left_til_remove: 120 gear_remove_thresh: 0/20 I, [2013-06-08T02:15:34.160364 #879] INFO -- : Starting haproxy_ctld D, [2013-06-08T02:15:34.162070 #879] DEBUG -- : GEAR_INFO - capacity: 0.0% gear_count: 2 sessions: 0 up/remove_thresh: 90.0%/40.5% sec_left_til_remove: 120 gear_remove_thresh: 0/20 I, [2013-06-08T02:17:37.282382 #879] INFO -- : GEAR_DOWN - capacity: 0.0% gear_count: 2 sessions: 0 remove_thresh: 40.5% I, [2013-06-08T02:18:28.756750 #6051] INFO -- : Starting haproxy_ctld D, [2013-06-08T02:18:28.764507 #6051] DEBUG -- : GEAR_INFO - capacity: 0.0% gear_count: 1 sessions: 0 up/remove_thresh: 90.0%/1.0% sec_left_til_remove: 120 gear_remove_thresh: 0/20 I, [2013-06-08T02:18:43.040586 #6510] INFO -- : Starting haproxy_ctld D, [2013-06-08T02:18:43.044220 #6510] DEBUG -- : GEAR_INFO - capacity: 0.0% gear_count: 1 sessions: 0 up/remove_thresh: 90.0%/1.0% sec_left_til_remove: 120 gear_remove_thresh: 0/20 ==> haproxy/logs/haproxy.log <== [WARNING] 158/014906 (2756) : config : log format ignored for proxy 'stats' since it has no log address. [WARNING] 158/014906 (2756) : config : log format ignored for proxy 'express' since it has no log address. ==> haproxy/logs/validate_config.log <== # Logfile created on 2013-06-08 01:49:07 -0400 by logger.rb/31641 ==> php/logs/access_log-20130608-000000-EST <== - - - [08/Jun/2013:02:58:32 -0400] "GET / HTTP/1.0" 200 5235 "-" "-" - - - [08/Jun/2013:02:58:34 -0400] "GET / HTTP/1.0" 200 5235 "-" "-" - - - [08/Jun/2013:02:58:36 -0400] "GET / HTTP/1.0" 200 5235 "-" "-"
Can you attach your express.conf file from ~/.openshift also open it up and check if the libra_server and default_rhlogin look correct to you.
Created attachment 760505 [details] express.conf
I have attached my express.conf. It is actual and everything works with it except rhc tail. Tried with a different user with the same result
I think, we might close this bug. I reinstalled the system, then installed all the rhc dependencies using "gem install" and the problem no longer persists. Most probably it was due to some rhc deps were previously installed using yum, but I can not remember which ones.
Tried on devenv_4125, and can not reproduce this issue. So verified this.