Bug 1020003 - rhc tail does not work in PROD...got authentication failed error. [NEEDINFO]
rhc tail does not work in PROD...got authentication failed error.
Status: CLOSED CURRENTRELEASE
Product: OpenShift Online
Classification: Red Hat
Component: Command Line Interface (Show other bugs)
2.x
Unspecified Unspecified
high Severity unspecified
: ---
: ---
Assigned To: Fabiano Franz
libra bugs
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2013-10-16 14:43 EDT by Peter Ruan
Modified: 2015-05-14 22:25 EDT (History)
4 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2014-01-23 22:24:35 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
ffranz: needinfo? (pruan)


Attachments (Terms of Use)

  None (edit)
Description Peter Ruan 2013-10-16 14:43:10 EDT
Description of problem:
rhc tail does not work in PROD...

Version-Release number of selected component (if applicable):
PROD/STG

How reproducible:
always

Steps to Reproduce:
1. create an app 'rhc app create app2 php-5.3'
2. tail the app by doing 'rhc tail app2'
3. I can ssh into the app using the 'ssh' message displayed.

Actual results:
[peter@unused-16-138 <CUCUSHIFT_PROD> cucushift]# rhc tail app2 
You can tail this application directly with:
ssh -t 525ed91f5973ca253e00021a@app2-foih0b.rhcloud.com 'tail */log*/*'
/home/peter/.rvm/gems/ruby-1.9.3-p448/gems/net-ssh-2.6.8/lib/net/ssh.rb:207:in `start': 525ed91f5973ca253e00021a (Net::SSH::AuthenticationFailed)
	from /home/peter/.rvm/gems/ruby-1.9.3-p448/gems/rhc-1.16.0/lib/rhc/ssh_helpers.rb:167:in `ssh_ruby'
	from /home/peter/.rvm/gems/ruby-1.9.3-p448/gems/rhc-1.16.0/lib/rhc/commands/tail.rb:40:in `tail'
	from /home/peter/.rvm/gems/ruby-1.9.3-p448/gems/rhc-1.16.0/lib/rhc/commands/tail.rb:21:in `run'
	from /home/peter/.rvm/gems/ruby-1.9.3-p448/gems/rhc-1.16.0/lib/rhc/commands.rb:285:in `execute'
	from /home/peter/.rvm/gems/ruby-1.9.3-p448/gems/rhc-1.16.0/lib/rhc/commands.rb:276:in `block (3 levels) in to_commander'
	from /home/peter/.rvm/gems/ruby-1.9.3-p448/gems/commander-4.1.3/lib/commander/command.rb:180:in `call'
	from /home/peter/.rvm/gems/ruby-1.9.3-p448/gems/commander-4.1.3/lib/commander/command.rb:180:in `call'
	from /home/peter/.rvm/gems/ruby-1.9.3-p448/gems/commander-4.1.3/lib/commander/command.rb:155:in `run'
	from /home/peter/.rvm/gems/ruby-1.9.3-p448/gems/commander-4.1.3/lib/commander/runner.rb:402:in `run_active_command'
	from /home/peter/.rvm/gems/ruby-1.9.3-p448/gems/rhc-1.16.0/lib/rhc/command_runner.rb:73:in `run!'
	from /home/peter/.rvm/gems/ruby-1.9.3-p448/gems/commander-4.1.3/lib/commander/delegates.rb:11:in `run!'
	from /home/peter/.rvm/gems/ruby-1.9.3-p448/gems/rhc-1.16.0/lib/rhc/cli.rb:37:in `start'
	from /home/peter/.rvm/gems/ruby-1.9.3-p448/gems/rhc-1.16.0/bin/rhc:20:in `<top (required)>'
	from /home/peter/.rvm/gems/ruby-1.9.3-p448/bin/rhc:23:in `load'
	from /home/peter/.rvm/gems/ruby-1.9.3-p448/bin/rhc:23:in `<main>'
	from /home/peter/.rvm/gems/ruby-1.9.3-p448/bin/ruby_noexec_wrapper:14:in `eval'
	from /home/peter/.rvm/gems/ruby-1.9.3-p448/bin/ruby_noexec_wrapper:14:in `<main>'


Expected results:


Additional info:
Comment 1 Jordan Liggitt 2013-10-18 14:47:51 EDT
Can't recreate. Can you rerun with 'rhc -d tail app2' and incloude result?
Comment 2 Peter Ruan 2013-10-21 19:17:05 EDT
[peter@unused-16-138 <DEV> ~]# rhc app create app2 php-5.3
Application Options
-------------------
  Domain:     lvt0u1
  Cartridges: php-5.3
  Gear Size:  default
  Scaling:    no

Creating application 'app2' ... done


Waiting for your DNS name to be available ... done

Cloning into 'app2'...
Warning: Permanently added 'app2-lvt0u1.rhcloud.com' (RSA) to the list of known hosts.

Your application 'app2' is now available.

  URL:        http://app2-lvt0u1.rhcloud.com/
  SSH to:     5265b549500446080a000171@app2-lvt0u1.rhcloud.com
  Git remote: ssh://5265b549500446080a000171@app2-lvt0u1.rhcloud.com/~/git/app2.git/
  Cloned to:  /home/peter/app2
[peter@unused-16-138 <DEV> ~]# rhc tail app2 -d
DEBUG: Using config file /home/peter/.openshift/express.conf
DEBUG: Git config 'git config --get rhc.app-id' returned ''
DEBUG: Git config 'git config --get rhc.app-name' returned ''
DEBUG: Git config 'git config --get rhc.domain-name' returned ''
DEBUG: Authenticating with RHC::Auth::Token
DEBUG: Connecting to https://openshift.redhat.com/broker/rest/api
DEBUG: Getting all domains
DEBUG: Client supports API versions 1.1, 1.2, 1.3, 1.4, 1.5
DEBUG: Using token authentication
DEBUG: Created new httpclient
DEBUG: Request GET https://openshift.redhat.com/broker/rest/api
DEBUG:    code 200  660 ms
DEBUG: Server supports API versions 1.0, 1.1, 1.2, 1.3, 1.4, 1.5, 1.6
DEBUG:    Using API version 1.5
DEBUG: Client API version 1.5 is not current. Refetching API
DEBUG: Using token authentication
DEBUG: Request GET https://openshift.redhat.com/broker/rest/api
DEBUG:    code 200  185 ms
DEBUG: Using token authentication
DEBUG: Request GET https://openshift.redhat.com/broker/rest/domains
DEBUG:    code 200  247 ms
DEBUG: Using token authentication
DEBUG: Request GET https://openshift.redhat.com/broker/rest/domains/lvt0u1/applications/app2?include=cartridges
DEBUG:    code 200  323 ms
DEBUG: Tail in progress for cartridge *
DEBUG: ssh -t 5265b549500446080a000171@app2-lvt0u1.rhcloud.com 'tail */log*/*'
DEBUG: Opening Net::SSH connection to app2-lvt0u1.rhcloud.com, 5265b549500446080a000171, tail */log*/*
You can tail this application directly with:
ssh -t 5265b549500446080a000171@app2-lvt0u1.rhcloud.com 'tail */log*/*'
/home/peter/.rvm/gems/ruby-1.9.3-p448/gems/net-ssh-2.6.8/lib/net/ssh.rb:207:in `start': 5265b549500446080a000171 (Net::SSH::AuthenticationFailed)
	from /home/peter/.rvm/gems/ruby-1.9.3-p448/gems/rhc-1.16.0/lib/rhc/ssh_helpers.rb:167:in `ssh_ruby'
	from /home/peter/.rvm/gems/ruby-1.9.3-p448/gems/rhc-1.16.0/lib/rhc/commands/tail.rb:40:in `tail'
	from /home/peter/.rvm/gems/ruby-1.9.3-p448/gems/rhc-1.16.0/lib/rhc/commands/tail.rb:21:in `run'
	from /home/peter/.rvm/gems/ruby-1.9.3-p448/gems/rhc-1.16.0/lib/rhc/commands.rb:285:in `execute'
	from /home/peter/.rvm/gems/ruby-1.9.3-p448/gems/rhc-1.16.0/lib/rhc/commands.rb:276:in `block (3 levels) in to_commander'
	from /home/peter/.rvm/gems/ruby-1.9.3-p448/gems/commander-4.1.3/lib/commander/command.rb:180:in `call'
	from /home/peter/.rvm/gems/ruby-1.9.3-p448/gems/commander-4.1.3/lib/commander/command.rb:180:in `call'
	from /home/peter/.rvm/gems/ruby-1.9.3-p448/gems/commander-4.1.3/lib/commander/command.rb:155:in `run'
	from /home/peter/.rvm/gems/ruby-1.9.3-p448/gems/commander-4.1.3/lib/commander/runner.rb:402:in `run_active_command'
	from /home/peter/.rvm/gems/ruby-1.9.3-p448/gems/rhc-1.16.0/lib/rhc/command_runner.rb:73:in `run!'
	from /home/peter/.rvm/gems/ruby-1.9.3-p448/gems/commander-4.1.3/lib/commander/delegates.rb:11:in `run!'
	from /home/peter/.rvm/gems/ruby-1.9.3-p448/gems/rhc-1.16.0/lib/rhc/cli.rb:37:in `start'
	from /home/peter/.rvm/gems/ruby-1.9.3-p448/gems/rhc-1.16.0/bin/rhc:20:in `<top (required)>'
	from /home/peter/.rvm/gems/ruby-1.9.3-p448/bin/rhc:23:in `load'
	from /home/peter/.rvm/gems/ruby-1.9.3-p448/bin/rhc:23:in `<main>'
	from /home/peter/.rvm/gems/ruby-1.9.3-p448/bin/ruby_noexec_wrapper:14:in `eval'
	from /home/peter/.rvm/gems/ruby-1.9.3-p448/bin/ruby_noexec_wrapper:14:in `<main>'
Comment 3 Fabiano Franz 2013-10-22 10:49:16 EDT
Couldn't reproduce either. 'rhc tail' uses ssh and the logs say 'Net::SSH::AuthenticationFailed' so it's probably related to ssh keys. Can you please try and include the results:

1. ssh directly: ssh 5265b549500446080a000171@app2-lvt0u1.rhcloud.com
2. tail manually: ssh -t 5265b549500446080a000171@app2-lvt0u1.rhcloud.com 'tail */log*/*'
3. list the existing ssh keys: rhc sshkey list

If we confirm it's an authentication issue we should probably catch that exception and display a user-friendly error message.
Comment 4 Jordan Liggitt 2013-10-22 13:29:31 EDT
Seems like it might be something env specific.

Are you using a proxy? If so, is it an authenticated proxy?
env | grep -i proxy

Can you also include the result of running the standalone ssh command in verbose mode?
ssh -v -t 5265b549500446080a000171@app2-lvt0u1.rhcloud.com 'tail */log*/*'
Comment 5 Fabiano Franz 2013-10-24 10:16:53 EDT
I was able to reproduce it by removing all SSH keys in my system (remove ~/.ssh). Added a fix for handling the AuthenticationFailed exception and displaying a proper error message to user. Already available on master.
Comment 6 Fabiano Franz 2013-10-25 13:52:13 EDT
Please test.
Comment 7 Wei Sun 2013-10-27 23:46:39 EDT
Verified on devenv_3953,rhc is rhc-1.16.4

After removing all SSH keys  in the system,try to tail the app,will get a proper error message.

Result:
[wsun@dhcp-8-229 dev]$ rhc tail app2
You can tail this application directly with:
ssh -t 526dd7b25ebecfd459000001@app2-wsundev3953.dev.rhcloud.com 'tail */log*/*'
Authentication to server app2-wsundev3953.dev.rhcloud.com with user 526dd7b25ebecfd459000001 failed

Note You need to log in before you can comment on or make changes to this bug.