Bug 877300 - Unable to connect to DB cartridge for scalable apps after upgrade on INT
Summary: Unable to connect to DB cartridge for scalable apps after upgrade on INT
Keywords:
Status: CLOSED DUPLICATE of bug 876942
Alias: None
Product: OKD
Classification: Red Hat
Component: Containers
Version: 2.x
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: ---
Assignee: Rob Millner
QA Contact: libra bugs
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2012-11-16 07:54 UTC by Jianwei Hou
Modified: 2015-05-14 23:02 UTC (History)
1 user (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2012-11-16 17:16:11 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Jianwei Hou 2012-11-16 07:54:21 UTC
Description of problem:
After upgrade of INT, ssh into scalable app's gear, try to connect to db cartridge, operation failed.
For some scalable applications(not all, wired), unable to ssh into the db gear

Version-Release number of selected component (if applicable):
On INT

How reproducible:
Always

Steps to Reproduce:
1. Create scalable applications embedded with db cartridges on INT before upgrade
2. After INT is upgraded(by ops team), ssh into scalable app's gear and try to connect to db

  
Actual results:
# rhc app show ruby19s
ruby19s @ http://ruby19s-jhou1.int.rhcloud.com/
===============================================
  Application Info
  ================
    UUID      = 320c70ca09d047459037d2fb4838dc6e
    Gear Size = small
    Git URL   = ssh://320c70ca09d047459037d2fb4838dc6e.rhcloud.com/~/git/ruby19s.git/
    SSH URL   = ssh://320c70ca09d047459037d2fb4838dc6e.rhcloud.com
    Created   = Nov 09  1:00 AM
  Cartridges
  ==========
    haproxy-1.4
    ruby-1.9
    postgresql-8.4 = postgresql://810af02fb4-jhou1.int.rhcloud.com:38486/
    mongodb-2.2    = mongodb://8c1f9f8222-jhou1.int.rhcloud.com:56706/
  Scaling Info
  ============
    Scaled x2 (minimum: 2, maximum: available gears) with haproxy-1.4 on small gears

On app's haproxy gear:
[ruby19s-jhou1.int.rhcloud.com ~]\> psql
psql: [ruby19s-jhou1.int.rhcloud.com ~]\> 
[ruby19s-jhou1.int.rhcloud.com ~]\> mongo
MongoDB shell version: 2.2.0
connecting to: 8c1f9f8222-jhou1.int.rhcloud.com:56706/admin
Fri Nov 16 02:28:21 DBClientCursor::init call() failed
Fri Nov 16 02:28:21 Error: Error during mongo startup. :: caused by :: 10276 DBClientBase::findN: transport error: 8c1f9f8222-jhou1.int.rhcloud.com:56706 ns: admin.$cmd query: { whatsmyuri: 1 } src/mongo/shell/mongo.js:93
exception: connect failed

[ruby19s-jhou1.int.rhcloud.com ~]\> env|grep DB|sort
OPENSHIFT_MONGODB_DB_GEAR_DNS=8c1f9f8222-jhou1.int.rhcloud.com
OPENSHIFT_MONGODB_DB_GEAR_UUID=8c1f9f8222f643c1a4ea00b78e0e21e6
OPENSHIFT_MONGODB_DB_HOST=8c1f9f8222-jhou1.int.rhcloud.com
OPENSHIFT_MONGODB_DB_PASSWORD=GutCJIx6mK_S
OPENSHIFT_MONGODB_DB_PORT=56706
OPENSHIFT_MONGODB_DB_URL=mongodb://admin:GutCJIx6mK_S.rhcloud.com:56706/
OPENSHIFT_MONGODB_DB_USERNAME=admin
OPENSHIFT_POSTGRESQL_DB_GEAR_DNS=810af02fb4-jhou1.int.rhcloud.com
OPENSHIFT_POSTGRESQL_DB_GEAR_UUID=810af02fb4a0410a9cb99f671e9090c8
OPENSHIFT_POSTGRESQL_DB_HOST=810af02fb4-jhou1.int.rhcloud.com
OPENSHIFT_POSTGRESQL_DB_PASSWORD=lfrlZAIdxxIH
OPENSHIFT_POSTGRESQL_DB_PORT=38486
OPENSHIFT_POSTGRESQL_DB_SOCKET=/var/lib/openshift/810af02fb4a0410a9cb99f671e9090c8//postgresql-8.4/socket/postgres.sock
OPENSHIFT_POSTGRESQL_DB_URL=://admin:lfrlZAIdxxIH.rhcloud.com:38486/
OPENSHIFT_POSTGRESQL_DB_USERNAME=admin


sshed into db gear and connect to db from db gear was OK. And then after exiting from db gear, I was able to connect to db from haproxy gear again, see below:

[ruby19s-jhou1.int.rhcloud.com ~]\> ssh 810af02fb4a0410a9cb99f671e9090c8.rhcloud.com

   ...............

[810af02fb4-jhou1.int.rhcloud.com ~]\> ls
app-root  postgresql-8.4
[810af02fb4-jhou1.int.rhcloud.com ~]\> psql
psql (8.4.13)
Type "help" for help.

ruby19s=# \q
[810af02fb4-jhou1.int.rhcloud.com ~]\> exit
[ruby19s-jhou1.int.rhcloud.com ~]\> psql
psql (8.4.13)
Type "help" for help.

ruby19s=# 

For mongodb's gear, I wasn't able to ssh into it
[ruby19s-jhou1.int.rhcloud.com ~]\> ssh 8c1f9f8222f643c1a4ea00b78e0e21e6.rhcloud.com -v
OpenSSH_5.3p1, OpenSSL 1.0.0-fips 29 Mar 2010
debug1: Reading configuration data /var/lib/openshift/320c70ca09d047459037d2fb4838dc6e//haproxy-1.4/.ssh/config
debug1: Connecting to 8c1f9f8222-jhou1.int.rhcloud.com [10.118.89.196] port 22.
debug1: Connection established.
debug1: identity file /var/lib/openshift/320c70ca09d047459037d2fb4838dc6e//haproxy-1.4/.ssh/haproxy_id_rsa type 1
debug1: Remote protocol version 2.0, remote software version OpenSSH_5.3
debug1: match: OpenSSH_5.3 pat OpenSSH*
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_5.3
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: server->client aes128-ctr hmac-md5 none
debug1: kex: client->server aes128-ctr hmac-md5 none
debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP
debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY
debug1: Host '8c1f9f8222-jhou1.int.rhcloud.com' is known and matches the RSA host key.
debug1: Found key in /var/lib/openshift/320c70ca09d047459037d2fb4838dc6e//haproxy-1.4/.ssh/known_hosts:3
debug1: ssh_rsa_verify: signature correct
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic
debug1: Next authentication method: publickey
debug1: Offering public key: /var/lib/openshift/320c70ca09d047459037d2fb4838dc6e//haproxy-1.4/.ssh/haproxy_id_rsa
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic
debug1: No more authentication methods to try.
Permission denied (publickey,gssapi-keyex,gssapi-with-mic).


Expected results:
Should be able to connect to db after upgrade without first sshing into db gear

Additional info:

Comment 1 Rob Millner 2012-11-16 17:16:11 UTC

*** This bug has been marked as a duplicate of bug 876942 ***


Note You need to log in before you can comment on or make changes to this bug.