Description of problem: After upgrade of INT, ssh into scalable app's gear, try to connect to db cartridge, operation failed. For some scalable applications(not all, wired), unable to ssh into the db gear Version-Release number of selected component (if applicable): On INT How reproducible: Always Steps to Reproduce: 1. Create scalable applications embedded with db cartridges on INT before upgrade 2. After INT is upgraded(by ops team), ssh into scalable app's gear and try to connect to db Actual results: # rhc app show ruby19s ruby19s @ http://ruby19s-jhou1.int.rhcloud.com/ =============================================== Application Info ================ UUID = 320c70ca09d047459037d2fb4838dc6e Gear Size = small Git URL = ssh://320c70ca09d047459037d2fb4838dc6e.rhcloud.com/~/git/ruby19s.git/ SSH URL = ssh://320c70ca09d047459037d2fb4838dc6e.rhcloud.com Created = Nov 09 1:00 AM Cartridges ========== haproxy-1.4 ruby-1.9 postgresql-8.4 = postgresql://810af02fb4-jhou1.int.rhcloud.com:38486/ mongodb-2.2 = mongodb://8c1f9f8222-jhou1.int.rhcloud.com:56706/ Scaling Info ============ Scaled x2 (minimum: 2, maximum: available gears) with haproxy-1.4 on small gears On app's haproxy gear: [ruby19s-jhou1.int.rhcloud.com ~]\> psql psql: [ruby19s-jhou1.int.rhcloud.com ~]\> [ruby19s-jhou1.int.rhcloud.com ~]\> mongo MongoDB shell version: 2.2.0 connecting to: 8c1f9f8222-jhou1.int.rhcloud.com:56706/admin Fri Nov 16 02:28:21 DBClientCursor::init call() failed Fri Nov 16 02:28:21 Error: Error during mongo startup. :: caused by :: 10276 DBClientBase::findN: transport error: 8c1f9f8222-jhou1.int.rhcloud.com:56706 ns: admin.$cmd query: { whatsmyuri: 1 } src/mongo/shell/mongo.js:93 exception: connect failed [ruby19s-jhou1.int.rhcloud.com ~]\> env|grep DB|sort OPENSHIFT_MONGODB_DB_GEAR_DNS=8c1f9f8222-jhou1.int.rhcloud.com OPENSHIFT_MONGODB_DB_GEAR_UUID=8c1f9f8222f643c1a4ea00b78e0e21e6 OPENSHIFT_MONGODB_DB_HOST=8c1f9f8222-jhou1.int.rhcloud.com OPENSHIFT_MONGODB_DB_PASSWORD=GutCJIx6mK_S OPENSHIFT_MONGODB_DB_PORT=56706 OPENSHIFT_MONGODB_DB_URL=mongodb://admin:GutCJIx6mK_S.rhcloud.com:56706/ OPENSHIFT_MONGODB_DB_USERNAME=admin OPENSHIFT_POSTGRESQL_DB_GEAR_DNS=810af02fb4-jhou1.int.rhcloud.com OPENSHIFT_POSTGRESQL_DB_GEAR_UUID=810af02fb4a0410a9cb99f671e9090c8 OPENSHIFT_POSTGRESQL_DB_HOST=810af02fb4-jhou1.int.rhcloud.com OPENSHIFT_POSTGRESQL_DB_PASSWORD=lfrlZAIdxxIH OPENSHIFT_POSTGRESQL_DB_PORT=38486 OPENSHIFT_POSTGRESQL_DB_SOCKET=/var/lib/openshift/810af02fb4a0410a9cb99f671e9090c8//postgresql-8.4/socket/postgres.sock OPENSHIFT_POSTGRESQL_DB_URL=://admin:lfrlZAIdxxIH.rhcloud.com:38486/ OPENSHIFT_POSTGRESQL_DB_USERNAME=admin sshed into db gear and connect to db from db gear was OK. And then after exiting from db gear, I was able to connect to db from haproxy gear again, see below: [ruby19s-jhou1.int.rhcloud.com ~]\> ssh 810af02fb4a0410a9cb99f671e9090c8.rhcloud.com ............... [810af02fb4-jhou1.int.rhcloud.com ~]\> ls app-root postgresql-8.4 [810af02fb4-jhou1.int.rhcloud.com ~]\> psql psql (8.4.13) Type "help" for help. ruby19s=# \q [810af02fb4-jhou1.int.rhcloud.com ~]\> exit [ruby19s-jhou1.int.rhcloud.com ~]\> psql psql (8.4.13) Type "help" for help. ruby19s=# For mongodb's gear, I wasn't able to ssh into it [ruby19s-jhou1.int.rhcloud.com ~]\> ssh 8c1f9f8222f643c1a4ea00b78e0e21e6.rhcloud.com -v OpenSSH_5.3p1, OpenSSL 1.0.0-fips 29 Mar 2010 debug1: Reading configuration data /var/lib/openshift/320c70ca09d047459037d2fb4838dc6e//haproxy-1.4/.ssh/config debug1: Connecting to 8c1f9f8222-jhou1.int.rhcloud.com [10.118.89.196] port 22. debug1: Connection established. debug1: identity file /var/lib/openshift/320c70ca09d047459037d2fb4838dc6e//haproxy-1.4/.ssh/haproxy_id_rsa type 1 debug1: Remote protocol version 2.0, remote software version OpenSSH_5.3 debug1: match: OpenSSH_5.3 pat OpenSSH* debug1: Enabling compatibility mode for protocol 2.0 debug1: Local version string SSH-2.0-OpenSSH_5.3 debug1: SSH2_MSG_KEXINIT sent debug1: SSH2_MSG_KEXINIT received debug1: kex: server->client aes128-ctr hmac-md5 none debug1: kex: client->server aes128-ctr hmac-md5 none debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP debug1: SSH2_MSG_KEX_DH_GEX_INIT sent debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY debug1: Host '8c1f9f8222-jhou1.int.rhcloud.com' is known and matches the RSA host key. debug1: Found key in /var/lib/openshift/320c70ca09d047459037d2fb4838dc6e//haproxy-1.4/.ssh/known_hosts:3 debug1: ssh_rsa_verify: signature correct debug1: SSH2_MSG_NEWKEYS sent debug1: expecting SSH2_MSG_NEWKEYS debug1: SSH2_MSG_NEWKEYS received debug1: SSH2_MSG_SERVICE_REQUEST sent debug1: SSH2_MSG_SERVICE_ACCEPT received debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic debug1: Next authentication method: publickey debug1: Offering public key: /var/lib/openshift/320c70ca09d047459037d2fb4838dc6e//haproxy-1.4/.ssh/haproxy_id_rsa debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic debug1: No more authentication methods to try. Permission denied (publickey,gssapi-keyex,gssapi-with-mic). Expected results: Should be able to connect to db after upgrade without first sshing into db gear Additional info:
*** This bug has been marked as a duplicate of bug 876942 ***