Bug 856920 - Can't connect to mongdb any more after move the mongodb gear which added into scalable app accross district
Can't connect to mongdb any more after move the mongodb gear which added into...
Status: CLOSED CURRENTRELEASE
Product: OpenShift Origin
Classification: Red Hat
Component: Kubernetes (Show other bugs)
2.x
Unspecified Unspecified
medium Severity medium
: ---
: ---
Assigned To: Rajat Chopra
libra bugs
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2012-09-13 02:52 EDT by Rony Gong
Modified: 2015-05-14 22:04 EDT (History)
0 users

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2012-09-17 17:28:54 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
development.log (22.98 KB, text/plain)
2012-09-13 02:52 EDT, Rony Gong
no flags Details
newest developement.log (22.97 KB, text/plain)
2012-09-13 22:28 EDT, Rony Gong
no flags Details

  None (edit)
Description Rony Gong 2012-09-13 02:52:47 EDT
Created attachment 612358 [details]
development.log

Description of problem:
After move, I found the mongodb process isn't up, so can't connect success.



Version-Release number of selected component (if applicable):
devenv_21


How reproducible:
always


Steps to Reproduce:
1.Create multile node env, create 2 districts and add node to them.
2.Create scalable app like(python,jboss, php),then embemd mongodb to this app.
3.Them move the mongodb gear accross district
rhc-admin-move --gear_uuid db7d9a70280e4aa9b8c68f1a9a65f084 -i ip-10-122-51-2 --allow_change_district

  
Actual results:
ssh into this app
[qsphp-qgong2.dev.rhcloud.com ~]\> mongo
MongoDB shell version: 2.0.7
connecting to: 318065ca46-qgong2.dev.rhcloud.com:38096/admin
Thu Sep 13 02:41:07 DBClientCursor::init call() failed
Thu Sep 13 02:41:07 Error: Error during mongo startup. :: caused by :: DBClientBase::findN: transport error: 318065ca46-qgong2.dev.rhcloud.com:38096 query: { whatsmyuri: 1 } shell/mongo.js:86
exception: connect failed



Expected results:
could connect mongodb success.


Additional info:
all other db like mysql and postgresql work well.
Comment 1 Rony Gong 2012-09-13 02:53:43 EDT
Same error for non_scalabel app has been fixed by https://bugzilla.redhat.com/show_bug.cgi?id=855825
Comment 2 Rajat Chopra 2012-09-13 18:58:00 EDT
Fixed. Pull request https://github.com/openshift/crankcase/pull/484
Comment 3 Rony Gong 2012-09-13 22:28:40 EDT
Created attachment 612706 [details]
newest  developement.log
Comment 4 Rony Gong 2012-09-13 22:31:42 EDT
Still reproduced on devenv_2176, this build has merged your pull,

after move:
[qsphp-qgong4.dev.rhcloud.com ~]\> mongo
MongoDB shell version: 2.0.7
connecting to: 81684aa525-qgong4.dev.rhcloud.com:38051/admin
Thu Sep 13 22:25:07 DBClientCursor::init call() failed
Thu Sep 13 22:25:07 Error: Error during mongo startup. :: caused by :: DBClientBase::findN: transport error: 81684aa525-qgong4.dev.rhcloud.com:38051 query: { whatsmyuri: 1 } shell/mongo.js:86
exception: connect failed


use mongodb user also can't log in server anymore(and I have waited some mins for DNS finish)
[qgong@localhost dev]$ ssh 81684aa5257b439e82b27ffc3fc9e772@81684aa525-qgong4.dev.rhcloud.com
Warning: Permanently added '81684aa525-qgong4.dev.rhcloud.com,184.72.66.10' (RSA) to the list of known hosts.
Permission denied (publickey,gssapi-keyex,gssapi-with-mic).
Comment 5 Rajat Chopra 2012-09-14 00:55:20 EDT
Yeah it was a bad fix. New (tested) fix is in with pull request - https://github.com/openshift/crankcase/pull/488

The mongodb user not being able to log in is a separate bug. It possibly exists with mysql also.
Comment 6 Rony Gong 2012-09-14 01:06:11 EDT
After move, can't find the process for the mongdb user in destination instance.
[root@ip-10-90-249-240 stickshift]# ps -ef|grep 1003
root     18136  1369  0 22:08 pts/1    00:00:00 grep 1003
Comment 7 Rajat Chopra 2012-09-14 01:16:14 EDT
How is ps -ef| grep 1003 suggesting the mongo process? 
If '1003' means uid, then uid can (and most likely would) change after move across districts. 

How to find out if mongo is still running/reachable? Connect to the web/haproxy gear and connect through OPENSHIFT_*DB_HOST/PORT.
Comment 8 Rony Gong 2012-09-14 02:26:29 EDT
(In reply to comment #7)
> How is ps -ef| grep 1003 suggesting the mongo process? 
> If '1003' means uid, then uid can (and most likely would) change after move
> across districts. 
> 
> How to find out if mongo is still running/reachable? Connect to the
> web/haproxy gear and connect through OPENSHIFT_*DB_HOST/PORT.

the 1003 is the already changed userid after move.

yes, we can Connect to the web/haproxy gear and connect through OPENSHIFT_*DB_HOST/PORT. type "mongo"
Comment 9 Rony Gong 2012-09-14 02:27:10 EDT
This issue fixed by this pull:
https://github.com/openshift/crankcase/pull/488
Comment 10 Rony Gong 2012-09-14 06:00:43 EDT
verified on devenv_2179

Note You need to log in before you can comment on or make changes to this bug.