Created attachment 678067 [details]
development.log

Description of problem:
Can't connect to db any more after move db gear of salable application across district.


Version-Release number of selected component (if applicable):
fork_ami_refctr1_404
rhc-1.3.2

How reproducible:
always

Steps to Reproduce:
1.Setup multi node env, and setup 2 districts to add nodes.
2.Create scalable app(nodejs) and embed db(mongodb).
3.Move this mongodb gear across district.
4.Connect to DB from haproxy gear
  
Actual results:
1.Move the db gear of salable application across district
[root@ip-10-84-219-69 openshift]# oo-admin-move --gear_uuid 50f3a0905b9f935407000162 -i domU-12-31-39-06-CC-FF --allow_change_district
URL: http://qsruby19-qgong12.dev.rhcloud.com
Login: qgong
App UUID: 50f396be5b9f93540700009a
Gear UUID: 50f3a0905b9f935407000162
DEBUG: Source district uuid: 50f376b85b9f93ba08000001
DEBUG: Destination district uuid: 50f39fa35b9f93c042000001
DEBUG: Getting existing app 'qsruby19' status before moving
DEBUG: Gear component 'ruby-1.9' was running
DEBUG: Stopping existing app cartridge 'mongodb-2.2' before moving
DEBUG: Performing cartridge level pre-move for embedded mongodb-2.2 for 'qsruby19' on ip-10-84-219-69
DEBUG: Reserved uid '1008' on district: '50f39fa35b9f93c042000001'
DEBUG: Creating new account for gear '50f3a0905b' on domU-12-31-39-06-CC-FF
DEBUG: Moving content for app 'qsruby19', gear '50f3a0905b' to domU-12-31-39-06-CC-FF
Identity added: /var/www/openshift/broker/config/keys/rsync_id_rsa (/var/www/openshift/broker/config/keys/rsync_id_rsa)
Agent pid 31429
unset SSH_AUTH_SOCK;
unset SSH_AGENT_PID;
echo Agent pid 31429 killed;
DEBUG: Performing cartridge level move for embedded mongodb-2.2 for 'qsruby19' on domU-12-31-39-06-CC-FF
DEBUG: Performing cartridge level post-move for embedded mongodb-2.2 for 'qsruby19' on domU-12-31-39-06-CC-FF
DEBUG: Starting cartridge 'mongodb-2.2' in 'qsruby19' after move on domU-12-31-39-06-CC-FF
DEBUG: Fixing DNS and mongo for gear '50f3a0905b' after move
DEBUG: Changing server identity of '50f3a0905b' from 'ip-10-84-219-69' to 'domU-12-31-39-06-CC-FF'
DEBUG: Deconfiguring old app 'qsruby19' on ip-10-84-219-69 after move
Successfully moved 'qsruby19' with gear uuid '50f3a0905b9f935407000162' from 'ip-10-84-219-69' to 'domU-12-31-39-06-CC-FF'
2. After move, check db connection by ssh into the haproxy gear
[qsruby19-qgong12.dev.rhcloud.com ~]\> mongo
+ mongo
+ test 0 -gt 0
+ uopt='--username admin'
+ '[' -n E3YwbatdlQd4 ']'
+ popt='--password E3YwbatdlQd4'
+ egrep '\-\-host|50f3a0905b-qgong12.dev.rhcloud.com'
+ echo
+ '[' -n 50f3a0905b-qgong12.dev.rhcloud.com ']'
+ hopt=50f3a0905b-qgong12.dev.rhcloud.com:38176/admin
+ unset LD_LIBRARY_PATH
+ /usr/bin/mongo 50f3a0905b-qgong12.dev.rhcloud.com:38176/admin --username admin --password E3YwbatdlQd4
MongoDB shell version: 2.2.0
connecting to: 50f3a0905b-qgong12.dev.rhcloud.com:38176/admin
Mon Jan 14 01:59:25 Error: couldn't connect to server 50f3a0905b-qgong12.dev.rhcloud.com:38176 src/mongo/shell/mongo.js:93
exception: connect failed


Expected results:
After move , should still could connect db successfully.

Additional info:
After move, check db connection by ssh into the db gear, connection well
[50f3a0905b-qgong12.dev.rhcloud.com ~]\> mongo
+ mongo
+ test 0 -gt 0
+ uopt='--username admin'
+ '[' -n E3YwbatdlQd4 ']'
+ popt='--password E3YwbatdlQd4'
+ egrep '\-\-host|127.1.248.1'
+ echo
+ '[' -n '' ']'
+ hopt=127.1.248.1:27017/admin
+ unset LD_LIBRARY_PATH
+ /usr/bin/mongo 127.1.248.1:27017/admin --username admin --password E3YwbatdlQd4
MongoDB shell version: 2.2.0
connecting to: 127.1.248.1:27017/admin
Welcome to the MongoDB shell.
For interactive help, type "help".
For more comprehensive documentation, see
	http://docs.mongodb.org/
Questions? Try the support group
	http://groups.google.com/group/mongodb-user