Hide Forgot
Description of problem: mongod process is not started after upgrade packages and run migrate script. Version-Release number of selected component (if applicable): rhc-node-0.84.23-1.el6_2.noarch rhc-cartridge-rockmongo-1.1-1.2.6-1.el6_2.noarch rhc-cartridge-mongodb-2.0-0.12.7-1.el6_2.noarch rhc-devenv-0.84.11-1.el6_2.noarch How reproducible: Always Steps to Reproduce: 1. Create apps, embed mongodb and rockmongo to it against old instance (stage_110) 2. Use rockmongo app access mongodb successfully. 3. Upgrade all the packages to the latest. # yum update -y 3. Running migrate script for li git repo. # ./migrate-2.0.3 4. Try to access mongodb using rockmongo app Actual results: mongodb can not be accessed with error message - "Transport endpoint is not connected". Log into app using rhcsh, found mongodb process is not listed in the output of "ps -ef". Start mongodb via appname_mongodb_ctl.sh, rockmongo can access mongodb now. Expected results: After upgrade, mongodb should be always running. Additional info: !!!!!NOTE!!!!! I am using a large instance (devenv_stage_110), before upgrade the disk usage about 79%; after upgrade the disk usage reached to 98%. I almost can not create more app. Or if there are already some more existing app before upgrade, after upgrade I think the disk usage must reach 100%. This is very risky. Check the size of /var/lib/mongodb/, found it occupied most of available disk usage. # du -sm /var/lib/mongodb/* 1108 /var/lib/mongodb/journal 1 /var/lib/mongodb/mongod.lock Found such a process is running after upgrade, compare with devenv_stage_112, this process should not be running. # ps -ef|grep mongo root 17257 14262 0 07:21 pts/0 00:00:00 grep mongo mongodb 24613 1 0 06:05 ? 00:00:01 /usr/bin/mongod --quiet -f /etc/mongodb.conf run
Just a note on /var/lib/mongodb/ that happens automatically when someone starts the system mongod package. Which I believe we're disabling. But if you're seeing that directory full, and you didn't start mongo manually, something's starting it, we'll have to figure that out. Also, a running system mongod could also explain why your application's mongod didn't start.
Here is some clue for your debug, when I "yum update", I found system mongod package is also been updated, so I guess maybe system mongod is started during pakcage update process.
Discussed with McGrath. Could not reproduce. Please re-open if you observe again.