Created attachment 612748 [details] mongodb.log Description of problem: After an upgrade on STG, the embedded mongodb cartridge is stopped. Version-Release number of selected component (if applicable): on STG How reproducible: Always Steps to Reproduce: 1. On STG, create app and embed mongodb, app and mongodb are both running 2. After STG has been upgraded, check apps and mongodb. Actual results: App is running, but mongodb is stopped. [hjw@hjwlaptop ruby19]$ rhc app cartridge status -a ruby19s -c mongodb-2.0 Password: ****** RESULT: MongoDB is stopped Here are some information in log file, full log is attached: Thu Sep 13 07:51:00 [PeriodicTask::Runner] task: DBConnectionPool-cleaner took: 4ms Thu Sep 13 10:51:16 got kill or ctrl c or hup signal 15 (Terminated), will terminate after current cmd ends Thu Sep 13 10:51:16 [interruptThread] now exiting Thu Sep 13 10:51:16 dbexit: Thu Sep 13 10:51:16 [interruptThread] shutdown: going to close listening sockets... Thu Sep 13 10:51:16 [interruptThread] closing listening socket: 9 Thu Sep 13 10:51:16 [interruptThread] shutdown: going to flush diaglog... Thu Sep 13 10:51:16 [interruptThread] shutdown: going to close sockets... Thu Sep 13 10:51:16 [interruptThread] shutdown: waiting for fs preallocator... Thu Sep 13 10:51:16 [interruptThread] shutdown: closing all files... Expected results: mongodb should be running. Additional info: I have one non-scaling perl app and a scaling ruby-1.9 app, they are both embedded with mongodb, which stopped for both of them. Have to start mongodb from CLI to get it get back to work
Upgrading the mongodb kills all instances of mongodb running in gears. Adding a restart to the migration script brings it back up. Li commit 5f87da6.
Master pull request: https://github.com/openshift/li/pull/373 Stage pull request: https://github.com/openshift/li/pull/375
Reset severity high based on this will wipe out all instances of mongodb on all gears in prod when we upgrade.
Merged into the stage and master branches.
Rob's patch is a part of devenv-stage_237 (ami-f58b3b9c)
Verified on STAGE Steps: 1. Create apps and embed mongodb before upgrade on STAGE 2. After STAGE has been upgraded check app and embedded mongodb Result: App and mongodb are both running.