Bug 1770552
| Summary: | mongo storage upgrade to wiredtiger fails and drops database if mongod is stopped | ||
|---|---|---|---|
| Product: | Red Hat Satellite | Reporter: | Francisco Garcia <fgarciad> |
| Component: | Installation | Assignee: | Chris Roberts <chrobert> |
| Status: | CLOSED CURRENTRELEASE | QA Contact: | Devendra Singh <desingh> |
| Severity: | medium | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 6.6.0 | CC: | bkearney, chrobert, pcreech |
| Target Milestone: | Unspecified | Keywords: | Triaged |
| Target Release: | Unused | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | foreman-installer-1.24.1.3-1 | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2020-05-01 13:05:25 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
|
Description
Francisco Garcia
2019-11-10 13:20:35 UTC
Will test this on upstream but I think this is fixed, if not will look into it more. I was able to reproduce this, and yeah that is bad with it blowing away the db, I will work on a pr for this: foreman-maintain service stop --exclude "rh-mongodb34-mongod","postgresql","tomcat","dynflowd","foreman-proxy","puppetserver" finished successfully! [ INFO 2020-01-09T17:30:51 verbose] Starting mongodump to /var/tmp/mongodb_engine_upgrade 2020-01-09T17:30:55.214-0500 Failed: error connecting to db server: no reachable servers mongodump --host localhost --out /var/tmp/mongodb_engine_upgrade failed! Check the output for error! [ERROR 2020-01-09T17:30:55 verbose] mongodump --host localhost --out /var/tmp/mongodb_engine_upgrade failed! Check the output for error! [ INFO 2020-01-09T17:30:55 verbose] Export done, stopping MongoDB to remove old datafiles Running Stop Services ================================================================================ Check if command is run as root user: [OK] -------------------------------------------------------------------------------- Stop applicable services: Stopping the following service(s): rh-mongodb34-mongod | All services stopped [OK] -------------------------------------------------------------------------------- foreman-maintain service stop --only rh-mongodb34-mongod finished successfully! [ INFO 2020-01-09T17:30:57 verbose] Removing contents from /var/lib/mongodb rm -rf /var/lib/mongodb/* finished successfully! [ INFO 2020-01-09T17:30:58 verbose] Changing config to WiredTiger and starting restore. sed -i.bak -e 's/mmapv1/wiredTiger/g' /etc/opt/rh/rh-mongodb34/mongod.conf finished successfully! mv /etc/opt/rh/rh-mongodb34/mongod.conf.bak /var/tmp/mongodb_engine_upgrade finished successfully! Running Start Services ================================================================================ Check if command is run as root user: [OK] -------------------------------------------------------------------------------- Start applicable services: Starting the following service(s): rh-mongodb34-mongod / All services started [OK] -------------------------------------------------------------------------------- foreman-maintain service start --only rh-mongodb34-mongod finished successfully! 2020-01-09T17:31:00.439-0500 the --db and --collection args should only be used when restoring from a BSON file. lude instead 2020-01-09T17:31:00.439-0500 Failed: mongorestore target '/var/tmp/mongodb_engine_upgrade/pulp_database' invalid rectory mongorestore --host localhost --db=pulp_database --drop --dir=/var/tmp/mongodb_engine_upgrade/pulp_database failed! [ERROR 2020-01-09T17:31:00 verbose] mongorestore --host localhost --db=pulp_database --drop --dir=/var/tmp/mongodb_ [ERROR 2020-01-09T17:31:00 verbose] The restore could not be completed correctly, reverting actions. [ INFO 2020-01-09T17:31:00 verbose] Stopping MongoDB Running Stop Services ================================================================================ Check if command is run as root user: [OK] -------------------------------------------------------------------------------- Stop applicable services: Stopping the following service(s): rh-mongodb34-mongod | All services stopped [OK] -------------------------------------------------------------------------------- foreman-maintain service stop --only rh-mongodb34-mongod finished successfully! [ INFO 2020-01-09T17:31:02 verbose] Restoring old config mv -f /var/tmp/mongodb_engine_upgrade/mongod.conf.bak /etc/opt/rh/rh-mongodb34/mongod.conf finished successfully! [ INFO 2020-01-09T17:31:02 verbose] Removing contents in /var/lib/mongodb rm -rf /var/lib/mongodb/* finished successfully! [ INFO 2020-01-09T17:31:02 verbose] Restoring database under MMAPV1 storage engine 2020-01-09T17:31:06.357-0500 Failed: error connecting to db server: no reachable servers mongorestore --host localhost --db=pulp_database --drop --dir=/var/tmp/mongodb_engine_upgrade/pulp_database failed! [ERROR 2020-01-09T17:31:06 verbose] mongorestore --host localhost --db=pulp_database --drop --dir=/var/tmp/mongodb_ [ INFO 2020-01-09T17:31:06 verbose] Starting MongoDB with old config and database files Running Start Services ================================================================================ Check if command is run as root user: [OK] -------------------------------------------------------------------------------- Start applicable services: Starting the following service(s): rh-mongodb34-mongod - All services started [OK] -------------------------------------------------------------------------------- foreman-maintain service start --only rh-mongodb34-mongod finished successfully! [ERROR 2020-01-09T17:31:08 verbose] Mongo started up in MMAPV1 mode, backup at /var/tmp/mongodb_engine_upgrade [ INFO 2020-01-09T17:31:08 verbose] Installer finished in 131.457323212 seconds Moving this bug to POST for triage into Satellite 6 since the upstream issue https://projects.theforeman.org/issues/28712 has been resolved. Verified on 6.7. Verfication point: # cat /etc/foreman-installer/custom-hiera.yaml .... # Added by foreman-installer during upgrade, run the installer with --upgrade-mongo-storage to upgrade to WiredTiger. mongodb::server::storage_engine: 'mmapv1' # systemctl stop mongod # satellite-installer --upgrade-mongo-storage-engine Ensuring MongoDB is running before upgrade. Running Start Services ================================================================================ Check if command is run as root user: [OK] .... ... mongorestore --host localhost --db=pulp_database --drop --dir=/var/tmp/mongodb_engine_upgrade/pulp_database finished successfully! rm -rf /var/tmp/mongodb_engine_upgrade finished successfully! sed -i -e 's/Added by foreman-installer during upgrade, run the installer with --upgrade-mongo-storage.* to upgrade to WiredTiger./Do not remove/g' /etc/foreman-installer/custom-hiera.yaml finished successfully! sed -i -e 's/mmapv1/wiredTiger/g' /etc/foreman-installer/custom-hiera.yaml finished successfully! sed -i -e 's/mmapv1/wiredTiger/g' /etc/foreman-installer/custom-hiera.yaml finished successfully! Installing Done [100%] [................................................................................................................................] Success! This was delivered with Satellite 6.7.0. |