Bug 1636328 - foreman-installer --reset cannot empty local mongo 3.4 database
Summary: foreman-installer --reset cannot empty local mongo 3.4 database
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Satellite 6
Classification: Red Hat
Component: Installer
Version: 6.3.0
Hardware: Unspecified
OS: Unspecified
unspecified
high vote
Target Milestone: 6.4.1
Assignee: Chris Roberts
QA Contact: Lai
URL: https://projects.theforeman.org/issue...
Whiteboard:
: 1637644 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-10-05 06:40 UTC by Ales Dujicek
Modified: 2019-10-30 19:38 UTC (History)
5 users (show)

Fixed In Version: katello-installer-base-3.7.0.11-1
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-12-06 22:32:53 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Foreman Issue Tracker 23620 'Normal' 'Closed' 'foreman-installer --reset cannot empty local mongo 3.4 database' 2019-12-09 11:19:15 UTC
Github Katello katello-installer pull 708 'None' 'closed' 'Fixes #23620 - Fix reset for local & remote MongoDB.' 2019-12-09 11:19:14 UTC
Red Hat Product Errata RHBA-2018:3799 None None None 2018-12-06 22:33:26 UTC

Description Ales Dujicek 2018-10-05 06:40:33 UTC
reset deletes the database this way:
<pre>
hooks/pre/10-reset_feature.rb
def empty_mongo
        'systemctl stop rh-mongodb34-mongod',
        'rm -f /var/lib/mongodb/pulp_database*',
        'systemctl start rh-mongodb34-mongod'
end
</pre>
but mongo 3.4 stores data in a different way than 2.6, so this does not work for 3.4



mongo 2.6
<pre>
# ls /var/lib/mongodb/
journal  local.0  local.ns  mongod.lock  pulp_database.0  pulp_database.ns
</pre>

mongo 3.4
<pre>
# ls /var/lib/mongodb | grep -v -e collection- -e index-
diagnostic.data
journal
_mdb_catalog.wt
mongod.lock
sizeStorer.wt
storage.bson
WiredTiger
WiredTigerLAS.wt
WiredTiger.lock
WiredTiger.turtle
WiredTiger.wt
</pre>

Comment 1 Ales Dujicek 2018-10-05 06:40:37 UTC
Created from redmine issue http://projects.theforeman.org/issues/23620

Comment 2 Ales Dujicek 2018-10-05 06:40:41 UTC
Upstream bug assigned to jomitsch@redhat.com

Comment 4 Ales Dujicek 2018-10-05 06:58:47 UTC
In my opinion this is still a problem
foreman-maintain restore runs installer --reset which does not remove mongo data files, because they are not called /var/lib/mongodb/pulp_database* anymore

foreman-maintain.log:
[ INFO 2018-10-02T09:16:50 verbose] Running installer with args [["-v", "--reset", "--disable-system-checks"]]
...
[ INFO 2018-10-02T10:18:13 verbose] Dropping Pulp database!
rm -f /var/lib/pulp/init.flag finished successfully!
systemctl stop httpd pulp_workers finished successfully!
systemctl stop rh-mongodb34-mongod finished successfully!
rm -f /var/lib/mongodb/pulp_database* finished successfully!
systemctl start rh-mongodb34-mongod finished successfully!
rm -rf /var/lib/pulp/{distributions,published,repos}/* finished successfully!


and later extracts mongo data directory backup

D, [2018-10-02 10:24:54+0200 #15345] DEBUG -- : Running command tar --selinux --extract --file=/tmp/backup/offline/satellite-backup-2018-10-02-09-00-55/mongo_data.tar.gz --overwrite --gzip --listed-incremental=/dev/null --directory=/ with stdin nil

which means that there may be hypothetically some extra files which are not overwritten by the restored files (just because they did not exist at the time of backup creation)

Comment 6 Brad Buckingham 2018-10-10 15:29:20 UTC
*** Bug 1637644 has been marked as a duplicate of this bug. ***

Comment 8 pm-sat@redhat.com 2018-10-10 16:05:41 UTC
Upstream bug assigned to jomitsch@redhat.com

Comment 11 Lai 2018-11-15 19:46:43 UTC
What's the expected result for this?  And how do I setup data in mongodb or verify that mongodb is emptied correctly?

Does running ls /var/lib/mongodb | grep -v -e collection- -e index- return nothing after I run the foreman-installer --reset?

Comment 12 Ales Dujicek 2018-11-16 06:47:21 UTC
you do not need to setup anything, pulp stores its data in mongo, there will be data

you can pause the execution of satellite-installer after it deletes mongo database and check directly in database that it was removed

# mongo
> show dbs
should not list pulp_database

No, ls /var/lib/mongodb | grep -v -e collection- -e index- would always list files because foreman-installer creates new database after it is removed.

Comment 13 Lai 2018-11-16 19:18:05 UTC
Checked mongo db before reset:

> show dbs
admin          0.000GB
local          0.000GB
pulp_database  0.001GB

Ran satellite-installer --reset and paused it after deleting mongo database.  Checked db.

# mongo
MongoDB shell version v3.4.9
MongoDB server version: 3.4.9
Server has startup warnings: 
2018-11-16T13:55:02.275-0500 I CONTROL  [initandlisten] 
2018-11-16T13:55:02.275-0500 I CONTROL  [initandlisten] ** WARNING: You are running on a NUMA machine.
2018-11-16T13:55:02.275-0500 I CONTROL  [initandlisten] **          We suggest launching mongod like this to avoid performance problems:
2018-11-16T13:55:02.275-0500 I CONTROL  [initandlisten] **              numactl --interleave=all mongod [other options]
2018-11-16T13:55:02.275-0500 I CONTROL  [initandlisten] 
2018-11-16T13:55:02.275-0500 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always'.
2018-11-16T13:55:02.275-0500 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
2018-11-16T13:55:02.275-0500 I CONTROL  [initandlisten] 
2018-11-16T13:55:02.275-0500 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/defrag is 'always'.
2018-11-16T13:55:02.275-0500 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
2018-11-16T13:55:02.275-0500 I CONTROL  [initandlisten] 
> show dbs
admin  0.000GB
local  0.000GB

Verified that installer does delete mongo database (pulp_database).  Verified on snap 6.4.1_1.0

Comment 15 errata-xmlrpc 2018-12-06 22:32:53 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:3799


Note You need to log in before you can comment on or make changes to this bug.