Description of problem: katello-remove does not completely remove the data on mounted directory structures Version-Release number of selected component (if applicable): Satellite 6.3.1 RHEL 7.5 How reproducible: Always Steps to Reproduce: 1. Install a new RHEL host with mounted filesystems following guidelines [1]: /var/lib/pgsql /var/lib/mongodb 2. Install Satellite server bits # yum install satellite # satellite-installer 3. Remove [2] Satellite 6.3 server bits: # katello-remove 4. Change hostname to something else # hostnamectl set-hostname satname2 5. Install Satellite server bits again # yum install satellite # satellite-installer Actual results: Since the data does not remove completely, remnants of the old install still stick around, for example, in this scenario, the hostname was changed during the second installation and the capsule id is now "2" instead of "1" for a fresh installation. /var/lib/pgsql was not cleaned up properly so the new installation took the data from this directory and used it for Satellite 6 Expected results: katello-remove should remove *all* the data in these directories Maybe add some logic to check if the filesystem is mounted, and ask the user if it's OK to remove the data in that path. We see that /var/lib/pgsql is a mounted filesystem, is it OK to remove /var/lib/pgsql/* ? Additional info: [1] https://access.redhat.com/documentation/en-us/red_hat_satellite/6.3/html-single/installation_guide/#hardware_storage_prerequisites [2] https://access.redhat.com/documentation/en-us/red_hat_satellite/6.3/html-single/installation_guide/#uninstalling_satellite_and_capsule_servers
After running katello-remove ... [root@sat6test pgsql]# df -k . Filesystem 1K-blocks Used Available Use% Mounted on /dev/mapper/pgsql-pgsql 10471424 120560 10350864 2% /var/lib/pgsql [root@sat6test pgsql]# pwd /var/lib/pgsql [root@sat6test pgsql]# ls data [root@sat6test pgsql]# cd data [root@sat6test data]# ls base pg_hba.conf pg_multixact pg_snapshots pg_tblspc pg_xlog global pg_ident.conf pg_notify pg_stat_tmp pg_twophase postgresql.conf pg_clog pg_log pg_serial pg_subtrans PG_VERSION postmaster.opts [root@sat6test data]# cd /var/lib/mongodb/ [root@sat6test mongodb]# ls journal local.0 local.ns mongod.lock pulp_database.0 pulp_database.ns [root@sat6test mongodb]# df -k . Filesystem 1K-blocks Used Available Use% Mounted on /dev/mapper/mongo-mongo 52399108 3342552 49056556 7% /var/lib/mongodb [root@sat6test mongodb]#
While attempting to refresh the manifest with the Satellite server in this state, we encounter this error as both are claiming to be the "default/local" capsule. "There was an issue with the backend service pulp: Couldn't find SmartProxy with 'id'=1"
Created attachment 1430370 [details] two default/local capsules
Note: when addressing this bugzilla, the solution should not destroy all databases on the server. Instead, it should only be those that are used by Satellite (e.g. foreman, candlepin...etc).
VERIFIED. @satellite-6.5.0-5.beta.el7sat.noarch katello-common-3.10.0-0.6.rc1.el7sat.noarch by the manual reproducer: 1) Set up a machine to have these mountpoints: # cat /etc/fstab ... /dev/mapper/machine01-pgsql /var/lib/pgsql xfs defaults 0 0 /dev/mapper/machine01-mongo /var/lib/mongodb xfs defaults 0 0 2) Install Satellite 3) Run katello-remove # katello-remove WARNING: This script will erase many packages and config files. Important packages such as the following will be removed: * httpd (apache) * mongodb * tomcat * puppet * ruby * rubygems * All Katello and Foreman Packages Once these packages and configuration files are removed there is no going back. If you use this system for anything other than Katello and Foreman you probably do not want to execute this script. Read the source at /usr/sbin/katello-remove for a list of what is removed. Are you sure(Y/N) y ARE YOU SURE?: This script permanently deletes data and configurations. Type [remove] to continue remove Starting removal Stopping services Removing RPMs Cleaning up configuration files Cleaning up log files Cleaning up certs Cleaning up content Removing Databases and Pulp storage on Filesystem /var/lib/mongodb is part of a mounted filesystem, skipping. You will need to remove /var/lib/mongodb manually. /var/lib/pgsql is part of a mounted filesystem, skipping. You will need to remove /var/lib/pgsql manually. Finished >>> katello-remove issues appropriate warnings about mountpoints to be deleted manually
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2019:1222