Bug 1717731

Summary: katello-remove failed to remove '/var/cache/pulp' because it is a mounted directory.
Product: Red Hat Satellite Reporter: Hao Chang Yu <hyu>
Component: InstallationAssignee: Chris Roberts <chrobert>
Status: CLOSED ERRATA QA Contact: Devendra Singh <desingh>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 6.5.0CC: ableisch, bkearney, chrobert, egolov, jerry_d_williams, patalber
Target Milestone: 6.6.0Keywords: Triaged
Target Release: Unused   
Hardware: Unspecified   
OS: Unspecified   
URL: https://projects.theforeman.org/issues/27186
Whiteboard:
Fixed In Version: katello-3.12.0-2 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1796615 (view as bug list) Environment:
Last Closed: 2019-10-22 19:51:46 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1796615    

Description Hao Chang Yu 2019-06-06 03:21:12 UTC
Description of problem:

# katello-remove -y

Starting removal

Stopping services

Removing RPMs

Cleaning up configuration files

Cleaning up log files

Cleaning up certs

Cleaning up content
/usr/share/ruby/fileutils.rb:1433:in `rmdir': Device or resource busy - /var/cache/pulp (Errno::EBUSY)
        from /usr/share/ruby/fileutils.rb:1433:in `block in remove_dir1'
        from /usr/share/ruby/fileutils.rb:1444:in `platform_support'
        from /usr/share/ruby/fileutils.rb:1432:in `remove_dir1'
        from /usr/share/ruby/fileutils.rb:1425:in `remove'
        from /usr/share/ruby/fileutils.rb:770:in `block in remove_entry'
        from /usr/share/ruby/fileutils.rb:1477:in `block (2 levels) in postorder_traverse'
        from /usr/share/ruby/fileutils.rb:1481:in `postorder_traverse'
        from /usr/share/ruby/fileutils.rb:1476:in `block in postorder_traverse'
        from /usr/share/ruby/fileutils.rb:1475:in `each'
        from /usr/share/ruby/fileutils.rb:1475:in `postorder_traverse'
        from /usr/share/ruby/fileutils.rb:768:in `remove_entry'
        from /usr/share/ruby/fileutils.rb:626:in `block in rm_r'
        from /usr/share/ruby/fileutils.rb:622:in `each'
        from /usr/share/ruby/fileutils.rb:622:in `rm_r'
        from /sbin/katello-remove:188:in `remove'
        from /sbin/katello-remove:231:in `<main>'


This following bugzilla doesn't fix the issue completely.

# katello-remove does not completely remove data on mounted filesystems
https://bugzilla.redhat.com/show_bug.cgi?id=1574257


I think the issue is in the codes below. It only checks whether "/var/cache" is a mounted directory or not, but it doesn't take the sub directories of "/var/cache/" into consideration.

# katello-remove
------------------------------
<snip>

CONTENT.map! do |dir|
  if File.readlines('/proc/mounts').any? { |line| line.split(' ')[1] =~ /#{dir}$/ }
    dir + '/*'
  else
    dir
  end
end

<snip>
-----------------------------

I think what it should do are:
1) Loop into the subdirectories of "/var/cache".
2) If any sub directory is a mounted directory then append it to the MOUNTS array.
3) If no mounted sub directories found in it then simply rm "/var/cache/*"

Comment 5 Jerry Williams 2019-06-07 14:40:38 UTC
There are more problems that just "/var/cache".
If for example you have a mounted file system "/var/log/candlepin" or "/var/log/httpd" it aborts and doesn't finish the cleanup and at that point katello-remove is also gone, so the only option to run it again is to reinstall katello-common.

It also might be nice to have an option like --real-clean, that would remove all the rpms in rhel-7-server-satellite-6*, or rhel-7-server-satellite-maintenance-6-rpms or rhel-server-rhscl-7-rpms.  Or even remove anything that isn't @base for the OS. --super-clean.

Comment 7 Devendra Singh 2019-08-30 14:17:57 UTC
This bug has successfully verified in 6.6.0 Snap17.

Katello-remove command executed successfully.
# katello-remove -y

Starting removal

Stopping services

Removing RPMs

Cleaning up configuration files

Cleaning up log files

Cleaning up certs

Cleaning up content

Finished

Comment 8 Bryan Kearney 2019-10-22 19:51:46 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2019:3172

Comment 9 patalber 2020-01-27 16:23:25 UTC
Hi Team,

I have a customer that wishes for this to be re-opened and a few code changes made which worked for him and may work for others. I will upload his code to this bugzilla.

Thank you.

--Patrick

Comment 11 Bryan Kearney 2020-01-27 20:08:57 UTC
Can you please open a new bug with this requests. Once the bug has been released via an errtaa, we can not re-open it.

-- bk