Bug 1357083

Summary: yum clean all does clean up cache from earlier configured and not enabled now repositories
Product: Red Hat Enterprise Linux 7 Reporter: Steffen Froemer <sfroemer>
Component: yumAssignee: Michal Domonkos <mdomonko>
Status: CLOSED ERRATA QA Contact: Eva Mrakova <emrakova>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 7.2CC: emrakova, james.antill, mdomonko, vmukhame
Target Milestone: rcKeywords: Patch
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: yum-3.4.3-154.el7 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-08-01 09:07:52 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1380360, 1393866    

Description Steffen Froemer 2016-07-15 16:24:14 UTC
Description of problem:
'yum clean all' function will only remove cache from configured yum repositories.
The directory in /var/cache/yum will be completely ignored
Neither 'subscription-manager repos --disable <reponame>' nor 'yum-config-manager --disable <reponame>' will remove files in  /var/cache/yum/.

Version-Release number of selected component (if applicable):
yum-3.4.3-132.el7.noarch
yum-rhn-plugin-2.0.1-5.el7.noarch
yum-utils-1.1.31-34.el7.noarch
yum-langpacks-0.4.2-4.el7.noarch
yum-metadata-parser-1.1.4-10.el7.x86_64


How reproducible:
all time

Steps to Reproduce:

[root@newrhel7 ~]# yum makecache
Loaded plugins: langpacks, product-id, search-disabled-repos, subscription-
              : manager
rhel-7-server-eus-rpms                                   | 3.7 kB     00:00     
rhel-7-server-rpms                                       | 3.7 kB     00:00     
rhel-ha-for-rhel-7-server-eus-rpms                       | 3.1 kB     00:00     
rhel-rs-for-rhel-7-server-eus-rpms                       | 3.1 kB     00:00     
(1/11): rhel-7-server-eus-rpms/7Server/x86_64/filelists_db |  11 MB   00:05     
(2/11): rhel-7-server-eus-rpms/7Server/x86_64/other_db     | 3.0 MB   00:01     
(3/11): rhel-7-server-rpms/7Server/x86_64/primary_db       |  22 MB   00:19     
(4/11): rhel-7-server-rpms/7Server/x86_64/filelists_db     |  11 MB   00:20     
(5/11): rhel-ha-for-rhel-7-server-eus-rpms/7Server/x86_64/ | 174 kB   00:01     
(6/11): rhel-ha-for-rhel-7-server-eus-rpms/7Server/x86_64/ | 133 kB   00:01     
(7/11): rhel-7-server-rpms/7Server/x86_64/other_db         | 3.0 MB   00:02     
(8/11): rhel-ha-for-rhel-7-server-eus-rpms/7Server/x86_64/ |  64 kB   00:00     
(9/11): rhel-rs-for-rhel-7-server-eus-rpms/7Server/x86_64/ |  83 kB   00:00     
(10/11): rhel-rs-for-rhel-7-server-eus-rpms/7Server/x86_64 | 189 kB   00:01     
(11/11): rhel-7-server-eus-rpms/7Server/x86_64/primary_db  |  22 MB   00:29     
(1/9): rhel-7-server-eus-rpms/7Server/x86_64/updateinfo    | 177 kB   00:00     
(2/9): rhel-7-server-eus-rpms/7Server/x86_64/group_gz      | 134 kB   00:01     
(3/9): rhel-7-server-eus-rpms/7Server/x86_64/productid     | 1.7 kB   00:00     
(4/9): rhel-7-server-rpms/7Server/x86_64/group_gz          | 134 kB   00:00     
(5/9): rhel-7-server-rpms/7Server/x86_64/productid         | 1.7 kB   00:00     
(6/9): rhel-7-server-rpms/7Server/x86_64/updateinfo        | 1.2 MB   00:01     
(7/9): rhel-ha-for-rhel-7-server-eus-rpms/7Server/x86_64/p | 1.8 kB   00:01     
(8/9): rhel-rs-for-rhel-7-server-eus-rpms/7Server/x86_64/p | 1.8 kB   00:02     
(9/9): rhel-ha-for-rhel-7-server-eus-rpms/7Server/x86_64/u | 6.9 kB   00:02     
Metadata Cache Created


[root@newrhel7 7Server]# du -h -d 1 .
280M	./rhel-7-server-eus-rpms
286M	./rhel-7-server-rpms
4,5M	./rhel-ha-for-rhel-7-server-eus-rpms
4,9M	./rhel-rs-for-rhel-7-server-eus-rpms
575M	.

[root@newrhel7 ~]# subscription-manager repos --disable rhel-rs-for-rhel-7-server-eus-rpms
Repository 'rhel-rs-for-rhel-7-server-eus-rpms' is disabled for this system.


[root@newrhel7 ~]# yum clean all
Loaded plugins: langpacks, product-id, search-disabled-repos, subscription-
              : manager
Cleaning repos: rhel-7-server-eus-rpms rhel-7-server-rpms
              : rhel-ha-for-rhel-7-server-eus-rpms
Cleaning up everything


[root@newrhel7 7Server]# du -h -d 1 .
8,0K	./rhel-7-server-eus-rpms
8,0K	./rhel-7-server-rpms
8,0K	./rhel-ha-for-rhel-7-server-eus-rpms
4,9M	./rhel-rs-for-rhel-7-server-eus-rpms
5,0M	.


[root@newrhel7 ~]# yum repolist
Loaded plugins: langpacks, product-id, search-disabled-repos, subscription-manager
rhel-7-server-eus-rpms                                                              | 3.7 kB  00:00:00     
rhel-7-server-rpms                                                                  | 3.7 kB  00:00:00     
rhel-ha-for-rhel-7-server-eus-rpms                                                  | 3.1 kB  00:00:00     
(1/3): rhel-ha-for-rhel-7-server-eus-rpms/7Server/x86_64/primary_db                 | 133 kB  00:00:00     
(2/3): rhel-7-server-rpms/7Server/x86_64/primary_db                                 |  22 MB  00:00:18     
(3/3): rhel-7-server-eus-rpms/7Server/x86_64/primary_db                             |  22 MB  00:00:18     
(1/5): rhel-ha-for-rhel-7-server-eus-rpms/7Server/x86_64/updateinfo                 | 6.9 kB  00:00:01     
(2/5): rhel-7-server-eus-rpms/7Server/x86_64/group_gz                               | 134 kB  00:00:01     
(3/5): rhel-7-server-eus-rpms/7Server/x86_64/updateinfo                             | 177 kB  00:00:01     
(4/5): rhel-7-server-rpms/7Server/x86_64/group_gz                                   | 134 kB  00:00:01     
(5/5): rhel-7-server-rpms/7Server/x86_64/updateinfo                                 | 1.2 MB  00:00:02     
repo id                                           repo name                                          status
rhel-7-server-eus-rpms/7Server/x86_64             Red Hat Enterprise Linux 7 Server - Extended Updat 11.050
rhel-7-server-rpms/7Server/x86_64                 Red Hat Enterprise Linux 7 Server (RPMs)           11.059
rhel-ha-for-rhel-7-server-eus-rpms/7Server/x86_64 Red Hat Enterprise Linux High Availability (for RH    211
repolist: 22.320




Actual results:
The cache-directory of the disabled repository stays untouched.

Expected results:
The directory of the not configured repository should be removed. It should be removed on
- yum clean all -> all repo cache, for whom no configured repository exist, will be deleted
- subscription-manager repos --disable -> should remove the cache for this repo
- yum-config-manager --disable -> should remove the cache for this repo

Additional info:

Comment 2 Valentina Mukhamedzhanova 2016-07-18 10:38:13 UTC
The reported behaviour is not a bug and is documented in the man page:

       The  following are the ways which you can invoke yum in clean mode. Note that "all files"
       in the commands below means "all files in currently enabled repositories".  If  you  want
       to  also  clean  any (temporarily) disabled repositories you need to use --enablerepo='*'
       option.

As this request comes up quite often demonstrating that this behaviour is widely misunderstood, I think it's a good idea to show a hint to use '--enablerepo=\*' whenever a user runs 'yum clean all'.

Comment 4 Valentina Mukhamedzhanova 2016-07-18 10:45:04 UTC
> Expected results:
> The directory of the not configured repository should be removed. It should
> be removed on
> - yum clean all -> all repo cache, for whom no configured repository exist,
> will be deleted
> - subscription-manager repos --disable -> should remove the cache for this
> repo
> - yum-config-manager --disable -> should remove the cache for this repo

Users may temporarily disable and enable repos for various reasons, and many of them rely on the fact that the cache stays intact. Cleaning cache when disabling a repo would definitely be a regression for many users.

Comment 5 Steffen Froemer 2016-07-18 11:28:48 UTC
(In reply to Valentina Mukhamedzhanova from comment #2)
> 
> As this request comes up quite often demonstrating that this behaviour is
> widely misunderstood, I think it's a good idea to show a hint to use
> '--enablerepo=\*' whenever a user runs 'yum clean all'.

Do you have a proposal, who does this look like? How do we handle this bug (more RFE now)? 
But what is with data in /var/cache/yum, which are from a repository, of whom no configuration file exist nevermore. 

Example.
Customer create /etc/yum.repo.d/my.repo and synced metadata. After a while, the file will be removed. All data will stay on /var/cache/yum. Should we recognize such behaviour or just ignore it, since it's not the default manner on dealing with repos in yum.

Comment 6 Valentina Mukhamedzhanova 2016-07-18 11:45:19 UTC
> Do you have a proposal, who does this look like? How do we handle this bug
> (more RFE now)? 

We will add a helpful message which users will see whenever they run 'yum clean all'.

> But what is with data in /var/cache/yum, which are from a repository, of
> whom no configuration file exist nevermore. 
> 
> Example.
> Customer create /etc/yum.repo.d/my.repo and synced metadata. After a while,
> the file will be removed. All data will stay on /var/cache/yum. Should we
> recognize such behaviour or just ignore it, since it's not the default
> manner on dealing with repos in yum.

This use case is fairly rare, and it doesn't make much sense to me to implement a command which would exactly duplicate the functionality of 'rm -rf /var/cache/yum'. But we can show a message hint for that too.

Comment 7 Steffen Froemer 2016-07-18 19:54:30 UTC
Okay, for summary.

- A note will be displayed, if using 'yum clean all' that only enabled repos cleaned up and if I want to clean up cache from disabled repos, I have to use command line option '--enablerepo=*'
- if there are folders from repositories, they don't exist in any of the configurations in /etc/yum.repos.d/*.repo a hint will be displayed, that there are files left in /var/cache/yum, which are not automatically cleaned up.

Are you agree?

Comment 8 Steffen Froemer 2016-07-19 08:37:28 UTC
I must revert my last comment. 
I not agree with the statement, that remaining cache from not configured repos is fairly rare.
What arguments are against that 'yum clean all' also removes data in cache-directory, which are not needed? Yes, it will duplicate functionality of rm -rf /var/cache/yum, but on the other hand, if I use this command, I also remove cache from disabled repos. That's not want I want.

'yum clean all' should do following three steps by default:
- clean all cache from enabled repos
- preserve cache of disabled repos
- remove all data from unknown repos

-> additional give a hint, that disabled repos will not be cleaned up.

Comment 9 Steffen Froemer 2016-09-01 12:41:22 UTC
Can I get some information on this topic?

Comment 10 Valentina Mukhamedzhanova 2016-09-05 11:44:11 UTC
> What arguments are against that 'yum clean all' also removes data in
> cache-directory, which are not needed?

For example, when rhn-channel disables its repository - yum has no knowledge of it, which makes it look for yum as if the repository is not simply disabled, but nonexistent. But as described in comment 4, we don't want to clean the cache in this case.

Comment 11 Steffen Froemer 2016-09-06 05:36:21 UTC
(In reply to Valentina Mukhamedzhanova from comment #10)

> For example, when rhn-channel disables its repository - yum has no knowledge
> of it, which makes it look for yum as if the repository is not simply
> disabled, but nonexistent. But as described in comment 4, we don't want to
> clean the cache in this case.

In my eyes, unsubscribing and disabling are real different things. If I temporarily don't want to have access on a channel/repo, I should disable this. 
But if we don't can change this behavior, do you see a possibility to add a new cleanup-command like 'yum clean all-unknown-data' or something similar, which will do a 'yum clean all' additionally remove the unknown repo-data?
In the end, my Customer would like to have a command available, which will also cleanup cache-data from not existing repos. Any recommondation on this?

Comment 15 Michal Domonkos 2017-03-03 16:50:41 UTC
Let's make it simple for now by just printing this hint when you run "yum clean":

  Only enabled repos are being cleaned. Use together with --enablerepo/--disablerepo for more control.

Adding a hint about running "rm -rf /var/cache/yum" would probably make it too verbose.  While we could only print it in case there's some leftover data from non-existent repos, that would require us to walk the whole directory tree and potentially slow down the whole operation.

Comment 21 errata-xmlrpc 2017-08-01 09:07:52 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:2295