Bug 1211255 - repositories with URLs without variables leave versioned cache dirs behind
Summary: repositories with URLs without variables leave versioned cache dirs behind
Status: CLOSED NEXTRELEASE
Alias: None
Product: Fedora
Classification: Fedora
Component: dnf
Version: 23
Hardware: Unspecified
OS: Unspecified
low
unspecified
Target Milestone: ---
Assignee: Michael Mráka
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Keywords:
: 1211252 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-04-13 12:42 UTC by Vít Ondruch
Modified: 2015-07-21 12:37 UTC (History)
9 users (show)

(edit)
Clone Of: 1173107
(edit)
Last Closed: 2015-07-21 12:37:00 UTC


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Bugzilla 1173107 None None None Never

Internal Trackers: 1173107

Description Vít Ondruch 2015-04-13 12:42:31 UTC
+++ This bug was initially created as a clone of Bug #1173107 +++

> 4. How does it look for Rawhide?
> 
> $ ls -l /var/cache/dnf/x86_64/
> celkem 12
> drwxr-xr-x. 2 root root 4096 22. říj 15.31 21
> drwxr-xr-x. 5 root root 4096 26. úno 12.42 22
> drwxr-xr-x. 3 root root 4096 26. úno 12.51 23
> 
> Ah, 3 cache folders and they will just grow, but the repository was always
> the same:
> 
> $ cat /etc/yum.repos.d/fedora-rawhide.repo | grep repo=rawhide\&
> metalink=https://mirrors.fedoraproject.org/
> metalink?repo=rawhide&arch=$basearch
> 
> So why there are still versioned cache directories? This wastes precious
> space on my test virtual machine :/

Yes, this may be a problem. Please, file a bug if you can.

Comment 1 Radek Holy 2015-04-13 13:14:00 UTC
OK, looking at the summary it seems that I might have misunderstood you. A thought that the problem is that if you have a repository that does not contain e.g. a $releasever variable, DNF downloads (and caches) the metadata for every different "--releasever" value if if it's unnecessary.

Is you problem related only to rawhide? Why shouldn't DNF cache the rawhide metadata?

Comment 2 Radek Holy 2015-04-13 13:15:02 UTC
*I thought
*even if it's unnecessary

Comment 3 Vít Ondruch 2015-04-13 13:25:14 UTC
The thing is that the $releasever changes during lifetime of rawhide. So you see that at 26th of February, it changed from 22 to 23. The cache was still valid. The content was still valid. The upstream repositoried did not changed. The $releasever is not used anywhere in the /etc/yum.repos.d/fedora-rawhide.repo but yet the cache become invalid.

The DNF actually caches the data for "rawhide" repository in "rawhide" directory, but this repository specific directory is stored under $releasever directory. This is the whole content of my cache directory on Rawhide:

$ find /var/cache/dnf/x86_64/ -type d
/var/cache/dnf/x86_64/
/var/cache/dnf/x86_64/23
/var/cache/dnf/x86_64/23/x86_64
/var/cache/dnf/x86_64/23/x86_64/23
/var/cache/dnf/x86_64/23/x86_64/23/vondruch-doublecmd
/var/cache/dnf/x86_64/23/x86_64/23/vondruch-doublecmd/repodata
/var/cache/dnf/x86_64/23/x86_64/23/vondruch-doublecmd/packages
/var/cache/dnf/x86_64/23/x86_64/23/rawhide
/var/cache/dnf/x86_64/23/x86_64/23/rawhide/repodata
/var/cache/dnf/x86_64/23/x86_64/23/rawhide/packages
/var/cache/dnf/x86_64/22
/var/cache/dnf/x86_64/22/vondruch-doublecmd
/var/cache/dnf/x86_64/22/vondruch-doublecmd/repodata
/var/cache/dnf/x86_64/22/vondruch-doublecmd/packages
/var/cache/dnf/x86_64/22/x86_64
/var/cache/dnf/x86_64/22/x86_64/22
/var/cache/dnf/x86_64/22/x86_64/22/vondruch-doublecmd
/var/cache/dnf/x86_64/22/x86_64/22/vondruch-doublecmd/repodata
/var/cache/dnf/x86_64/22/x86_64/22/rawhide
/var/cache/dnf/x86_64/22/x86_64/22/rawhide/repodata
/var/cache/dnf/x86_64/22/rawhide
/var/cache/dnf/x86_64/22/rawhide/repodata
/var/cache/dnf/x86_64/22/rawhide/packages
/var/cache/dnf/x86_64/21

Comment 4 Radek Holy 2015-04-13 13:40:33 UTC
Oh, I see. OK, this is valid. This would require a substantial change in the DNF's caching mechanism.

For the further consideration: On one side, there are repositories where we want to preserve cache for all $releasevers (e.g. Fedora), on the other side, there are repositories where we might not want this (e.g. Rawhide).

Comment 5 Radek Holy 2015-04-13 13:49:43 UTC
Heh, well, it's fairly simple. The repositories, where we want the "versioned cache", are those that contain some variables. Otherwise, we don't want the cache to be "versioned" :-)

Comment 6 Michael Mráka 2015-04-13 13:53:52 UTC
Wouldn't be easier to make cache directories based on repo url hash (after variable expansion)? If name changes after variable substitution it leads into a different hash and the other way round if url stays unchanged even after substitution hash won't change and cache will be reused.

Comment 7 Radek Holy 2015-04-13 14:00:14 UTC
Yes, that might be one solution. I personally hate using hashes since they are not unique and (thus) they cannot be decoded which would make DNF more difficult to debug but that's just my opinion. On the other hand, base the cache on URLs is probably the best what we can do.

Comment 8 Jan Kurik 2015-07-15 14:17:09 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 23 development cycle.
Changing version to '23'.

(As we did not run this process for some time, it could affect also pre-Fedora 23 development
cycle bugs. We are very sorry. It will help us with cleanup during Fedora 23 End Of Life. Thank you.)

More information and reason for this action is here:
https://fedoraproject.org/wiki/BugZappers/HouseKeeping/Fedora23

Comment 9 Michael Mráka 2015-07-21 12:34:41 UTC
*** Bug 1211252 has been marked as a duplicate of this bug. ***

Comment 10 Michael Mráka 2015-07-21 12:37:00 UTC
Changes in https://github.com/rpm-software-management/dnf/pull/309
removes releasever from cache path.


Note You need to log in before you can comment on or make changes to this bug.