+++ This bug was initially created as a clone of Bug #628963 +++
Description of problem:
on all our RHEL5 servers, /var/cache/yum fills up with sqlite files. We haven't performed any recent yum request or command, by hand or automatically (cron for example). On every day, two sqlite files are added, around 25 MB in size. the filenames are unique. An example:
/var/cache/yum/rhel-i386-server-5# ls -ltrh
drwxr-xr-x 2 root root 1.0K Jun 27 2008 headers
-rw-r--r-- 1 root root 25M Aug 11 17:24 884e1ab88b7ff9d73cb753b0ecf8a6b64d186f0d-primary.xml.gz.sqlite
-rw-r--r-- 1 root root 25M Aug 12 01:24 c0177689c2bd3e5f4d7d0bb83dbe6d1ff85a78e7-primary.xml.gz.sqlite
-rw-r--r-- 1 root root 25M Aug 12 09:24 87b643feda4ecb2501a148be00b8ca898fa5e61a-primary.xml.gz.sqlite
-rw-r--r-- 1 root root 25M Aug 13 17:02 4e16200c14addb78d8a990ab72451bf6e7dbb04e-primary.xml.gz.sqlite
-rw-r--r-- 1 root root 25M Aug 23 10:20 c06e93c323308c3b5eec5a2ece6f0e2d0282e5cf-primary.xml.gz.sqlite
-rw-r--r-- 1 root root 25M Aug 24 02:20 b0dedffd33caaad5db7489e97f3a286bdf7a4b7d-primary.xml.gz.sqlite
-rw-r--r-- 1 root root 25M Aug 24 10:20 4c16cc09c7cce4d0a6cdd4616483bce69f0058a1-primary.xml.gz.sqlite
-rw-r--r-- 1 root root 25M Aug 25 03:09 e4d34c13365eb035521e63bbb74a7d8b0540b875-primary.xml.gz.sqlite
-rw-r--r-- 1 root root 25M Aug 25 19:10 4bde5ac9c9c5c1e2a0e96d5a97b7672ab75d436b-primary.xml.gz.sqlite
drwxr-xr-x 2 root root 6.0K Aug 30 10:42 packages
-rw-r--r-- 1 root root 1.5K Aug 30 15:38 repomd.xml
-rw-r--r-- 1 root root 3.0M Aug 30 15:38 4686da25d73a218058eaf57d81acfdeb22a27de1-primary.xml.gz
-rw-r--r-- 1 root root 25M Aug 30 18:54 4686da25d73a218058eaf57d81acfdeb22a27de1-primary.xml.gz.sqlite
-rw-r--r-- 1 root root 0 Aug 31 10:54 cachecookie
This happens on both recently updated and newly installed servers, as on servers that haven't been updated a while ago. This behaviour makes me wonder whether it's a local yum or yum plugin-problem, or something that is initiated from the RHN, through yum-updatesd since a few weeks.
Version-Release number of selected component (if applicable):
Red Hat Enterprise Linux Server release 5.5 (Tikanga)
on all servers, just sit and wait and you should see about 1, 2 or more unique xml.gz.sqlite files appear on every day.
Steps to Reproduce:
1. Install a new RHEL5.5 or take any existing RHEL 5.5 or RHEL5.4 installation
2. register at RHN
3. sit back and look in /var/cache/yum after a few days
unique -primary.xml.gz.sqlite files (around 25 MB) appear every day in /var/cache/yum, sometimes 1, 2 or more per day.
I don't know the previous behaviour, but no more than two of those primary.xml.gz.sqlite files should be present. The previous one and the one that is being updated before the previous one is replaced.
if needed, contact me by e-mail.
--- Additional comment from email@example.com on 2010-08-31 10:51:05 EDT ---
Do you have yum-updatesd running? and/or the rhn auto-check?
B/c that would cause it to grab a new one each time the repo updates.
if you run: yum clean metadata
does it clean them all up?
--- Additional comment from firstname.lastname@example.org on 2010-08-31 16:10:27 EDT ---
This issue has just started on one of my RHEL5 boxes. Will be checking the others.
A clean metadata works for me
--- Additional comment from email@example.com on 2010-08-31 17:16:38 EDT ---
I believe this started when the CDN support was added. clean dbcache takes care of it but otherwise they seem to just accumulate now. If I remember correctly before the CDN changes the sqlite files were updated in place (using the same filename anyway). Thanks for looking into it.
--- Additional comment from firstname.lastname@example.org on 2010-09-01 03:33:28 EDT ---
@ seth vidal:
both services (yum-updatesd and rhnsd) are running:
# chkconfig --list yum-updatesd
yum-updatesd 0:off 1:off 2:off 3:on 4:on 5:on 6:off
# chkconfig --list rhnsd
rhnsd 0:off 1:off 2:on 3:on 4:on 5:on 6:off
and the command: yum clean metadata does indeed clean all the sqlite files up:
/var/cache/yum/rhel-i386-server-5# ls -l
/var/cache/yum/rhel-i386-server-5# yum clean metadata
Loaded plugins: rhnplugin
8 metadata files removed
30 sqlite files removed
0 metadata files removed
/var/cache/yum/rhel-i386-server-5# ls -l
drwxr-xr-x 2 root root 4096 Aug 30 10:38 packages
--- Additional comment from email@example.com on 2010-09-08 13:33:03 EDT ---
Yes, this appears to be a bug even in the latest upstream yum. With XML only repodata + unique filenames, yum doesn't delete the generated .sqlite.
--- Additional comment from firstname.lastname@example.org on 2010-09-09 16:05:28 EDT ---
Ok, I have a patch for this ... much simpler than I feared.
I haven't tested this, but as far as I know we should hit the same problem on RHEL-6 (RHN generating unique filenames, but giving us .xml not .sqlite).
It'll get fixed in 6.1, with any rebase. But depending on how much the MD changes between 6.0 => 6.1, people might be unhappy (esp. small virts with small /var/cache).
I can't mark it as 6.0.z, so marking it for 6.0.0 (although I realize that's "unlikely").
Thank you for your bug report. This issue was evaluated for inclusion
in the current release of Red Hat Enterprise Linux. Unfortunately, we
are unable to address this request in the current release. Because we
are in the final stage of Red Hat Enterprise Linux 6 development, only
significant, release-blocking issues involving serious regressions and
data corruption can be considered.
If you believe this issue meets the release blocking criteria as
defined and communicated to you by your Red Hat Support representative,
please ask your representative to file this issue as a blocker for the
current release. Otherwise, ask that it be evaluated for inclusion in
the next minor release of Red Hat Enterprise Linux.
Technical note added. If any revisions are required, please edit the "Technical Notes" field
accordingly. All revisions will be proofread by the Engineering Content Services team.
Cause: yum didn't remove old generated .sqlite files
Consequence: when using unique filenames /var/cache/yum would grow a _lot_
Fix: remove old generated filenames
Result: cache directory didn't grow without bound
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.