Red Hat Bugzilla – Bug 201136
yum work increases exponentially after first download of primary.xml.gz into sqlite
Last modified: 2014-01-21 17:54:55 EST
Description of problem: Yum will work for several hours after unpacking primary.xml.gz and loading the sqlite database. This is for the development repository. After completing, the next repository is downloaded. The extras-development, although 30% larger, only takes a few minutes (less than 5). If I do a "rm -f /var/cache/yum/development/primary*", the next run of yum loads the repository it just a minute or two. But on the next run, the extremely long run reoccurs. The time has been increasing. A recently completed run took 516 minutes to run. Version-Release number of selected component (if applicable): yum-2.9.3-1 sqlite-3.3.6-1.1 python-sqlite-1.1.7-1.2.1 How reproducible: Always Steps to Reproduce: 1. rm -f /var/cache/yum/development/primary* 2. time yum update 3. Answer NO 4. rm -f /var/cache/yum/development/cachecookie 4. time yum update 5. Answer NO Actual results: First run will take 1 to 5 minutes. Second run will 2.5 to 4.5 hours to complete. Expected results: All runs should take only a few minutes. Additional info: The problem only occurs on a 300 MHz Pentium II notebook. On a 1.6GHz Pentium M notebook, the problen DOES NOT occur.
Another (important) data point. On this system, /var/cache/yum/development is mounted via NFS. No other directory under /var/cache/yum is mounted that way and they are free of the increased processing. So it's the merging of the sqlite database over NFS that is causing the slow down. A quick workaround is to delete the sqlite database before each run of yum: rm -f /var/cache/yum/development/primary* Worse comes to worse, I can change the NFS volume to the package directory.
sqlite and nfs being slow-ish would make sense.
My setup is that /var/cache/yum/development is a symlink to an NFS mount hierarchy. To work around this problem, I deleted the symlink and created the directory /var/cache/yum/development, and under that, I created two symlinks for headers and packages. This eliminates the 2 to 4 hour wait caused by sqlite.
NFS seems to be a red herring. I'm seeing the same issue, when /etc/yum.conf contains "cachedir=/usr/src/yum" and /usr/src is a local ext3 filesystem. The problem seems to be worse for yum-updatesd - yum itself usually manages to be not-so-piggy. The *first* run after removing *.sqlite usually chews about 45 seconds to a minute of CPU, the next few seem to take on the order of 5-10 minutes, and after 4-5 iterations, it's up in the hours range. Tools like vmstat and top and gkrellm indicate 95% and higher user-mode, only 2-3% system CPU, and almost no actual I/O. It's looking like a poor scaling algorithm somewhere inside sqlite - like it's inserting stuff into the DB based on a hash function, but it's a *poor* hash, so instead of choosing one of 10K hash buckets with short 2-3 entry linked lists, it's colliding every time and there's a 30K entry linked list hanging off one bucket. Just hypothesizing, but that's my guess.
sqlite 3.3.17 has some performance improvements - please check from latest in rawhide and see if it's still poor.
User pnasrat@redhat.com's account has been closed
Based on the date this bug was created, it appears to have been reported against rawhide during the development of a Fedora release that is no longer maintained. In order to refocus our efforts as a project we are flagging all of the open bugs for releases which are no longer maintained. If this bug remains in NEEDINFO thirty (30) days from now, we will automatically close it. If you can reproduce this bug in a maintained Fedora version (7, 8, or rawhide), please change this bug to the respective version and change the status to ASSIGNED. (If you're unable to change the bug's version or status, add a comment to the bug and someone will change it for you.) Thanks for your help, and we apologize again that we haven't handled these issues to this point. The process we're following is outlined here: http://fedoraproject.org/wiki/BugZappers/F9CleanUp We will be following the process here: http://fedoraproject.org/wiki/BugZappers/HouseKeeping to ensure this doesn't happen again.
This bug has been in NEEDINFO for more than 30 days since feedback was first requested. As a result we are closing it. If you can reproduce this bug in the future against a maintained Fedora version please feel free to reopen it against that version. The process we're following is outlined here: http://fedoraproject.org/wiki/BugZappers/F9CleanUp