Bug 141481

Summary: Yum results with ~20,000 rpms in a repo.
Product: [Fedora] Fedora Reporter: taj <taj>
Component: yumAssignee: Jeremy Katz <katzj>
Status: CLOSED NOTABUG QA Contact:
Severity: medium Docs Contact:
Priority: medium    
Version: rawhideCC: katzj
Target Milestone: ---Keywords: FutureFeature
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2005-02-13 22:02:14 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description taj 2004-12-01 14:58:27 UTC
Just some observations with ~20,000 rpms in a repo created by accident
 for information purposes only

yum-2.1.11-3 python 2.4-0.c1.1 + some grail water to get them working.

On a laptop with 580megs ram.

$ yum check-update
Setting up Repo:  laptop
repomd.xml                100% |=========================|  951 B    00:00
Reading repository metadata in from local files
primary.xml.gz            100% |=========================| 6.6 MB    00:10
MD Read   : ################################################# 
24666/24668Traceback (most recent call last):
  File "/usr/bin/yum", line 8, in ?
    yummain.main(sys.argv[1:])
  File "/usr/share/yum-cli/yummain.py", line 68, in main
    result, resultmsgs = base.doCommands()
  File "/usr/share/yum-cli/cli.py", line 428, in doCommands
    ypl = self.returnPkgLists()
  File "/usr/share/yum-cli/cli.py", line 954, in returnPkgLists
    ypl = self.doPackageLists(pkgnarrow=pkgnarrow)
  File "/usr/lib/python2.4/site-packages/yum/__init__.py", line 631,
in doPackageLists
  File "/usr/share/yum-cli/cli.py", line 78, in doRepoSetup
    self.doSackSetup()
  File "/usr/lib/python2.4/site-packages/yum/__init__.py", line 93, in
doSackSetup
  File "/usr/lib/python2.4/site-packages/yum/repos.py", line 203, in
populateSack
  File "/usr/lib/python2.4/site-packages/yum/mdcache.py", line 103, in
getPrimary
  File "/usr/lib/python2.4/site-packages/yum/mdcache.py", line 97, in
_getGeneric
  File "/usr/lib/python2.4/site-packages/yum/mdcache.py", line 53, in
_pickle
MemoryError
[<LAPPY - root>] $ free
             total       used       free     shared    buffers     cached
Mem:        580640     159860     420780          0       6432      61476
-/+ buffers/cache:      91952     488688
Swap:            0          0          0

On a larger machine typical top results:

top - 08:14:46 up 256 days,  3:32, 18 users,  load average: 0.95,
0.42, 0.22
Tasks: 228 total,   3 running, 224 sleeping,   0 stopped,   1 zombie
Cpu(s):  50.0% user,   1.6% system,   0.0% nice,  47.6% idle
Mem:   4016832k total,  4007232k used,     9600k free,   138148k buffers
Swap: 11727432k total,   122580k used, 11604852k free,  1977856k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM   TIME #C COMMAND
16879 root      25   0  780m 780m 4224 R 99.9 19.9   2:17  1 yum

.. success with the 4 gigs ram.  Though it appears yum only uses one
of the CPUs

Comment 1 Seth Vidal 2004-12-01 15:08:17 UTC
yah, umm,
Don't do that.

Don't have 20000 rpms in a single repo.

That's a bad idea.

Comment 2 taj 2004-12-01 17:15:43 UTC
more info for kicks.

[root@localhost repodata]# gunzip *.gz
[root@localhost repodata]# ls -l
total 977400
-rw-r--r--  1 root root 439704026 Dec  1 07:38 filelists.xml
-rw-r--r--  1 root root 495450492 Dec  1 07:38 other.xml
-rw-r--r--  1 root root  64696881 Dec  1 07:38 primary.xml
-rw-r--r--  1 root root       951 Dec  1 07:38 repomd.xml