Hide Forgot
Description of problem: I understand that this process is memory intensive and uses lot of swap, but it should be able to handle atleast a couple processes with 4G's of total memory and a 1.5G of swap. sudo createrepo --database --checksum sha256 -g /var/lib/pulp//repos/content/dist/rhel/rhui/server-6/u Cannot access/write repodata filespdates/6Server/x86_64/os/repodata/comps-rhel-x86_64-server-6.xml /var/lib/pulp//repos/content/dist/rhel/rhui/server-6/updates/6Server/x86_64/os Spawning worker 0 with 5043 pkgs: [Errno 12] Cannot allocate memory Version-Release number of selected component (if applicable): 0.9.9-3
How many rpms do you have? How much memory are they using?
(In reply to comment #1) > How many rpms do you have? one repo has 5043 and other 3991 > > How much memory are they using? So it pretty much brings down the system to its knees before crashing and this has 4G os total ram with 1.5G of swap.
I've just done a couple of tests, with over 8,000 packages (7GB+ of size) in "c" a plain "createrepo c" (basically the same as your command, but no groups file) used at peak (on x86_64): main: VSZ=330,952 RSS=20,876 worker: VSZ=357,668 RSS=48,192 ...and then after the worker had finished, main gained a bit of memory: main: VSZ=350,756 RSS=41,804 ...so I find it hard to believe that a couple of this would be killing a 4GB box, without help. Obv. that doesn't mean it's doing the same thing for you :). Do you have any traces to see what's happening? Do you have a box I can login and reproduce it on?
are you using --update at all?
yes and I saw your patch on rpm-metadata list. Is it in raw hide yet? I assume that should fix this issue as well
I, too, am trying to track down a problem with createrepo using lots of memory (using --update), causing a VM with 8GB ram to thrash. What is this patch you speak of? Is there another bug report associated with it?
This is the main one: http://createrepo.baseurl.org/gitweb?p=createrepo.git;a=commitdiff;h=0a67bc57a9eda626735513a4015d8087f3f4bb29 ...but you need some yum changes too, so (by far) the easiest thing is to just take the rawhide creatrepo+yum (or rebuild rawhide creatrepo + use the yum rawhide rebuild repo). I might be able to get the new yum into F16, at which point a new creatrepo is fairly simple.
This message is a reminder that Fedora 16 is nearing its end of life. Approximately 4 (four) weeks from now Fedora will stop maintaining and issuing updates for Fedora 16. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as WONTFIX if it remains open with a Fedora 'version' of '16'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version prior to Fedora 16's end of life. Bug Reporter: Thank you for reporting this issue and we are sorry that we may not be able to fix it before Fedora 16 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged to click on "Clone This Bug" and open it against that version of Fedora. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete. The process we are following is described here: http://fedoraproject.org/wiki/BugZappers/HouseKeeping
Fedora 16 changed to end-of-life (EOL) status on 2013-02-12. Fedora 16 is no longer maintained, which means that it will not receive any further security or bug fix updates. As a result we are closing this bug. If you can reproduce this bug against a currently maintained version of Fedora please feel free to reopen this bug against that version. Thank you for reporting this bug and we are sorry it could not be fixed.