I ran createrepo on a file list with 30k RPMs.
It failed with :
Traceback (most recent call last):
File "/usr/share/createrepo/genpkgmetadata.py", line 278, in <module>
File "/usr/share/createrepo/genpkgmetadata.py", line 252, in main
File "/usr/lib/python2.7/site-packages/createrepo/__init__.py", line 396, in doPkgMetadata
File "/usr/lib/python2.7/site-packages/createrepo/__init__.py", line 631, in writeMetadataDocs
File "/usr/lib64/python2.7/subprocess.py", line 672, in __init__
File "/usr/lib64/python2.7/subprocess.py", line 1202, in _execute_child
OSError: [Errno 7] Argument list too long
The traceback wasn't actually thrown, I added 'raise' to createrepo code to see what happened there.
Wouldn't be possible to feed /usr/share/createrepo/worker.py in a different way?
It could read file list from stdin, for example.
I can probably make a tempfile per worker and let them read the pkgs out of that. It means less back-and-forth via stdin and less likelihood for the pipe interface of python to do something horrible.
I'll have a patch today - do you think you can test it on your 30K pkgs case to see if it makes everything happy?
patch push upstream
(In reply to comment #1)
> I'll have a patch today - do you think you can test it on your 30K pkgs case to
> see if it makes everything happy?
Thanks for the patch.
I'll test it either tomorrow morning or early next week.
It fails on undefined self._worker_tmp_path.
You should use worker_tmp_path instead.
btw, I'm getting only this message:
Spawning worker 0 with 29588 pkgs
Worker 0: reading packages/fakechroot/2.9/24.5.el6/i686/fakechroot-libs-2.9-24.5.el6.i686.rpm
Is it intentional? I'd expect all processed packages to be printed.
Createrepo seems to be stuck, but it's too early to confirm that. I'll send you more details later (or create a new bug since it looks like another issue).
Did you apply the patch ontop of what's in f16/rawhide or did you take the whole snapshot?
I see... I applied patch on the latest F16 build. That explains why it didn't work.
This message is a notice that Fedora 15 is now at end of life. Fedora
has stopped maintaining and issuing updates for Fedora 15. It is
Fedora's policy to close all bug reports from releases that are no
longer maintained. At this time, all open bugs with a Fedora 'version'
of '15' have been closed as WONTFIX.
(Please note: Our normal process is to give advanced warning of this
occurring, but we forgot to do that. A thousand apologies.)
Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, feel free to reopen
this bug and simply change the 'version' to a later Fedora version.
Bug Reporter: Thank you for reporting this issue and we are sorry that
we were unable to fix it before Fedora 15 reached end of life. If you
would still like to see this bug fixed and are able to reproduce it
against a later version of Fedora, you are encouraged to click on
"Clone This Bug" (top right of this page) and open it against that
version of Fedora.
Although we aim to fix as many bugs as possible during every release's
lifetime, sometimes those efforts are overtaken by events. Often a
more recent Fedora release includes newer upstream software that fixes
bugs or makes them obsolete.
The process we are following is described here: