Red Hat Bugzilla – Bug 344301
Anaconda use of FTP pessimal when downloading headers to /tmp/cache
Last modified: 2008-08-04 10:22:31 EDT
Description of problem:
When wondering why anaconda was taking quite some time doing a remote ftp
install I started stracing it and noticed it was doing quite silly things for
grabbing the headers. Basically for every file it does a new ftp command
session, then traversed the server one directory at a time (USER anonymous OK
PASSWORD foo OK CWD pub OK CWD os OK CWD linux OK etc.). Quite a bit of latency,
it's being doing that for half an hour or so and it's downloaded 40 MB (1.2MB/s
from the same ftp server doing bulk transfers ;) )!
Fortunately it works, but it's something that might be possible to optimize for
F9. Could be python-urlgrabber or even ftplib from python that's the real culprit.
Based on the date this bug was created, it appears to have been reported
during the development of Fedora 8. In order to refocus our efforts as
a project we are changing the version of this bug to '8'.
If this bug still exists in rawhide, please change the version back to
(If you're unable to change the bug's version, add a comment to the bug
and someone will change it for you.)
Thanks for your help and we apologize for the interruption.
The process we're following is outlined here:
We will be following the process here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping to ensure this
doesn't happen again.
Reconfirmed on rawhide 20080422. From what I could gather from the code it's
urlgrabber (and python ftplib under it) that's the real culprit, but it's
anaconda that's using it in a way that makes performance hurt badly in the "get
headers for all rpms" phase :) . I'd probably make it use pycurl underneath, but
that's another can of worms and AFAIK the yum/urlgrabber people aren't fond of
the idea at all.
Changing version to '9' as part of upcoming Fedora 9 GA.
More information and reason for this action is here:
Does this persist in F9?