Bug 344301

Summary: Anaconda use of FTP pessimal when downloading headers to /tmp/cache
Product: [Fedora] Fedora Reporter: Pekka Pietikäinen <pp>
Component: anacondaAssignee: Joel Andres Granados <jgranado>
Status: CLOSED INSUFFICIENT_DATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: low Docs Contact:
Priority: low    
Version: 9CC: alindebe, cra, triage
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard: bzcl34nup
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2008-08-04 10:22:31 EDT Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
Bug Depends On:    
Bug Blocks: 235705    

Description Pekka Pietikäinen 2007-10-21 07:21:24 EDT
Description of problem:

When wondering why anaconda was taking quite some time doing a remote ftp
install I started stracing it and noticed it was doing quite silly things for
grabbing the headers. Basically for every file it does a new ftp command
session, then traversed the server one directory at a time (USER anonymous OK
PASSWORD foo OK CWD pub OK CWD os OK CWD linux OK etc.). Quite a bit of latency,
it's being doing that for half an hour or so and it's downloaded 40 MB (1.2MB/s
from the same ftp server doing bulk transfers ;) )!

Fortunately it works, but it's something that might be possible to optimize for
F9. Could be python-urlgrabber or even ftplib from python that's the real culprit.
Comment 1 Bug Zapper 2008-04-04 10:14:07 EDT
Based on the date this bug was created, it appears to have been reported
during the development of Fedora 8. In order to refocus our efforts as
a project we are changing the version of this bug to '8'.

If this bug still exists in rawhide, please change the version back to
(If you're unable to change the bug's version, add a comment to the bug
and someone will change it for you.)

Thanks for your help and we apologize for the interruption.

The process we're following is outlined here:

We will be following the process here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping to ensure this
doesn't happen again.
Comment 2 Pekka Pietikäinen 2008-04-22 14:23:19 EDT
Reconfirmed on rawhide 20080422. From what I could gather from the code it's
urlgrabber (and python ftplib under it) that's the real culprit, but it's
anaconda that's using it in a way that makes performance hurt badly in the "get
headers for all rpms" phase :) . I'd probably make it use pycurl underneath, but
that's another can of worms and AFAIK the yum/urlgrabber people aren't fond of
the idea at all.
Comment 3 Bug Zapper 2008-05-13 23:41:26 EDT
Changing version to '9' as part of upcoming Fedora 9 GA.
More information and reason for this action is here:
Comment 4 Andy Lindeberg 2008-06-02 16:11:57 EDT
Does this persist in F9?