Bug 344301

Summary:	Anaconda use of FTP pessimal when downloading headers to /tmp/cache
Product:	[Fedora] Fedora	Reporter:	Pekka Pietikäinen <pp>
Component:	anaconda	Assignee:	Joel Andres Granados <jgranado>
Status:	CLOSED INSUFFICIENT_DATA	QA Contact:	Fedora Extras Quality Assurance <extras-qa>
Severity:	low	Docs Contact:
Priority:	low
Version:	9	CC:	alindebe, cra, triage
Target Milestone:	---
Target Release:	---
Hardware:	All
OS:	Linux
Whiteboard:	bzcl34nup
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2008-08-04 14:22:31 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	235705

Description Pekka Pietikäinen 2007-10-21 11:21:24 UTC

Description of problem:

When wondering why anaconda was taking quite some time doing a remote ftp
install I started stracing it and noticed it was doing quite silly things for
grabbing the headers. Basically for every file it does a new ftp command
session, then traversed the server one directory at a time (USER anonymous OK
PASSWORD foo OK CWD pub OK CWD os OK CWD linux OK etc.). Quite a bit of latency,
it's being doing that for half an hour or so and it's downloaded 40 MB (1.2MB/s
from the same ftp server doing bulk transfers ;) )!

Fortunately it works, but it's something that might be possible to optimize for
F9. Could be python-urlgrabber or even ftplib from python that's the real culprit.

Comment 1 Bug Zapper 2008-04-04 14:14:07 UTC

Based on the date this bug was created, it appears to have been reported
during the development of Fedora 8. In order to refocus our efforts as
a project we are changing the version of this bug to '8'.

If this bug still exists in rawhide, please change the version back to
rawhide.
(If you're unable to change the bug's version, add a comment to the bug
and someone will change it for you.)

Thanks for your help and we apologize for the interruption.

The process we're following is outlined here:
http://fedoraproject.org/wiki/BugZappers/F9CleanUp

We will be following the process here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping to ensure this
doesn't happen again.

Comment 2 Pekka Pietikäinen 2008-04-22 18:23:19 UTC

Reconfirmed on rawhide 20080422. From what I could gather from the code it's
urlgrabber (and python ftplib under it) that's the real culprit, but it's
anaconda that's using it in a way that makes performance hurt badly in the "get
headers for all rpms" phase :) . I'd probably make it use pycurl underneath, but
that's another can of worms and AFAIK the yum/urlgrabber people aren't fond of
the idea at all.

Comment 3 Bug Zapper 2008-05-14 03:41:26 UTC

Changing version to '9' as part of upcoming Fedora 9 GA.
More information and reason for this action is here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

Comment 4 Andy Lindeberg 2008-06-02 20:11:57 UTC

Does this persist in F9?