Bug 909744 - [rfe] support parallel downloads
[rfe] support parallel downloads
Status: CLOSED ERRATA
Product: Fedora
Classification: Fedora
Component: dnf (Show other bugs)
rawhide
Unspecified Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: Zdeněk Pavlas
Fedora Extras Quality Assurance
:
Depends On:
Blocks: 871892
  Show dependency treegraph
 
Reported: 2013-02-10 15:32 EST by Rahul Sundaram
Modified: 2014-02-02 17:29 EST (History)
5 users (show)

See Also:
Fixed In Version: dnf-0.4.2-1.gitc1716d7.fc20
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2013-10-09 10:20:38 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:


Attachments (Terms of Use)

  None (edit)
Description Rahul Sundaram 2013-02-10 15:32:23 EST
Description of problem:

yum (after the dnf fork) has added support for parallel downloads and other enhancements, bug fixes.  merge those changes.
Comment 1 Rahul Sundaram 2013-02-28 11:04:50 EST
While looking into this, you might want to incorporate some improvements from

http://repos.fedorapeople.org/repos/hedayat/yum-fast-downloader/

It downloads aggressively from multiple mirrors by using aria2 as a backend
Comment 2 Zdeněk Pavlas 2013-02-28 11:18:02 EST
There was a considerable backlash against using too many connections (downloading 2 files from the same mirror in parallel, or downloading 1 file from 2+ mirrors with range request).  People that run the mirrors were seriously concerned (and the parallel downloading code in Yum is not aggressive by any means)

http://lists.fedoraproject.org/pipermail/devel/2012-May/167122.html
Comment 3 Rahul Sundaram 2013-02-28 12:58:19 EST
From the thread, there isn't really much of a backlash.  There are some mirror admins who were concerned that yum is using more than one connection to the same mirror but yum fast downloader distributes this load.  Either way, yes, dnf developer would need  to talk to the mirror admins and figure out what is the optimal way to handle this.
Comment 4 Ales Kozumplik 2013-04-29 03:33:32 EDT
The DNF version in F19 and beyond uses librepo for the repodata and package download backend. By default librepo downloads different metadata files per repo in parallel. It currently however does not download repositories in parallel, nor packages in parallel. 

Careful extension of librepo API will have to happen first, leaving this open until then.
Comment 5 Ales Kozumplik 2013-06-27 10:12:07 EDT
(In reply to Ales Kozumplik from comment #4)
> Careful extension of librepo API will have to happen first, leaving this
> open until then.

Tomas, can librepo already do parallel download of packages?
Comment 6 Ales Kozumplik 2013-08-13 10:58:34 EDT
Moving over to Zdenek who implemented parallel downloads in Yum and is in touch with tmlcoch about how to do this with librepo.
Comment 7 Tomas Mlcoch 2013-09-17 08:48:59 EDT
Librepo since 1.0.0 supports parallel downloads out-of-box.
Comment 8 Fedora Update System 2013-10-01 09:51:56 EDT
dnf-0.4.2-1.gitc1716d7.fc20 has been submitted as an update for Fedora 20.
https://admin.fedoraproject.org/updates/dnf-0.4.2-1.gitc1716d7.fc20
Comment 9 Fedora Update System 2013-10-02 02:49:20 EDT
Package dnf-0.4.2-1.gitc1716d7.fc20, librepo-1.2.0-1.fc20, hawkey-0.4.2-1.git4c51f65.fc20:
* should fix your issue,
* was pushed to the Fedora 20 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing dnf-0.4.2-1.gitc1716d7.fc20 librepo-1.2.0-1.fc20 hawkey-0.4.2-1.git4c51f65.fc20'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2013-18123/librepo-1.2.0-1.fc20,hawkey-0.4.2-1.git4c51f65.fc20,dnf-0.4.2-1.gitc1716d7.fc20
then log in and leave karma (feedback).
Comment 10 Fedora Update System 2013-10-09 10:20:38 EDT
dnf-0.4.2-1.gitc1716d7.fc20, librepo-1.2.0-1.fc20, hawkey-0.4.2-1.git4c51f65.fc20 has been pushed to the Fedora 20 stable repository.  If problems still persist, please make note of it in this bug report.

Note You need to log in before you can comment on or make changes to this bug.