Description of problem: dnf check-update and dnf upgrade is very slow (50 - 150 KB/s), and sometimes it is normal speed (4.5 - 5.3 MB/s), but its more often slow than fast. I would need a way to tell DNF to always prefer HTTP mirrors over FTP mirrors (when the hostname is the same), or perhaps you could implement that logic on mirrors.fedoraproject.org. Besides the lower latency of HTTP vs FTP, the FTP mirror here is order of magnitude slower than HTTP on same server. Using fastestmirror=1 (which is NOT the default despite what the docs say at https://fedoraproject.org/wiki/Yum_to_DNF_Cheatsheet) works this around, but there was a bug https://bugzilla.redhat.com/show_bug.cgi?id=1051554 asking to turn it off, so turning it back on again would just shift the problem to other users. Version-Release number of selected component (if applicable): 1.0.1 How reproducible: Dnf seems to randomly pick between http and ftp URL of same mirror. When it picks ftp it is always MUCH slower. Steps to Reproduce: 1. dnf clean all 2. dnf check-update 3. look at download speed Actual results: very slow, between 50-200 KB/s Expected results: normal speed, around 4.5 - 5.3 MB/s Additional info: By default dnf sometimes picks this ftp mirror which is very slow: ftp://fedora.mirrors.telekom.ro/fedora/linux/updates/22/x86_64/ If I enable fastestmirror=1 in /etc/dnf/dnf.conf then it always picks the http mirror: http://fedora.mirrors.telekom.ro/pub/fedora/linux/updates/22/x86_64 FWIW both connect to the same IPv4 address, I don't know why FTP is that much slower, but since I don't see any advantage of using FTP over HTTP, I'd just always want to use HTTP(S) by default where available. Here is the curl -O output showing the speed difference: curl -O http://fedora.mirrors.telekom.ro/pub/fedora/linux/updates/22/x86_64/repodata/bf4506b4c2b5040bc6da647a7a1e72b9beffaeab697165ab48fb3f4e5b13262a-filelists.xml.gz % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 8758k 100 8758k 0 0 6076k 0 0:00:01 0:00:01 --:--:-- 6073k curl -O ftp://fedora.mirrors.telekom.ro/fedora/linux/updates/22/x86_64/repodata/bf4506b4c2b5040bc6da647a7a1e72b9beffaeab697165ab48fb3f4e5b13262a-filelists.xml.gz % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 8758k 100 8758k 0 0 49911 0 0:02:59 0:02:59 --:--:-- 46419
*** Bug 1245753 has been marked as a duplicate of this bug. ***
We can add config option for protocol preference and sort mirrorlists accordingly.
What about changing the timeout's: [MIRROR] libpurple-devel-2.10.11-12.fc22.i686.rpm: Curl error (28): Timeout was reached for ftp://mirror.steadfast.net/fedora/updates/22/i386/l/libpurple-devel-2.10.11-12.fc22.i686.rpm [Connection time-out] [MIRROR] vala-0.28.1-1.fc22.i686.rpm: Curl error (28): Timeout was reached for ftp://mirror.steadfast.net/fedora/updates/22/i386/v/vala-0.28.1-1.fc22.i686.rpm [Connection time-out]
(In reply to EMR_Fedora from comment #3) > What about changing the timeout's: Completely irrelevant for this bug, if ftp doesn't work, or isn't allowed thought a firewall all you're then doing then doing by changing the timeout is prolonging the pain and making the problem worse.
*** Bug 1258191 has been marked as a duplicate of this bug. ***
I'm afraid FTP/slower might be a red herring in this case. I've no problem getting fast downloads from FTP server 200.93.227.165. If dnf connects to FTP server 200.236.31.8, then it just gets stuck, consistently.
Created attachment 1069006 [details] FTP Connection to 200.93.227.165 working
Created attachment 1069007 [details] FTP Connection to 200.236.31.8 getting stuck
(In reply to Giovanni Tirloni from comment #6) > I'm afraid FTP/slower might be a red herring in this case. > > I've no problem getting fast downloads from FTP server 200.93.227.165. If > dnf connects to FTP server 200.236.31.8, then it just gets stuck, > consistently. But the point is that yum never had this problem. I changed my timeout to 5 seconds and it seems to have improved performance.
Is there any specific reason in Fedora 22, while I run "dnf check-update" or "dnf update" after "dnf clean all", it first request the repomd.xml by HTTP and then It request the same data by FTP as well? (Both receive the data successfully according to network traffic) Why it request it two time? HTTP: Request: GET /fedora/linux/releases/22/Everything/x86_64/os/repodata/repomd.xml HTTP/1.1\r\n Response: HTTP/1.1 200 OK\r\n eXtensible Markup Language <repomd xmlns="http://linux.duke.edu/metadata/repo"... FTP: Request:RETR repomd.xml\r\n Response:FTP Data (<?xml version="1.0" encoding="UTF-8"?>\n<repomd xmlns="http://linux.duke.edu/metadata/repo" xmlns:rpm="http://linux.duke.edu/metadata/rpm">\n <revision>1441821652</revision>\n <tags>\n <content>binary-x86_64</content>\n </t...
I would like to know how it design, when it use FTP and When HTTP, in default?
The problem with DNF is libcurl. Curl vs Wget is below at 2minutes vs 0.1sec respectively!! $ curl -o foobar ftp://mirror.symnds.com/distributions/fedora/releases/22/Everything/i386/os/Packages/m/maven-scm-javadoc-1.9.2-2.fc22.noarch.rpm % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 1409k 100 1409k 0 0 11313 0 0:02:07 0:02:07 --:--:-- 348k $ wget ftp://mirror.symnds.com/distributions/fedora/releases/22/Everything/i386/os/Packages/m/maven-scm-javadoc-1.9.2-2.fc22.noarch.rpm --2015-09-11 12:56:58-- ftp://mirror.symnds.com/distributions/fedora/releases/22/Everything/i386/os/Packages/m/maven-scm-javadoc-1.9.2-2.fc22.noarch.rpm => 'maven-scm-javadoc-1.9.2-2.fc22.noarch.rpm.1' Resolving mirror.symnds.com (mirror.symnds.com)... 63.245.196.124 Connecting to mirror.symnds.com (mirror.symnds.com)|63.245.196.124|:21... connected. Logging in as anonymous ... Logged in! ==> SYST ... done. ==> PWD ... done. ==> TYPE I ... done. ==> CWD (1) /distributions/fedora/releases/22/Everything/i386/os/Packages/m ... done. ==> SIZE maven-scm-javadoc-1.9.2-2.fc22.noarch.rpm ... 1442888 ==> PASV ... done. ==> RETR maven-scm-javadoc-1.9.2-2.fc22.noarch.rpm ... done. Length: 1442888 (1.4M) (unauthoritative) maven-scm-javadoc-1.9.2-2.fc22.noarch.rpm.1 100%[=======================================================================================================================>] 1.38M --.-KB/s in 0.1s 2015-09-11 12:56:59 (10.6 MB/s) - 'maven-scm-javadoc-1.9.2-2.fc22.noarch.rpm.1' saved [1442888]
It seems curl is using EPSV by default, while wget is not. Could you try curl with `--verbose` and `--disable-epsv` and compare?
Yeah, you were right! $ curl --disable-epsv -o foobar ftp://mirror.symnds.com/distributions/fedora/releases/22/Everything/i386/os/Packages/m/maven-scm-javadoc-1.9.2-2.fc22.noarch.rpm % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 1409k 100 1409k 0 0 2819k 0 --:--:-- --:--:-- --:--:-- 2818k So how do we make dnf use --disable-epsv??
Created attachment 1072651 [details] Add ftp_use_epsv option to dnf
Created attachment 1072652 [details] Exposes libcurl EPSV enable/disable option
A "me too" post, with suggested temporary fix. This seems to be more or less the same problem as reported in Bug 1229050 ??? This problem can be very annoying because it seems to have nothing to do with the peak transfer speeds your network connection is capable of. Results with a Telebit Worldblazer modem set for 50 baud and results with 10^100 Google fibre lines are identical. The comments above appear to be correct; if curl is configured to disable the use of the EPSV command, everything runs smooth as silk. A suggested short term fix (works for me) is to create a .curlrc in the home directory of the root user (/root/.curlrc) containing; # # .curlrc - curl configuration file # --disable-epsv
I've tried the .curlrc suggestion but it didn't work here. I can see from the packet traces that it's still using EPSV whenever it connects to a FTP server. Maybe someone with contacts to the developers could point out this bug report contains patches to address this issue? It's very annoying.
Giovanni, you are right. My suggestion was premature as I started to have this problem all over again this weekend. (As you no doubt noticed from the process of putting together your patch, dnf uses libcurl, rather than running the curl application. The curl application reads the .curlrc file, but libcurl does not). I think we just need to bring this to the attention of the right people. I have cleaned up your patches to librepo (Gratze!) and will submit them to librepo upstream on Github (just some changes for shifts of line numbers due to recently added code). Once that gets fixed, we can ping the dnf folks with the other patch. Indeed, this problem is very annoying. It has been said; "The definition of insanity is trying something over and over, and expecting a different result". That is exactly what dnf does, so I guess it has gone insane!! I did a little detective work to determine why this problem went away for me (for awhile...) and it seemed related to the fact that I also had enabled the dnf fastestmirror option. The following seems to work as a short term fix; 1) Edit /etc/dnf/dnf.conf and add the line "fastestmirror=true" 2) Whenever dnf gets stuck because it insists on using an ftp based mirror, edit the file /var/cache/dnf/fastestmirror.cache. Look for any ftp entries, and change the value assigned to the connectime parameter to -1, so it looks something like this; [ftp://mirror.cs.princeton.edu] ts=1453835022 connectime=-1 It may be necessary to edit the file all over again when dnf updates it. If you mess it up, it looks like you can just delete it, and dnf will regenerate it the next time it runs. A big pain for sure, but pretty minor when compared to the agony of watching dnf go on and on (and on... and on... and on... and on... and on...) for hours just to download a few kilobytes.
This package has changed ownership in the Fedora Package Database. Reassigning to the new owner of this component.
Fedora 22 changed to end-of-life (EOL) status on 2016-07-19. Fedora 22 is no longer maintained, which means that it will not receive any further security or bug fix updates. As a result we are closing this bug. If you can reproduce this bug against a currently maintained version of Fedora please feel free to reopen this bug against that version. If you are unable to reopen this bug, please file a new report against the current release. If you experience problems, please add a comment to this bug. Thank you for reporting this bug and we are sorry it could not be fixed.