Description of problem: Start 'dnf upgrade -y' and it will take an eternity untill mirrors are asked, package information and packages be downloaded If '-vv' option is used there are lots of messages seen like [MIRROR] Curl error (28): Timeout was reached for ftp://fedora.mirrors.ovh.net/download.fedora.redhat.com/linux/updates/22/x86_64/drpms/ [Connection time-out] [MIRROR] Curl error (28): Timeout was reached for ... (many other mirrors) or the like. The network connection speed is 767.72 Mbps (download) and 130.15 Mbps (upload). I've tested this in different European countries with similar fast network connections. Something is wrong with the mirror list. Fedora machines with yum do the updates on the same networks very fast. Version-Release number of selected component (if applicable): Version : 1.0.0 Release : 1.fc22
Thanx, seems to work from today on!
Again, dnf is extremely slow: time dnf upgrade -y -- snip -- real 32m2.613s user 9m1.751s sys 2m0.421s
Actually dnf is only extremely slow, when it has to rebuild its cache. So normally it works fine, just after 'dnf clean all' it takes a long time (ca 30 min) to rebuild the cached data and then doing what is is supposed to do. So the problem is here the long time for rebuilding the cache.
Is this is "the future", I'll stick to the past (of yum, which at least worked). ;-\
Daniel, can you point out in which part is the DNF slow, please? A-B, B-C or C-D? # sudo dnf update --refresh <A> Fedora 19 - x86_ 50 kB/s | 44 kB 00:00 Fedora 19 - x86_ 44 MB/s | 35 MB 00:00 tools 55 kB/s | 66 kB 00:01 RPM Fusion for F 3.4 MB/s | 408 kB 00:00 RPM Fusion for F 4.0 MB/s | 475 kB 00:00 ... <B> Last metadata expiration check performed 0:00:00 ago on Fri Jun 26 13:07:53 2015. <C> Dependencies resolved. ================================================= Package Arch Version Repository Size ================================================= ... <D> Total download size: 123 M Is this ok [y/N]:
If the cache is not cleaned, the A-B: Tor experimental sou| 7.3 kB/s | 1.7 kB 00:00 RPM Fusion for F... | 5.6 kB/s | 399 B 00:00 ... all additional repos ... is quite fast. It begins to hang starting from Fedora 22 - x86_64- Updates 0% [ ] --- B/s | 0 B --:-- ETA Over all it takes real 16m19.434s user 2m13.698s sys 0m49.299s without any downloads! When cleaning the cache with 'dnf clean all' it starts even slower and hangs at exactly the same point: Fedora 22 - x86_64- Updates 0% [ ] --- B/s | 0 B --:-- ETA All other repositories a quite fast. real 22m46.599s user 2m33.758s sys 0m52.639s Doing the same on a F21 system with yum, even after cleaning cache: # yum clean all # time yum update -y fedora/21/x86_64 | 24 kB 00:00:00 fedora | 3.8 kB 00:00:00 rpmfusion-free | 2.5 kB 00:00:00 rpmfusion-free-up..| 2.7 kB 00:00:00 rpmfusion-nonfree | 1.2 kB 00:00:00 rpmfusion-nonfree..| 2.7 kB 00:00:00 updates/21/x86_64 | 22 kB 00:00:00 updates | 4.9 kB 00:00:00 (1/4): fedora/21/..| 232 kB 00:00:00 .... ================================================= Package Arch Version Repository Size ================================================= ... Total download size ... Downloading packages ... Running transaction check Running transaction test Transaction test succeeded Running transaction (shutdown inhibited)... real 0m18.642s user 0m9.157s sys 0m1.463s
[root@infinity duff]# time dnf upgrade Error: Fallo al sincronizar la cache para el repositorio 'updates' desde 'https://mirrors.fedoraproject.org/metalink?repo=updates-released-f22&arch=x86_64':Cannot prepare internal mirrorlist: Curl error (28): Timeout was reached for https://mirrors.fedoraproject.org/metalink?repo=updates-released-f22&arch=x86_64 [Connection timed out after 120001 milliseconds] real 2m7.510s user 0m0.552s sys 0m0.181s Mirroring in my system is awful tbh.
Daniel, can you please set LIBREPO_DEBUG=1 [1] before running the update and share with us what it prints to the output? [1] https://github.com/rpm-software-management/dnf/wiki/Bug-Reporting#connection-issue
Created attachment 1044666 [details] dnf.log.gz
# export LIBREPO_DEBUG=1 # time dnf upgrade -y 2>&1 | tee -a DNF.log -- snip -- real 15m24.685s user 1m54.624s sys 0m52.298s Have attached 2 files: dnf.log.gz, DNF.log.gz. Thanx for yr help 'n have a nice day!
Created attachment 1044667 [details] DNF.log.gz
I see two two improvements that could be made: * don't download filelists (bug 968006) * download metadata in parallel (bug 1172752) *** This bug has been marked as a duplicate of bug 1172752 ***
For some reason DNF attempts to connect at high ports (instead of FTP/HTTP) and I see a SYN_SENT which is being blocked by our network's firewall, so it just stalls. Yum always seemed to work fast and connect only on HTTP/FTP ports. Is there a way to force DNF to use ONLY http?
FYI: This should NOT have been marked as a dupe. So I just tried to dnf download some data, and even with the timeout set to 5 seconds, it took forever cycling through the servers, and just quit (without alternating) downloading meta-data. They were all FTP sites. Yet, when I used wget to each ftp URL that curl timed out on, it worked fine. I am not sure what's happening but it might be an upstream libcurl problem with FTP. [MIRROR] libid3tag-0.15.1b-19.fc22.i686.rpm: Curl error (28): Timeout was reached for ftp://mirror.symnds.com/distributions/fedora/releases/22/Everything/i386/os/Packages/l/libid3tag-0.15.1b-19.fc22.i686.rpm [Connection time-out] vs wget ftp://mirror.cogentco.com/pub/linux/fedora/linux/releases/22/Everything/i386/os/Packages/l/libid3tag-0.15.1b-19.fc22.i686.rpm --2015-09-01 11:09:32-- ftp://mirror.cogentco.com/pub/linux/fedora/linux/releases/22/Everything/i386/os/Packages/l/libid3tag-0.15.1b-19.fc22.i686.rpm => 'libid3tag-0.15.1b-19.fc22.i686.rpm.1' Resolving mirror.cogentco.com (mirror.cogentco.com)... 204.157.3.70 Connecting to mirror.cogentco.com (mirror.cogentco.com)|204.157.3.70|:21... connected. Logging in as anonymous ... Logged in! ==> SYST ... done. ==> PWD ... done. ==> TYPE I ... done. ==> CWD (1) /pub/linux/fedora/linux/releases/22/Everything/i386/os/Packages/l ... done. ==> SIZE libid3tag-0.15.1b-19.fc22.i686.rpm ... 53100 ==> PASV ... done. ==> RETR libid3tag-0.15.1b-19.fc22.i686.rpm ... done. Length: 53100 (52K) (unauthoritative) libid3tag-0.15.1b-19.fc22.i6 100%[===============================================>] 51.86K --.-KB/s in 0.02s 2015-09-01 11:09:32 (3.25 MB/s) - 'libid3tag-0.15.1b-19.fc22.i686.rpm.1' saved [53100]
Why did you close that bug? Nothing is solved. Dnf is still extremely slow. In most cases it takes 15 to 30 min. Dnf tries to download packages with curl from servers via ftp and permanently gets a timeout: [MIRROR] kernel-4.1.6-201.fc22.x86_64.rpm: Curl error (28): Timeout was reached for ftp://fr2.rpmfind.net/linux/fedora/linux/updates/22/x86_64/k/kernel-4.1.6-201.fc22.x86_64.rpm [Connection time-out] Curl error (28): Timeout was reached for https://mirrors.fedoraproject.org/metalink?repo=fedora-22&arch=x86_64 [Connection timed out after 120002 milliseconds] ... Maybe u should make the timeout shorter and then switch automatically to http. Sorry for bothering, but I do not see any progress.
This affects me too, so much so that I've resorted to creating a fedora 22 mirror on my local network. Given that I have to build and rebuild both vm's and docker images constantly, dnf's problems with mirrors make a local mirror a necessity. I've had issues even when I'm targeting dnf at a specific mirror that has proven fast and reliable for use with centos7 (mirror.hmc.edu). Also, even with a local mirror providing download speeds of ~160MB/s, 'dnf clean && dnf -y update' takes 20s on a 3.5ghz 12 core+32gb+nvme host. Definitely a step backwards performance-wise from yum, which isn't exactly a high bar.
Unfortunately, nothing has really changed in F23. Dnf massively produces errors of type "Curl error (28): Timeout was reached for ftp://"
Still no changes. Dnf massively produces errors of type "Curl error (28): Timeout was reached for ftp://". These sites function with http but not with ftp. Why don't you start with http and try then ftp later if the first does not work? Both protocols are cleartext and have no security. I don't see any advantages of ftp and obviously, most mirror servers have not implemented it.
(In reply to EMR_Fedora from comment #13) > For some reason DNF attempts to connect at high ports (instead of FTP/HTTP) > and I see a SYN_SENT which is being blocked by our network's firewall, so it > just stalls. Yum always seemed to work fast and connect only on HTTP/FTP > ports. Is there a way to force DNF to use ONLY http? Yes I agree with you! I am facing timeout with my fresh FC23 and DNF. And DNF is so slow ! I am searching information of TCP port used by DNF.... I posted here: http://forums.fedoraforum.org/showthread.php?t=308304 I don't allow all packets to go out with my iptables. When I launch an update with dnf , I see drop in the iptables log. Then it works few sec later because it must use ftp ou http, protocol that I allowed. I see tcp out to 39086 , 32054 etc How can we prevent this ? I didn't find anything for the moment. $netstat -tup tcp 0 1 myhost:57472 excellent.ircam.f:39086 SYN_SENT 15081/python3 tcp 0 0 myhost:45086 excellent.ircam.fr:ftp ESTABLISHED 15081/python3 tcp 0 1 myhost:46394 excellent.ircam.fr:32054 SYN_SENT 15081/python3 # dig +short excellent.ircam.fr 129.102.1.37 janv. 25 19:09:26 myhost kernel: [DROP]: IN= OUT=enp3s0 SRC=192.168.0.254 DST=129.102.1.37 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=31126 DF PROTO=TCP SPT=46394 DPT=32054 WINDOW=29200 RES=0x00 SYN URGP=0 janv. 25 19:09:26 myhost kernel: [DROP]: IN= OUT=enp3s0 SRC=192.168.0.254 DST=129.102.1.37 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=11880 DF PROTO=TCP SPT=57472 DPT=39086 WINDOW=29200 RES=0x00 SYN URGP=0
(In reply to letfid from comment #19) > I don't allow all packets to go out with my iptables. > When I launch an update with dnf , I see drop in the iptables log. > Then it works few sec later because it must use ftp ou http, protocol that I > allowed. > I see tcp out to 39086 , 32054 etc > > How can we prevent this ? > I didn't find anything for the moment. I guess DNF uses libcurl and this is a known problem with the upstream provider. They should go back to NOT using libcurl, or have libcurl fix their library. *sigh* I am done.
This package has changed ownership in the Fedora Package Database. Reassigning to the new owner of this component.
Fedora 22 changed to end-of-life (EOL) status on 2016-07-19. Fedora 22 is no longer maintained, which means that it will not receive any further security or bug fix updates. As a result we are closing this bug. If you can reproduce this bug against a currently maintained version of Fedora please feel free to reopen this bug against that version. If you are unable to reopen this bug, please file a new report against the current release. If you experience problems, please add a comment to this bug. Thank you for reporting this bug and we are sorry it could not be fixed.
I am really sorry that there is not much progress with the issue. So far We changed timeouts from 120 s to 30 s (this is identical to YUM). We also believe that there is some progress in development in libcurl library. We also refactored dnf itself, where dnf-2.5.1 provides a lot improvements and we work on parallel metadata download. Because the original issue was reported on Fc22 and later on Fc23 and from that time no other reports I believe that the issue was fixed. Please you can test the issue with dnf-2.5.1 (available for rawhide, Fedora 26 or from our test repository for Fedora 24+ (dnf copr enable rpmsoftwaremanagement/dnf-nightly)). And if you will be able to reproduce the issue with our latest dnf, do not hesitate to reopen the bug report, and I will directly start with solving the issue (of course with your help). Thanks a lot.
Hi Jaro, the issue is long gone - I can't even remember! :-) In F25 and F26beta there are absolutely no problems of that kind anymore.
Thanks Daniel.