Bug 1236291 - dnf often fails to work in constrained network environments with metalink
Summary: dnf often fails to work in constrained network environments with metalink
Keywords:
Status: CLOSED INSUFFICIENT_DATA
Alias: None
Product: Fedora
Classification: Fedora
Component: dnf
Version: 23
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Packaging Maintenance Team
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-06-27 11:35 UTC by Peter Robinson
Modified: 2017-07-03 12:12 UTC (History)
10 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2015-07-29 12:05:02 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
dnf.log (1.62 KB, text/plain)
2015-06-29 16:27 UTC, Peter Robinson
no flags Details
dnf.rpm.log (49 bytes, text/plain)
2015-06-29 16:28 UTC, Peter Robinson
no flags Details
hawkey.log (2.19 KB, text/plain)
2015-06-29 16:28 UTC, Peter Robinson
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 1219283 0 unspecified CLOSED temporary unavailability of fedora servers 2021-02-22 00:41:40 UTC

Internal Links: 1219283

Description Peter Robinson 2015-06-27 11:35:16 UTC
dnf managed to get the updates details using the metalink but then regularly fails to actually download anything.

yum-deprecated works every time without fail. 

The main error is:

Error: Error downloading packages:
  Curl error (28): Timeout was reached for https://mirrors. fedoraproject.org/metalink?repo=fedora-22&arch=aarch64 [Connection timed out after 120001 milliseconds]

Full output:

# dnf upgrade
Last metadata expiration check performed 2:30:24 ago on Sat Jun 27 08:46:12 2015.
Dependencies resolved.
======================================================================================
 Package                           Arch      Version            Repository       Size
======================================================================================
Installing:
 git-core                          aarch64   2.4.3-1.fc22       updates          5.1 M
 kernel                            aarch64   4.0.5-300.fc22     updates-testing   66 k
 kernel-core                       aarch64   4.0.5-300.fc22     updates-testing   15 M
 kernel-modules                    aarch64   4.0.5-300.fc22     updates-testing   12 M
 python-firewall                   noarch    0.3.14.1-1.fc22    updates          235 k
     replacing  python2-firewall. noarch 0.3.13-7.fc22
 python-idna                       noarch    1.0-1.fc22         fedora            46 k
 python-ipaddress                  noarch    1.0.7-3.fc22       updates           36 k
 python-pip                        noarch    6.0.8-1.fc22       fedora           1.7 M
Upgrading:
 SDL2                              aarch64   2.0.3-5.fc22       updates          335 k
 anaconda                          aarch64   22.20.13-1.fc22    fedora           121 k
 anaconda-core                     aarch64   22.20.13-1.fc22    fedora           1.5 M
 anaconda-gui                      aarch64   22.20.13-1.fc22    fedora           424 k
 anaconda-tui                      aarch64   22.20.13-1.fc22    fedora           206 k
 anaconda-user-help                noarch    22.4-1.fc22        updates           43 k
 anaconda-widgets                  aarch64   22.20.13-1.fc22    fedora           146 k
 augeas-libs                       aarch64   1.4.0-1.fc22       updates          340 k
 bash                              aarch64   4.3.39-1.fc22      updates          1.6 M
 brlapi                            aarch64   0.6.3-3.fc22       updates          150 k
 brltty                            aarch64   5.2-3.fc22         updates          1.0 M
 coreutils                         aarch64   8.23-10.fc22       updates          2.8 M
 createrepo_c                      aarch64   0.9.0-1.fc22       updates           73 k
 createrepo_c-libs                 aarch64   0.9.0-1.fc22       updates           85 k
 cups-libs                         aarch64   1:2.0.3-1.fc22     updates-testing  366 k
 cyrus-sasl                        aarch64   2.1.26-23.fc22     updates           91 k
 cyrus-sasl-lib                    aarch64   2.1.26-23.fc22     updates          155 k
 cyrus-sasl-md5                    aarch64   2.1.26-23.fc22     updates           58 k
 dbus                              aarch64   1:1.8.18-1.fc22    updates          338 k
 dbus-libs                         aarch64   1:1.8.18-1.fc22    updates          158 k
 efibootmgr                        aarch64   0.12-1.fc22        updates           33 k
 efivar-libs                       aarch64   0.20-1.fc22        updates           50 k
fedora-release                    noarch    22-1                fedora            23 k
fedora-repos                      noarch    22-1                fedora            20 k
 fedup-dracut                      aarch64   0.9.2-1.fc22       fedora            36 k
 fedup-dracut-plymouth             noarch    0.9.2-1.fc22       fedora           203 k
 firewalld                         noarch    0.3.14.1-1.fc22    updates          334 k
 firewalld-filesystem              noarch    0.3.14.1-1.fc22    updates           47 k
 fontconfig                        aarch64   2.11.94-1.fc22     updates          239 k
 fontpackages-filesystem           noarch    1.44-12.fc22       updates           14 k
 fpaste                            noarch    0.3.7.4-1.fc22     updates           37 k
 fuse                              aarch64   2.9.4-1.fc22       updates           90 k
 fuse-libs                         aarch64   2.9.4-1.fc22       updates           92 k
 genisoimage                       aarch64   1.1.11-28.fc22     updates          293 k
 git                               aarch64   2.4.3-1.fc22       updates          4.5 M
 gnupg2                            aarch64   2.1.4-1.fc22       updates          1.7 M
 hawkey                            aarch64   0.5.7-1.fc22       updates           86 k
 hwdata                            noarch    0.278-1.fc22       updates          1.3 M
 kmod                              aarch64   21-1.fc22          updates-testing  107 k
 kmod-libs                         aarch64   21-1.fc22          updates-testing   57 k
 langtable                         noarch    0.0.33-1.fc22      updates           39 k
 langtable-data                    noarch    0.0.33-1.fc22      updates          625 k
 langtable-python                  noarch    0.0.33-1.fc22      updates           34 k
 less                              aarch64   471-4.fc22         updates          124 k
 libblockdev                       aarch64   0.13-2.fc22        fedora            66 k
 libblockdev-btrfs                 aarch64   0.13-2.fc22        fedora            26 k
 libblockdev-crypto                aarch64   0.13-2.fc22        fedora            26 k
 libblockdev-dm                    aarch64   0.13-2.fc22        fedora            25 k
 libblockdev-loop                  aarch64   0.13-2.fc22        fedora            23 k
 libblockdev-lvm                   aarch64   0.13-2.fc22        fedora            29 k
 libblockdev-mdraid                aarch64   0.13-2.fc22        fedora            26 k
 libblockdev-mpath                 aarch64   0.13-2.fc22        fedora            22 k
 libblockdev-plugins-all           aarch64   0.13-2.fc22        fedora            18 k
 libblockdev-swap                  aarch64   0.13-2.fc22        fedora            23 k
 libblockdev-utils                 aarch64   0.13-2.fc22        fedora            25 k
 libcap-ng                         aarch64   0.7.5-2.fc22       updates           29 k
 libdb                             aarch64   5.3.28-12.fc22     updates          656 k
 libdb-utils                       aarch64   5.3.28-12.fc22     updates          109 k
 libgcc                            aarch64   5.1.1-3.fc22       updates-testing   75 k
 libgomp                           aarch64   5.1.1-3.fc22       updates-testing  142 k
 libgudev1                         aarch64   219-18.fc22        updates-testing   60 k
 libguestfs                        aarch64   1:1.29.46-1.fc22   updates-testing  1.9 M
 libgusb                           aarch64   0.2.5-1.fc22       updates           42 k
 libosinfo                         aarch64   0.2.12-1.fc22      updates          258 k
 libpcap                           aarch64   14:1.7.3-1.fc22    updates          141 k
 librados2                         aarch64   1:0.94.1-4.fc22    updates-testing  1.5 M
 librbd1                           aarch64   1:0.94.1-4.fc22    updates-testing  1.6 M
 librepo                           aarch64   1.7.16-1.fc22      updates           77 k
 libreport                         aarch64   2.6.0-1.fc22       updates-testing  452 k
 libreport-anaconda                aarch64   2.6.0-1.fc22       updates-testing   52 k
 libreport-cli                     aarch64   2.6.0-1.fc22       updates-testing   56 k
 libreport-filesystem              aarch64   2.6.0-1.fc22       updates-testing   43 k
 libreport-gtk                     aarch64   2.6.0-1.fc22       updates-testing  108 k
 libreport-plugin-bugzilla         aarch64   2.6.0-1.fc22       updates-testing   88 k
 libreport-plugin-reportuploader   aarch64   2.6.0-1.fc22       updates-testing   65 k
 libreport-python                  aarch64   2.6.0-1.fc22       updates-testing   77 k
 libreport-web                     aarch64   2.6.0-1.fc22       updates-testing   59 k
 libseccomp                        aarch64   2.2.1-0.fc22       updates           53 k
 libselinux                        aarch64   2.3-10.fc22        updates-testing  146 k
 libselinux-python                 aarch64   2.3-10.fc22        updates-testing  280 k
 libselinux-utils                  aarch64   2.3-10.fc22        updates-testing  144 k
 libstdc++                         aarch64   5.1.1-3.fc22       updates-testing  389 k
 libtiff                           aarch64   4.0.3-20.fc22      updates          160 k
 libtirpc                          aarch64   0.3.1-0.0.fc22     updates           87 k
 libusal                           aarch64   1.1.11-28.fc22     updates          136 k
 libvirt-client                    aarch64   1.2.13.1-2.fc22    updates-testing  4.3 M
 libvirt-daemon                    aarch64   1.2.13.1-2.fc22    updates-testing  477 k
 libvirt-daemon-config-network     aarch64   1.2.13.1-2.fc22    updates-testing   45 k
 libvirt-daemon-driver-interface   aarch64   1.2.13.1-2.fc22    updates-testing   84 k
 libvirt-daemon-driver-network     aarch64   1.2.13.1-2.fc22    updates-testing  209 k
 libvirt-daemon-driver-nodedev     aarch64   1.2.13.1-2.fc22    updates-testing   84 k
 libvirt-daemon-driver-nwfilter    aarch64   1.2.13.1-2.fc22    updates-testing  106 k
 libvirt-daemon-driver-qemu        aarch64   1.2.13.1-2.fc22    updates-testing  430 k
 libvirt-daemon-driver-secret      aarch64   1.2.13.1-2.fc22    updates-testing   78 k
 libvirt-daemon-driver-storage     aarch64   1.2.13.1-2.fc22    updates-testing  229 k
 libvirt-daemon-kvm                aarch64   1.2.13.1-2.fc22    updates-testing   43 k
 libvirt-daemon-qemu               aarch64   1.2.13.1-2.fc22    updates-testing   43 k
 libxcb                            aarch64   1.11-5.fc22        updates          191 k
 lorax                             aarch64   22.12-1.fc22       updates-testing  171 k
 lz4                               aarch64   r129-1.fc22        updates           74 k
 man-db                            aarch64   2.7.1-5.fc22       updates          818 k
 mariadb-common                    aarch64   1:10.0.19-2.fc22   updates           74 k
 mariadb-config                    aarch64   1:10.0.19-2.fc22   updates           25 k
 mariadb-libs                      aarch64   1:10.0.19-2.fc22   updates          617 k
 nfs-utils                         aarch64   1:1.3.2-8.fc22     updates-testing  360 k
 nss                               aarch64   3.19.1-1.0.fc22    updates          800 k
 nss-softokn                       aarch64   3.19.1-1.0.fc22    updates          297 k
 nss-softokn-freebl                aarch64   3.19.1-1.0.fc22    updates          186 k
 nss-sysinit                       aarch64   3.19.1-1.0.fc22    updates           54 k
 nss-tools                         aarch64   3.19.1-1.0.fc22    updates          501 k
 nss-util                          aarch64   3.19.1-1.0.fc22    updates           76 k
 ntfs-3g                           aarch64   2:2015.3.14-2.fc22 updates          269 k
 ntfsprogs                         aarch64   2:2015.3.14-2.fc22 updates          289 k
 openssh                           aarch64   6.8p1-8.fc22       updates-testing  427 k
 openssh-clients                   aarch64   6.8p1-8.fc22       updates-testing  585 k
 openssh-server                    aarch64   6.8p1-8.fc22       updates-testing  440 k
 pango                             aarch64   1.36.8-5.fc22      updates          283 k
 perl                              aarch64   4:5.20.2-326.fc22  updates          8.0 M
 perl-Getopt-Long                  noarch    2.46-1.fc22        updates           61 k
 perl-Git                          noarch    2.4.3-1.fc22       updates           60 k
 perl-HTTP-Tiny                    noarch    0.056-1.fc22       updates           52 k
 perl-Pod-Escapes                  noarch    1:1.06-326.fc22    updates           62 k
 perl-Scalar-List-Utils            aarch64   1:1.42-1.fc22      updates           60 k
 perl-libs                         aarch64   4:5.20.2-326.fc22  updates          675 k
 perl-macros                       aarch64   4:5.20.2-326.fc22  updates           54 k
 pinentry                          aarch64   0.9.2-1.fc22       updates           71 k
 python                            aarch64   2.7.10-1.fc22      updates           93 k
 python-cryptography               aarch64   0.9-1.fc22         updates          450 k
 python-dnf-plugins-extras-common  noarch    0.0.8-1.fc22       updates           26 k
 python-dnf-plugins-extras-migrate noarch    0.0.8-1.fc22       updates           23 k
 python-enum34                     noarch    1.0.4-1.fc22       updates           56 k
 python-hawkey                     aarch64   0.5.7-1.fc22       updates           72 k
 python-libguestfs                 aarch64   1:1.29.46-1.fc22   updates-testing  261 k
 python-librepo                    aarch64   1.7.16-1.fc22      updates           52 k
 python-libs                       aarch64   2.7.10-1.fc22      updates          5.7 M
 python-requests                   noarch    2.7.0-1.fc22       updates           99 k
 python-setuptools                 noarch    17.0-1.fc22        updates          424 k
 python3-setuptools                noarch    17.0-1.fc22        updates          406 k
 qemu                              aarch64   2:2.3.0-5.fc22     updates-testing   55 k
 qemu-common                       aarch64   2:2.3.0-5.fc22     updates-testing  286 k
 qemu-img                          aarch64   2:2.3.0-5.fc22     updates-testing  603 k
 qemu-kvm                          aarch64   2:2.3.0-5.fc22     updates-testing   54 k
 qemu-system-aarch64               aarch64   2:2.3.0-5.fc22     updates-testing  1.9 M
 qemu-system-alpha                 aarch64   2:2.3.0-5.fc22     updates-testing  1.5 M
 qemu-system-arm                   aarch64   2:2.3.0-5.fc22     updates-testing  1.9 M
 qemu-system-cris                  aarch64   2:2.3.0-5.fc22     updates-testing  1.1 M
 qemu-system-lm32                  aarch64   2:2.3.0-5.fc22     updates-testing  1.1 M
 qemu-system-m68k                  aarch64   2:2.3.0-5.fc22     updates-testing  1.4 M
 qemu-system-microblaze            aarch64   2:2.3.0-5.fc22     updates-testing  2.1 M
 qemu-system-mips                  aarch64   2:2.3.0-5.fc22     updates-testing  6.5 M
 qemu-system-moxie                 aarch64   2:2.3.0-5.fc22     updates-testing  1.1 M
 qemu-system-or32                  aarch64   2:2.3.0-5.fc22     updates-testing  1.0 M
 qemu-system-ppc                   aarch64   2:2.3.0-5.fc22     updates-testing  5.4 M
 qemu-system-s390x                 aarch64   2:2.3.0-5.fc22     updates-testing  1.3 M
 qemu-system-sh4                   aarch64   2:2.3.0-5.fc22     updates-testing  2.8 M
 qemu-system-sparc                 aarch64   2:2.3.0-5.fc22     updates-testing  2.5 M
 qemu-system-tricore               aarch64   2:2.3.0-5.fc22     updates-testing  1.1 M
 qemu-system-unicore32             aarch64   2:2.3.0-5.fc22     updates-testing  1.0 M
 qemu-system-x86                   aarch64   2:2.3.0-5.fc22     updates-testing  3.3 M
 qemu-system-xtensa                aarch64   2:2.3.0-5.fc22     updates-testing  2.1 M
 qemu-user                         aarch64   2:2.3.0-5.fc22     updates-testing  6.9 M
 rpcbind                           aarch64   0.2.3-0.1.fc22     updates-testing   61 k
 satyr                             aarch64   0.18-1.fc22        updates-testing  101 k
 screen                            aarch64   4.2.1-5.fc22       updates          545 k
 selinux-policy                    noarch    3.13.1-128.1.fc22  updates-testing  405 k
 selinux-policy-targeted           noarch    3.13.1-128.1.fc22  updates-testing  4.0 M
 sqlite                            aarch64   3.8.10.2-1.fc22    updates          433 k
 systemd                           aarch64   219-18.fc22        updates-testing  5.4 M
 systemd-libs                      aarch64   219-18.fc22        updates-testing  314 k
 unbound-libs                      aarch64   1.5.3-4.fc22       updates-testing  323 k
 virt-install                      noarch    1.2.1-1.fc22       updates           89 k
 virt-manager-common               noarch    1.2.1-1.fc22       updates          1.0 M
 webkitgtk3                        aarch64   2.4.9-1.fc22       updates           11 M
Removing:
 kernel                            aarch64   3.19.5-200.fc21   @System               0  
 kernel-core                       aarch64   3.19.5-200.fc21   @System            69 M
 kernel-modules                    aarch64   3.19.5-200.fc21   @System            55 M

Transaction Summary
======================================================================================
Install    8 Packages
Upgrade  189 Packages
Remove     3 Packages

Total download size: 192 M
Is this ok [y/N]: y
Downloading Packages:
Error: Error downloading packages:
  Curl error (28): Timeout was reached for https://mirrors. fedoraproject.org/metalink?repo=fedora-22&arch=aarch64 [Connection timed out after 120001 milliseconds]

Comment 1 Honza Silhan 2015-06-29 15:45:25 UTC
Pasted from bug 1219283:
(In reply to philip ross from comment #24)
> (In reply to Peter Robinson from comment #19)
> > Actually I'm still seeing it, and we have enough resilience in our
> > servers/mirrors that it shouldn't fail with the loss of a single server. I'm
> > also seeing it on some devices but not others.
> 
> If the underlying cause of the problem is because DNF cannot download files
> from some hosts when it uses extended passive mode (epsv), is that a bug in
> DNF? In the uk when DNF tries to download repomd.xml from
> ftp://ftp.mirrorservice.org it fails when it tries to switch the ftp
> transfer into epsv mode.
> 
> When this happens, the host is alive and I can download the file manually in
> curl by specifying the -disable-epsv flag.
> 
> In my mind, this should be classified as a bug in DNF because it is new
> behaviour compared to YUM in Fedora 21 and there appears to be no
> configuration or option to tell DNF to not consider epsv mode. The -4 option
> to DNF doesn't seem to help, but one possibility would be to have DNF
> disable epsv mode when the -4 option is present, as my understanding is that
> epsv only difference to pasv is support for ipv6.

(In reply to Peter Robinson from comment #26)
> > The problem with the EPSV mode is a duplicate of the bug 1220269 which is a
> > duplicate of the bug 1219817 which should be fixed by librepo-1.7.16-1.fc22.
> > Is this your case, Peter?
> 
> Nope, I have 12 Fedora machines running ARM, 8 of them don't have an issue.
> They're also running on a dual stack IPv4/IPv6 native routed (not via a
> tunneling network) so there's means of them getting out completely without
> NAT. 


Peter, can you please run again `dnf update --refresh`, attach `/var/log/dnf.librepo.log` and post output of `rpm -q dnf librepo`?

Comment 2 Peter Robinson 2015-06-29 15:47:59 UTC
With the following versions:

dnf-1.0.0-1.fc22.noarch
librepo-1.7.16-1.fc22.aarch64
hawkey-0.5.7-1.fc22.aarch64

still get the timeout

  Curl error (28): Timeout was reached for https://mirrors.fedoraproject.org/metalink?repo=fedora-22&arch=aarch64 [Connection timed out after 120001 milliseconds]

Sorry, it seems I forgot to post that update. I'll collect the output of the above logs the next time I'm on the platform.

Comment 3 Peter Robinson 2015-06-29 16:24:02 UTC
> Peter, can you please run again `dnf update --refresh`, attach
> `/var/log/dnf.librepo.log` and post output of `rpm -q dnf librepo`?

When running a "dnf update --refresh" I don't get a dnf.librepo.log. I get a dnf.log dnf.rpm.log and hawkey.log

Comment 4 Peter Robinson 2015-06-29 16:27:28 UTC
Created attachment 1044416 [details]
dnf.log

Comment 5 Peter Robinson 2015-06-29 16:28:05 UTC
Created attachment 1044417 [details]
dnf.rpm.log

Comment 6 Peter Robinson 2015-06-29 16:28:40 UTC
Created attachment 1044418 [details]
hawkey.log

Comment 7 Honza Silhan 2015-06-30 09:31:42 UTC
Thanks. The dnf.librepo.log is the only one I need - it's generated in dnf-1.0.1. The content of it could be parsed from stdout while setting LIBREPO_DEBUG=1 [1]. Can you also try `dnf update dnf --refresh --setopt=minrate=1000`?

[1] https://github.com/rpm-software-management/dnf/wiki/Bug-Reporting#connection-issue

Comment 8 Jan Kurik 2015-07-15 13:54:09 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 23 development cycle.
Changing version to '23'.

(As we did not run this process for some time, it could affect also pre-Fedora 23 development
cycle bugs. We are very sorry. It will help us with cleanup during Fedora 23 End Of Life. Thank you.)

More information and reason for this action is here:
https://fedoraproject.org/wiki/BugZappers/HouseKeeping/Fedora23

Comment 9 Honza Silhan 2015-07-29 12:05:02 UTC
Closing, we are not able to decect the problem without requested data. Feel free to attach them additionally and reopen.

Comment 10 Felix Miata 2015-08-31 23:25:37 UTC
Might the DNF timeouts I describe in https://lists.fedoraproject.org/pipermail/devel/2015-August/213852.html be appropriate to follow-up with info needed to continue this?

Comment 11 Peter Robinson 2017-07-03 12:12:46 UTC
Clean needinfo


Note You need to log in before you can comment on or make changes to this bug.