Bug 1767781 - Network errors, cannot dnf system-upgrade download Fedora 31
Summary: Network errors, cannot dnf system-upgrade download Fedora 31
Keywords:
Status: NEW
Alias: None
Product: Fedora
Classification: Fedora
Component: dnf-plugin-system-upgrade
Version: 31
Hardware: Unspecified
OS: Unspecified
low
unspecified
Target Milestone: ---
Assignee: Orphan Owner
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-11-01 12:10 UTC by Kostya Vasilyev
Modified: 2019-12-04 23:50 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:


Attachments (Terms of Use)

Description Kostya Vasilyev 2019-11-01 12:10:07 UTC
Description of problem:

dnf system-upgrade cannot download version 31 from any of the mirrors I've tried, including:

mirror.yandex.ru - our local Russia mirror, very large and fast
University of Chemnitz - a Tier 1 Fedora mirror
fedora.ip-connect.vn.ua - a mirror in Ukraine which I've used in the past

Getting the failed files with curl or wget works just fine and very fast.

My Internet connection is 1000 Mbit theoretical ~750 Mbit in practice, no issues with Speedtest.net or any other sites.

Version-Release number of selected component (if applicable):

curl-7.65.3-4.fc30.x86_64
dnf-4.2.11-2.fc30.noarch
libcurl-7.65.3-4.fc30.x86_64
dnf-plugins-core-4.0.10-1.fc30.noarch

How reproducible:

100%

Steps to Reproduce:

1.

sudo dnf system-upgrade download --refresh --releasever=31

Install      53 Packages
Upgrade    1824 Packages
Remove        3 Packages
Downgrade    14 Packages

2.

Total size: 1.8 G
Total download size: 1.8 G
DNF will only download packages, install gpg keys, and check the transaction.
Is this ok [y/N]: y

Actual results:


[MIRROR] python3-dnf-plugin-system-upgrade-4.0.5-3.fc31.noarch.rpm: Curl error (28): Timeout was reached for https://ftp.tu-chemnitz.de/pub/linux/fedora/linux/updates/testing/31/Everything/x86_64/Packages/p/python3-dnf-plugin-system-upgrade-4.0.5-3.fc31.noarch.rpm [Resolving timed out after 84009 milliseconds]
[MIRROR] dnf-plugins-core-4.0.9-1.fc31.noarch.rpm: Curl error (28): Timeout was reached for https://fedora.ip-connect.vn.ua/linux/releases/31/Everything/x86_64/os/Packages/d/dnf-plugins-core-4.0.9-1.fc31.noarch.rpm [Resolving timed out after 84007 milliseconds]
[MIRROR] python3-dnf-plugins-core-4.0.9-1.fc31.noarch.rpm: Curl error (28): Timeout was reached for https://ftp.tu-chemnitz.de/pub/linux/fedora/linux/releases/31/Everything/x86_64/os/Packages/p/python3-dnf-plugins-core-4.0.9-1.fc31.noarch.rpm [Resolving timed out after 84006 milliseconds]


Expected results:

A smooth download of Fedora 31

Additional info:

Comment 1 Kostya Vasilyev 2019-11-01 16:58:21 UTC
Found the culprit. Still a bug.

I had this in my dnf.conf:

max_parallel_downloads=10

This worked just fine until recently, more specifically an update earlier today that included these:

------

Packages Altered:
    Install  python3-xmltodict-0.12.0-2.fc30.noarch          @updates
    Install  python3-ntlm-auth-1.1.0-4.fc30.noarch           @fedora
    Install  python3-requests_ntlm-1.1.0-4.fc30.noarch       @fedora
    Install  python3-winrm-0.3.0-4.fc30.noarch               @fedora
    Upgrade  ansible-2.8.6-1.fc30.noarch                     @updates-testing
    Upgraded ansible-2.8.5-1.fc30.noarch                     @@System
    Upgrade  daxctl-libs-67-1.fc30.x86_64                    @updates-testing
    Upgraded daxctl-libs-66-1.fc30.x86_64                    @@System
    Upgrade  keepassxc-2.5.0-1.fc30.x86_64                   @updates-testing
    Upgraded keepassxc-2.4.3-6.fc30.x86_64                   @@System
    Upgrade  libburn-1.5.2-1.fc30.x86_64                     @updates-testing
    Upgraded libburn-1.5.0-2.fc30.x86_64                     @@System
    Upgrade  libisofs-1.5.2-1.fc30.x86_64                    @updates-testing
    Upgraded libisofs-1.5.0-2.fc30.x86_64                    @@System
    Upgrade  mkpasswd-5.5.2-2.fc30.x86_64                    @updates-testing
    Upgraded mkpasswd-5.5.2-1.fc30.x86_64                    @@System
    Upgrade  ndctl-67-1.fc30.x86_64                          @updates-testing
    Upgraded ndctl-66-1.fc30.x86_64                          @@System
    Upgrade  ndctl-libs-67-1.fc30.x86_64                     @updates-testing
    Upgraded ndctl-libs-66-1.fc30.x86_64                     @@System
    Upgrade  polkit-0.116-2.fc30.2.x86_64                    @updates-testing
    Upgraded polkit-0.116-2.fc30.1.x86_64                    @@System
    Upgrade  polkit-libs-0.116-2.fc30.2.x86_64               @updates-testing
    Upgraded polkit-libs-0.116-2.fc30.1.x86_64               @@System
    Upgrade  system-config-printer-1.5.12-2.fc30.x86_64      @updates-testing
    Upgraded system-config-printer-1.5.12-1.fc30.x86_64      @@System
    Upgrade  system-config-printer-libs-1.5.12-2.fc30.noarch @updates-testing
    Upgraded system-config-printer-libs-1.5.12-1.fc30.noarch @@System
    Upgrade  system-config-printer-udev-1.5.12-2.fc30.x86_64 @updates-testing
    Upgraded system-config-printer-udev-1.5.12-1.fc30.x86_64 @@System
    Upgrade  webkit2gtk3-2.26.1-3.fc30.x86_64                @updates-testing
    Upgraded webkit2gtk3-2.26.1-2.fc30.x86_64                @@System
    Upgrade  webkit2gtk3-jsc-2.26.1-3.fc30.x86_64            @updates-testing
    Upgraded webkit2gtk3-jsc-2.26.1-2.fc30.x86_64            @@System
    Upgrade  whois-nls-5.5.2-2.fc30.noarch                   @updates-testing
    Upgraded whois-nls-5.5.2-1.fc30.noarch                   @@System
    Upgrade  zlib-1.2.11-19.fc30.i686                        @updates-testing
    Upgraded zlib-1.2.11-18.fc30.i686                        @@System
    Upgrade  zlib-1.2.11-19.fc30.x86_64                      @updates-testing
    Upgraded zlib-1.2.11-18.fc30.x86_64                      @@System
    Upgrade  google-chrome-stable-78.0.3904.87-1.x86_64      @google-chrome
    Upgraded google-chrome-stable-78.0.3904.70-1.x86_64      @@System


-----

After this, dnf was not able to either download system-update packages, or any other "--refresh update"'s.

Once I've now removed

max_parallel_downloads=10

system-upgrade is proceeding just fine with its .rpm downloads.

Seems to me that dnf (or maybe python itself, oh no) has some kind of thread race / SMP / locking / waiting bug that only triggers with a sufficiently high number (>= 10) of parallel downloads. May be still worth fixing, good hygiene and all.

Comment 2 Jason Birch 2019-11-04 00:03:59 UTC
Ah, thank you Kostya!

I ran into this bug on one of four machines, all of which set max parallel to 10 or 20. The machine that ran into this issue was "fixed" by dropping it to 5, but that eventually froze up too — reverting to the default let me slowly get through the entire upgrade.


Note You need to log in before you can comment on or make changes to this bug.