Bug 1767781

Summary: Network errors, cannot dnf system-upgrade download Fedora 31
Product: [Fedora] Fedora Reporter: Kostya Vasilyev <kmansoft>
Component: dnf-plugin-system-upgradeAssignee: Orphan Owner <extras-orphan>
Status: CLOSED EOL QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: low    
Version: 34CC: extras-orphan, jbirch, jrohel, todoleza
Target Milestone: ---Keywords: Triaged
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-06-07 20:56:54 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Kostya Vasilyev 2019-11-01 12:10:07 UTC
Description of problem:

dnf system-upgrade cannot download version 31 from any of the mirrors I've tried, including:

mirror.yandex.ru - our local Russia mirror, very large and fast
University of Chemnitz - a Tier 1 Fedora mirror
fedora.ip-connect.vn.ua - a mirror in Ukraine which I've used in the past

Getting the failed files with curl or wget works just fine and very fast.

My Internet connection is 1000 Mbit theoretical ~750 Mbit in practice, no issues with Speedtest.net or any other sites.

Version-Release number of selected component (if applicable):

curl-7.65.3-4.fc30.x86_64
dnf-4.2.11-2.fc30.noarch
libcurl-7.65.3-4.fc30.x86_64
dnf-plugins-core-4.0.10-1.fc30.noarch

How reproducible:

100%

Steps to Reproduce:

1.

sudo dnf system-upgrade download --refresh --releasever=31

Install      53 Packages
Upgrade    1824 Packages
Remove        3 Packages
Downgrade    14 Packages

2.

Total size: 1.8 G
Total download size: 1.8 G
DNF will only download packages, install gpg keys, and check the transaction.
Is this ok [y/N]: y

Actual results:


[MIRROR] python3-dnf-plugin-system-upgrade-4.0.5-3.fc31.noarch.rpm: Curl error (28): Timeout was reached for https://ftp.tu-chemnitz.de/pub/linux/fedora/linux/updates/testing/31/Everything/x86_64/Packages/p/python3-dnf-plugin-system-upgrade-4.0.5-3.fc31.noarch.rpm [Resolving timed out after 84009 milliseconds]
[MIRROR] dnf-plugins-core-4.0.9-1.fc31.noarch.rpm: Curl error (28): Timeout was reached for https://fedora.ip-connect.vn.ua/linux/releases/31/Everything/x86_64/os/Packages/d/dnf-plugins-core-4.0.9-1.fc31.noarch.rpm [Resolving timed out after 84007 milliseconds]
[MIRROR] python3-dnf-plugins-core-4.0.9-1.fc31.noarch.rpm: Curl error (28): Timeout was reached for https://ftp.tu-chemnitz.de/pub/linux/fedora/linux/releases/31/Everything/x86_64/os/Packages/p/python3-dnf-plugins-core-4.0.9-1.fc31.noarch.rpm [Resolving timed out after 84006 milliseconds]


Expected results:

A smooth download of Fedora 31

Additional info:

Comment 1 Kostya Vasilyev 2019-11-01 16:58:21 UTC
Found the culprit. Still a bug.

I had this in my dnf.conf:

max_parallel_downloads=10

This worked just fine until recently, more specifically an update earlier today that included these:

------

Packages Altered:
    Install  python3-xmltodict-0.12.0-2.fc30.noarch          @updates
    Install  python3-ntlm-auth-1.1.0-4.fc30.noarch           @fedora
    Install  python3-requests_ntlm-1.1.0-4.fc30.noarch       @fedora
    Install  python3-winrm-0.3.0-4.fc30.noarch               @fedora
    Upgrade  ansible-2.8.6-1.fc30.noarch                     @updates-testing
    Upgraded ansible-2.8.5-1.fc30.noarch                     @@System
    Upgrade  daxctl-libs-67-1.fc30.x86_64                    @updates-testing
    Upgraded daxctl-libs-66-1.fc30.x86_64                    @@System
    Upgrade  keepassxc-2.5.0-1.fc30.x86_64                   @updates-testing
    Upgraded keepassxc-2.4.3-6.fc30.x86_64                   @@System
    Upgrade  libburn-1.5.2-1.fc30.x86_64                     @updates-testing
    Upgraded libburn-1.5.0-2.fc30.x86_64                     @@System
    Upgrade  libisofs-1.5.2-1.fc30.x86_64                    @updates-testing
    Upgraded libisofs-1.5.0-2.fc30.x86_64                    @@System
    Upgrade  mkpasswd-5.5.2-2.fc30.x86_64                    @updates-testing
    Upgraded mkpasswd-5.5.2-1.fc30.x86_64                    @@System
    Upgrade  ndctl-67-1.fc30.x86_64                          @updates-testing
    Upgraded ndctl-66-1.fc30.x86_64                          @@System
    Upgrade  ndctl-libs-67-1.fc30.x86_64                     @updates-testing
    Upgraded ndctl-libs-66-1.fc30.x86_64                     @@System
    Upgrade  polkit-0.116-2.fc30.2.x86_64                    @updates-testing
    Upgraded polkit-0.116-2.fc30.1.x86_64                    @@System
    Upgrade  polkit-libs-0.116-2.fc30.2.x86_64               @updates-testing
    Upgraded polkit-libs-0.116-2.fc30.1.x86_64               @@System
    Upgrade  system-config-printer-1.5.12-2.fc30.x86_64      @updates-testing
    Upgraded system-config-printer-1.5.12-1.fc30.x86_64      @@System
    Upgrade  system-config-printer-libs-1.5.12-2.fc30.noarch @updates-testing
    Upgraded system-config-printer-libs-1.5.12-1.fc30.noarch @@System
    Upgrade  system-config-printer-udev-1.5.12-2.fc30.x86_64 @updates-testing
    Upgraded system-config-printer-udev-1.5.12-1.fc30.x86_64 @@System
    Upgrade  webkit2gtk3-2.26.1-3.fc30.x86_64                @updates-testing
    Upgraded webkit2gtk3-2.26.1-2.fc30.x86_64                @@System
    Upgrade  webkit2gtk3-jsc-2.26.1-3.fc30.x86_64            @updates-testing
    Upgraded webkit2gtk3-jsc-2.26.1-2.fc30.x86_64            @@System
    Upgrade  whois-nls-5.5.2-2.fc30.noarch                   @updates-testing
    Upgraded whois-nls-5.5.2-1.fc30.noarch                   @@System
    Upgrade  zlib-1.2.11-19.fc30.i686                        @updates-testing
    Upgraded zlib-1.2.11-18.fc30.i686                        @@System
    Upgrade  zlib-1.2.11-19.fc30.x86_64                      @updates-testing
    Upgraded zlib-1.2.11-18.fc30.x86_64                      @@System
    Upgrade  google-chrome-stable-78.0.3904.87-1.x86_64      @google-chrome
    Upgraded google-chrome-stable-78.0.3904.70-1.x86_64      @@System


-----

After this, dnf was not able to either download system-update packages, or any other "--refresh update"'s.

Once I've now removed

max_parallel_downloads=10

system-upgrade is proceeding just fine with its .rpm downloads.

Seems to me that dnf (or maybe python itself, oh no) has some kind of thread race / SMP / locking / waiting bug that only triggers with a sufficiently high number (>= 10) of parallel downloads. May be still worth fixing, good hygiene and all.

Comment 2 Jason Birch 2019-11-04 00:03:59 UTC
Ah, thank you Kostya!

I ran into this bug on one of four machines, all of which set max parallel to 10 or 20. The machine that ran into this issue was "fixed" by dropping it to 5, but that eventually froze up too — reverting to the default let me slowly get through the entire upgrade.

Comment 3 Ben Cotton 2020-11-03 17:29:38 UTC
This message is a reminder that Fedora 31 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora 31 on 2020-11-24.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
Fedora 'version' of '31'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 31 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 4 Ben Cotton 2021-02-09 16:25:01 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 34 development cycle.
Changing version to 34.

Comment 5 Ben Cotton 2022-05-12 15:08:10 UTC
This message is a reminder that Fedora Linux 34 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora Linux 34 on 2022-06-07.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
'version' of '34'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, change the 'version' 
to a later Fedora Linux version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora Linux 34 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora Linux, you are encouraged to change the 'version' to a later version
prior to this bug being closed.

Comment 6 Ben Cotton 2022-06-07 20:56:54 UTC
Fedora Linux 34 entered end-of-life (EOL) status on 2022-06-07.

Fedora Linux 34 is no longer maintained, which means that it
will not receive any further security or bug fix updates. As a result we
are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release.

Thank you for reporting this bug and we are sorry it could not be fixed.