Bug 1775184

Summary: dnf/yum commands fail if fails if mirrorlist cannot be found
Product: Red Hat Enterprise Linux 8 Reporter: Paul Dwyer <pdwyer>
Component: dnfAssignee: amatej
Status: CLOSED ERRATA QA Contact: Luca Berton <lberton>
Severity: medium Docs Contact:
Priority: medium    
Version: 8.1CC: amatej, fedoraproject.org, james.antill, lberton, mdomonko, nsella, pdwyer
Target Milestone: rcKeywords: Triaged
Target Release: 8.0Flags: pm-rhel: mirror+
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: libdnf-0.48.0-1.el8 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-11-04 01:52:20 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1825061    

Description Paul Dwyer 2019-11-21 14:32:22 UTC
Description of problem:
With both mirrorlist and baseurl defined in a repo file, if the mirrorlist cannot be found it does not fall back to baseurl.

This worked on previous RHEL major versions (5/6/7)

Version-Release number of selected component (if applicable):
dnf-4.2.7-6.el8.noarch

How reproducible:
Always

Steps to Reproduce:

Create custom repo file, create a mirrorlist that is not available

[RHEL-8-20191106_3]
#BaseOS
name=RHEL-8-20191106_3
#fake mirror list to replicate issue
mirrorlist=http://192.168.122.1:8080/BaseOSXX
enabled=1
baseurl=http://192.168.122.1:8080/BaseOS
gpgcheck=0
repo_gpgcheck=1
gpgkey=http://192.168.122.1:8080/BaseOS/my.key

running yum/dnf commands

# dnf  -v --disablerepo=* --enablerepo=RHEL-8-20191106_3 list
Loaded plugins: builddep, changelog, config-manager, copr, debug, debuginfo-install, download, generate_completion_cache, needs-restarting, playground, product-id, repoclosure, repodiff, repograph, repomanage, reposync, uploadprofile
DNF version: 4.2.7
cachedir: /var/cache/dnf
repo: downloading from remote: RHEL-8-20191106_3
error: Status code: 404 for http://192.168.122.1:8080/BaseOSXX (http://192.168.122.1:8080/BaseOSXX).
RHEL-8-20191106_3                                22 kB/s | 195  B     00:00    
Cannot download 'http://192.168.122.1:8080/BaseOSXX': Cannot prepare internal mirrorlist: Status code: 404 for http://192.168.122.1:8080/BaseOSXX.
Failed to download metadata for repo 'RHEL-8-20191106_3'
Error: Failed to download metadata for repo 'RHEL-8-20191106_3'


Actual results:
cannot find mirrorlist so fails to download repo metadata but basurl (which is available) is never tried


Expected results:
Fall back to basurl if mirrorlist is unavailable similar to what happened on previous RHEL releases

behaviour on RHEL7
# yum --disablerepo=* --enablerepo=myrepo list | more
Loaded plugins: boks-protect, changelog, fastestmirror, langpacks, tmprepo,
              : verify, versionlock
Determining fastest mirrors
Could not retrieve mirrorlist http://url/myrepoXX error was
14: HTTP Error 404 - Not Found
 * myrepo
Installed Packages
GeoIP.x86_64                           1.5.0-13.el7             @repo
NetworkManager.x86_64                  1:1.12.0-10.el7_6        @repo
NetworkManager-config-server.noarch    1:1.12.0-10.el7_6        @repo
NetworkManager-libnm.x86_64            1:1.12.0-10.el7_6        @repo


Additional info:

Comment 3 amatej 2020-03-09 07:26:42 UTC
I have created PRs to fix this:
https://github.com/rpm-software-management/librepo/pull/184
https://github.com/rpm-software-management/libdnf/pull/911

And some tests:
https://github.com/rpm-software-management/ci-dnf-stack/pull/804
(the second test will pass even with the current version, but not because mirrorlist is preferred over baseurl, but because libdnf doesn't even pass the baseurl to librepo, my PR fixes this)

Comment 4 amatej 2020-04-06 12:49:49 UTC
Proposing AC:
- When dnf is using a repository with both baseurl and mirrorlist specified and the mirrorlist file cannot be found, baseurl is used as a fallback.
- When dnf is using a repository with both baseurl and mirrorlist specified and all the mirrorlist urls are invalid, baseurl is used as a fallback.

Comment 6 amatej 2020-04-21 12:07:47 UTC
I have realized my previous PR wasn't sufficient so here is a second part:
https://github.com/rpm-software-management/librepo/pull/189
Also extended the tests PR with new missed cases (mirrorlist file cannot be found):
https://github.com/rpm-software-management/ci-dnf-stack/pull/804

Comment 12 errata-xmlrpc 2020-11-04 01:52:20 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (yum bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2020:4510