Bug 2216205 - dnf5 seems to default to skip_if_unavailable=False behavior
Summary: dnf5 seems to default to skip_if_unavailable=False behavior
Keywords:
Status: CLOSED COMPLETED
Alias: None
Product: Fedora
Classification: Fedora
Component: dnf5
Version: 39
Hardware: Unspecified
OS: Linux
high
medium
Target Milestone: ---
Assignee: Evan Goode
QA Contact:
URL:
Whiteboard:
: 2221233 (view as bug list)
Depends On:
Blocks: 2166026
TreeView+ depends on / blocked
 
Reported: 2023-06-20 13:48 UTC by Adam Williamson
Modified: 2024-05-09 10:32 UTC (History)
8 users (show)

Fixed In Version: fedora-release-41-0.10
Clone Of:
Environment:
Last Closed: 2024-05-09 10:32:14 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)

Description Adam Williamson 2023-06-20 13:48:50 UTC
I ran into some openQA failures with the dnf5 switch where tests were failing because dnf was failing due to a repository being unavailable. Kamil Paral reminded me (thanks!) that, for dnf 4, we actually intentionally set skip_if_unavailable=True in /etc/dnf/dnf.conf - overriding the upstream default - because of how commonly people use third-party repositories (e.g. the Chrome or NVIDIA repos) which can quite commonly be "unavailable" (e.g. when upgrading to a new release before the third-party vendor manages to get the repo ready).

For all the main Fedora repos, skip_if_unavailable=False is set in the repo config file to make them "unskippable". But the default at the tool level is, intentionally, True.

It seems like dnf5 has reversed this, and is behaving like 'skip_if_unavailable=False' for all repos. We're worried this will bring back the problems that made us set the Fedora default to True in the first place.

Reproducible: Always

Steps to Reproduce:
1. Configure a repo such that it's "unavailable" (e.g. points to a non-existent URL or directory), and has no explicit skip_if_unavailable setting at the repo level
2. Try and use dnf5
3. Try and use dnf4
Actual Results:  
dnf4 skips the repo and works, but dnf5 fails

Expected Results:  
to be consistent with previous expectations, dnf5 should skip the repo and work.

Comment 1 Jaroslav Mracek 2023-06-26 07:30:46 UTC
This would be surprise, but libdnf5 has the same default like libdnf. The difference is that we stop to ship overrides in /etc/dnf/dnf.conf.

If user would like to have a different default, it is possible to add override to `dnf.conf`. We have a plan to add drop in directory for configuration, therefore other packages can provide override without touching `dnf.conf`.

I am suggesting to document the change in https://dnf5.readthedocs.io/en/latest/changes.html

The reasons why dnf5 uses ``skip_if_unavailable=False` are clear - security. Unavailable repo means that user cannot consume updates from that repository including security updates. The additional issue that dnf is unable to even report that there is a security problem, because metadata are not available.

Additional value is in unifying behavior with other distribution - RHEL.

Comment 2 Kamil Páral 2023-06-26 11:25:33 UTC
Hi Jaroslav, if you intend to stop shipping the override that you shipped before, I think it should be documented in https://fedoraproject.org/wiki/Changes/ReplaceDnfWithDnf5 and this particular point should be discussed by FESCo, because it's a major functionality change that wasn't communicated.

Regarding security, I think this cuts both ways:
* skip_if_unavailable=False is probably good for servers and similar environments with a skilled system administrator. It helps them spot an issue with a remote repository, which might have security implications.
* On the other hand, our experience is that this is a terrible default for desktops, because many user apps install their own repo (e.g. Dropbox, Chrome, etc), they just follow defaults and don't override skip_if_unavailable themselves, and then when those repos are unavailable (the Dropbox repo is unavailable 50% of the time, it's awful), they prevent all system updates. For a general user, who has no administrator skills, this is very bad situation to be in, because it can mean they are running a months old system just because "now that you mention it, the system hasn't offered me an update for a long time, that's interesting". This is why Fedora has defaulted to skip_if_unavailable=True by default, but used skip_if_unavailable=False for all Fedora-provided repos.

That means we either need to continue shipping that override (in dnf or some other package, if dnf starts supporting conf.d/ style config), or we need FESCo to approve this 180° change, or we need some other solution (e.g. it might be reasonable to only ship this override in desktop variants).

Which path DNF wants to take? If to drop the override, can you please take an action so that FESCo is at least informed? Thanks!

Comment 3 Adam Williamson 2023-06-26 20:01:25 UTC
"This would be surprise, but libdnf5 has the same default like libdnf. The difference is that we stop to ship overrides in /etc/dnf/dnf.conf."

Yes. That's why this is filed on bugzilla.redhat.com and not github. The *upstream* default is the same, but the *downstream* - that is, Fedora - default has changed, which - as Kamil explained - is a significant change.

Comment 4 Jaroslav Mracek 2023-06-27 12:17:12 UTC
Let me share with you a part of the plan. The problem is that we talk about default for Fedora, but Fedora has multiple variants and we deliver only one default. In comment 2 Kamil mention different expectation for Server and Workstation and I fully agree.

With DNF5 we would like to change configuration deployment by implementing support of drop-in directory for configuration. Why I mention that here? It allows to distribution (not in dnf downstream) to ship overrides for particular variant. The modification of dnf configuration will be possible by other rpms. This approach has one prerequisite - dnf.conf must be shipped empty and must be reserved for user overrides (manual editing or by dnf config-manager). The feature is still not implemented, but it will be soon. The similar approach we would like to have for repository, but it will be tricky.

The change is an opportunity, but right now it represents some level of discomfort.

Additionally we would like provide a solution for packages that currently adds configuration values to dnf.conf during installation. This approach is bad and risky. dnf.conf supports definition of repository in other section then main, therefore you don't know what you modify.

Comment 5 Kamil Páral 2023-06-27 13:37:15 UTC
That sounds like a reasonable approach. So, to re-iterate, I believe this is significant enough that it should be described in https://fedoraproject.org/wiki/Changes/ReplaceDnfWithDnf5 , it should be handled as part of that change (that probably means announcing "dnf defaults can now differ per variant", also announcing that especially skip_if_unavailable default is going to change unless they act, and asking all variants to implement their preferred overrides probably in their fedora-release-* rpm), and also have a fallback plan ready if something doesn't work out (e.g. still shipping the skip_if_unavailable override globally until the problems are fixed). Does that sound reasonable to you?

Comment 6 Pavla Kratochvilova 2023-07-11 06:45:55 UTC
*** Bug 2221233 has been marked as a duplicate of this bug. ***

Comment 7 Jaroslav Mracek 2023-07-18 05:55:48 UTC
Plan:
1. As a solution of this bug we will deliver support of drop in directories
2. I will create a request on fedora-release package to deliver per variant dnf configuration overrides
3. When the support of drop-in overrides is available, I will announce it on Fedora devel. I do not feel comfortable with modification of system-wide change because I got negative feedback when I did it in past.

Comment 8 Jaroslav Mracek 2023-07-20 10:58:21 UTC
PR with support of drop-in directories - https://github.com/rpm-software-management/dnf5/pull/741

Comment 9 Fedora Release Engineering 2023-08-16 08:11:14 UTC
This bug appears to have been reported against 'rawhide' during the Fedora Linux 39 development cycle.
Changing version to 39.

Comment 10 Jan Kolarik 2023-12-01 14:15:31 UTC
The drop-in directories support was eventually delivered in the https://github.com/rpm-software-management/dnf5/pull/813.

Comment 11 Jan Kolarik 2024-02-28 08:56:44 UTC
Currently, remaining tasks there seem to be just to create a request on fedora-release package to deliver per variant dnf configuration overrides. Documentation of the new behavior should be covered upstream under the https://github.com/rpm-software-management/dnf5/issues/1282.

Comment 12 Evan Goode 2024-04-15 19:05:50 UTC
fedora-release PR: https://src.fedoraproject.org/rpms/fedora-release/pull-request/331

Comment 13 Evan Goode 2024-04-18 16:57:31 UTC
I've closed fedora-release#331 in favor of https://src.fedoraproject.org/rpms/fedora-release/pull-request/332. For now we are proposing to just set skip_if_unavailable=True for _all variants_, not just Fedora Workstation, as it was in DNF 4.

We may propose to change the default to skip_if_unavailable=True for some (or all) variants in the future, but such a change should go through the Fedora change proposal process.

Comment 14 Kamil Páral 2024-04-19 08:09:34 UTC
That sounds reasonable.


Note You need to log in before you can comment on or make changes to this bug.