Bug 2124511

Summary: [dnf5] Add dedicated function to refresh metadata
Product: [Fedora] Fedora Reporter: Milan Crha <mcrha>
Component: dnf5Assignee: rpm-software-management
Status: CLOSED EOL QA Contact:
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 40CC: daniel.mach, jkolarik, jmracek, jrohel, mblaha, mpitt, nsella, pkratoch, ppisar, rpm-software-management
Target Milestone: ---Keywords: Triaged
Target Release: ---Flags: tpopela: mirror+
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2025-05-16 07:40:24 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
dnfdaemon D-Bus API test for checking force refresh API none

Description Milan Crha 2022-09-06 11:47:10 UTC
The gnome-software has a dedicated function to refresh metadata, it's usually ran on start of the application, but can be also executed when needed. The function takes one argument, the `cache_age` in seconds. Caches not being that old are automatically skipped for the refresh. An explicit refresh can help to speed up execution of the following commands, the same as when the refresh happens during the dedicated time it's much better experience for the users.

A side note: the first time I ran the dnf5daemon-server and connected to it to list installed packages the dnf5daemon-server realized it has no/stale cache and begun to refresh the repository information. I have installed quite few repos, thus it took longer to download all the repo information. It took as much as the D-Bus call timed out. That led me to set unlimited time for the D-Bus calls to the dnf5daemon-server to avoid any such problem in the future.

Comment 1 Ben Cotton 2023-02-07 14:54:41 UTC
This bug appears to have been reported against 'rawhide' during the Fedora Linux 38 development cycle.
Changing version to 38.

Comment 2 Martin Pitt 2023-07-24 05:03:19 UTC
I just tested dnfdaemon again in https://issues.redhat.com/browse/COCKPIT-952 , and am happy to report that all crashes that I previously ran into now got fixed, thanks! So this missing API is the remaining gap which blocks us from moving to dnfdaemon in Cockpit.

Comment 3 Milan Crha 2023-07-25 10:30:33 UTC
It seems the 5.1.0 has `read_all_repos` method on the `org.rpm.dnf.v0.Base` interface. It does not contain the "cache_age" argument (in fact it has no argument at all), but if it's the way to go, then why not.

I think this can be closed, unless someone is willing to add the "cache_age" argument.

Comment 4 Martin Pitt 2023-12-07 16:28:56 UTC
Neither the .Base interface nor the read_all_repos() method are on https://dnf5.readthedocs.io/en/latest/dnf_daemon/dnf5daemon_dbus_api.8.html . On current Fedora rawhide with dnf5-5.1.8-1.fc40.x86_64 there's only the .SessionManager:

# busctl introspect org.rpm.dnf.v0 /org/rpm/dnf/v0
NAME                                TYPE      SIGNATURE RESULT/VALUE FLAGS
org.freedesktop.DBus.Introspectable interface -         -            -
.Introspect                         method    -         s            -
org.freedesktop.DBus.Peer           interface -         -            -
.GetMachineId                       method    -         s            -
.Ping                               method    -         -            -
org.freedesktop.DBus.Properties     interface -         -            -
.Get                                method    ss        v            -
.GetAll                             method    s         a{sv}        -
.Set                                method    ssv       -            -
.PropertiesChanged                  signal    sa{sv}as  -            -
org.rpm.dnf.v0.SessionManager       interface -         -            -
.close_session                      method    o         b            -
.open_session                       method    a{sv}     o            -


I suspect that is an API on the session objects, but I don't know of any way to introspect that on the CLI. I naïvely tried to call it on the session object, and it does succeed, but takes no measurable time -- it can't possibly refresh the indexes in the blink of an eye. So I suppose this is async somehow? It doesn't have a .resolve() method though. I'm afraid this needs to be documented.

Comment 5 Martin Pitt 2023-12-08 06:13:04 UTC
I also tried to call Base.read_all_repos() on the *same* session object before calling rpm.Rpm.list() on it, just in case it works more like an option than a transaction. Same outcome though, no failure but also no effect.

Comment 6 Martin Pitt 2023-12-08 06:33:35 UTC
Created attachment 2003302 [details]
dnfdaemon D-Bus API test for checking force refresh API

This is my current Python script to exercise the D-Bus dnfdaemon API. The "===========  force refreshing" part succeeds, but it does way too fast, i.e. it doesn't actually do anything. After "dnf5 clean all" it hangs for a while (presumably downloading metadata), but it doesn't send out any signals during that which would indicate progress. Is that a bug, or am I calling it wrong?

Thanks!

Comment 7 Aoife Moloney 2024-05-07 15:48:32 UTC
This message is a reminder that Fedora Linux 38 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora Linux 38 on 2024-05-21.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
'version' of '38'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, change the 'version' 
to a later Fedora Linux version. Note that the version field may be hidden.
Click the "Show advanced fields" button if you do not see it.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora Linux 38 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora Linux, you are encouraged to change the 'version' to a later version
prior to this bug being closed.

Comment 8 Milan Crha 2024-08-27 16:50:54 UTC
A bit of additional info relevant to the current dnf5daemon-server code available here:
https://github.com/rpm-software-management/dnf5/issues/1659

Comment 9 Jan Kolarik 2025-03-26 09:56:50 UTC
The functionality to reset the current dnf5daemon session has been available since the fix for https://github.com/rpm-software-management/dnf5/issues/1653. Does this sufficiently address the issue, or is there any additional behavior that needs to be implemented in dnf5daemon?

Comment 10 Martin Pitt 2025-03-26 10:38:43 UTC
So what is the API to force refreshing of the metadata now? My previous reproducer called .read_all_repos(), which succeeds but doesn't refresh. I changed that to .reset() but that behaves the same -- succeeds, but it's way too fast to actually refresh metadata. I also tried calling both, no difference.

But also, https://github.com/rpm-software-management/dnf5/pull/1757 was about something else? Re-using a session object to do more than one operation. Indeed that's highly useful (it is/was annoying to create a new session for everything), but this is unrelated to forcing a re-download of archive metadata.

Comment 11 Jan Kolarik 2025-03-26 11:33:07 UTC
You’re right that the API is currently missing in dnf5daemon. The question is, what is your specific intention with cache refreshing? I recall discussing with Milan the primary use case of reloading the system repository after executing a transaction. When the base is reset, all repositories are dropped and can be reloaded again. However, you’re correct that the cached metadata cannot be explicitly reloaded and still follows the "metadata_expire" value from the configuration.

Comment 12 Milan Crha 2025-03-27 07:16:14 UTC
Right, this is to have `dnf update --refresh` equivalent. In case of the gnome-software, the user can click the "Refresh" button on the Updates page, when he/she expects to have the update information updated, to see new packages for update, if such exist and were not shown before due to "stale" local cache.

This one is related: https://github.com/rpm-software-management/dnf5/pull/2055 Once done, if approved, or a similar change, the "clear("expire-cache")", followed by "read_all_repos()", should do the trick (needs to be tested).

Comment 13 Martin Pitt 2025-03-27 07:29:48 UTC
Similar use case for Cockpit, which also has a "refresh" button. If dnf5 isn't meant to have this functionality any more, we could disable it for dnf5 of course, and just keep it for dnf 4/apt/etc. So this isn't a strong requirement, more of a "we always used to have this" from our PoV. Thanks!

Comment 14 Aoife Moloney 2025-04-25 10:03:33 UTC
This message is a reminder that Fedora Linux 40 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora Linux 40 on 2025-05-13.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
'version' of '40'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, change the 'version' 
to a later Fedora Linux version. Note that the version field may be hidden.
Click the "Show advanced fields" button if you do not see it.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora Linux 40 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora Linux, you are encouraged to change the 'version' to a later version
prior to this bug being closed.

Comment 15 Aoife Moloney 2025-05-16 07:40:24 UTC
Fedora Linux 40 entered end-of-life (EOL) status on 2025-05-13.

Fedora Linux 40 is no longer maintained, which means that it
will not receive any further security or bug fix updates. As a result we
are closing this bug.

If you can reproduce this bug against a currently maintained version of Fedora Linux
please feel free to reopen this bug against that version. Note that the version
field may be hidden. Click the "Show advanced fields" button if you do not see
the version field.

If you are unable to reopen this bug, please file a new report against an
active release.

Thank you for reporting this bug and we are sorry it could not be fixed.