Fedora Account System
Red Hat Associate
Red Hat Customer
Description of problem: After installing normal Fedora 44 updates through KDE Discover and rebooting for the offline update, Discover reported: Failed Offline Update Failed to update 0 packages Cannot load libdnf plugin enabled from: /etc/dnf/libdnf5-plugins/00-systemd-inhibit.conf The update upgraded DNF5/libdnf5 from 5.4.3.0-2.fc44 to 5.4.4.0-1.fc44 and installed the new weak dependency: libdnf5-plugin-systemd-inhibit-5.4.4.0-1.fc44.x86_64 This came from the normal Fedora "updates" repository. updates-testing is disabled. Despite PackageKit reporting that the offline update failed, the package transaction appears to have completed successfully. `dnf5 history info last` reports: Status : Ok The offline-update journal also shows the transaction reaching 100% and successfully updating/installing the packages before PackageKit records the failure: package installing libdnf5-plugin-systemd-inhibit-5.4.4.0-1.fc44.x86_64 (updates) percentage 100% ... update-packages transaction ... finished with failed failed to update system: Cannot load libdnf plugin enabled from: /etc/dnf/libdnf5-plugins/00-systemd-inhibit.conf After the subsequent normal boot, DNF5 and PackageKit both work correctly. These commands complete successfully: dnf5 repo list --enabled pkcon get-updates rpm -V libdnf5-plugin-systemd-inhibit `pkcon get-updates` reports no updates available and the current boot contains no PackageKit/libdnf/systemd-inhibit errors. Version information: Fedora Linux 44 KDE Architecture: x86_64 PackageKit-1.3.6-1.fc44.x86_64 PackageKit-glib-1.3.6-1.fc44.x86_64 dnf5-5.4.4.0-1.fc44.x86_64 libdnf5-5.4.4.0-1.fc44.x86_64 libdnf5-plugin-systemd-inhibit-5.4.4.0-1.fc44.x86_64 Steps to reproduce: 1. Have a Fedora 44 KDE installation with DNF5/libdnf5 5.4.3.0-2.fc44. 2. Install the current Fedora updates through KDE Discover. 3. Allow Discover to perform its offline update during reboot. 4. The transaction upgrades DNF5/libdnf5 to 5.4.4.0-1.fc44 and installs libdnf5-plugin-systemd-inhibit as a weak dependency. 5. After returning to the desktop, Discover reports that the offline update failed with the systemd-inhibit.conf plugin-loading error. Actual result: Discover/PackageKit reports an offline-update failure even though the RPM/DNF transaction appears to have completed successfully. Expected result: The offline update should complete without PackageKit reporting a false failure. Additional information: Multiple Fedora 44 users have independently reported the same error shortly after this update reached the stable Fedora 44 updates repository. The failure appears to occur during the transition to DNF5 5.4.4 and the newly installed systemd-inhibit plugin. I do not know whether the underlying cause is PackageKit retaining the previous libdnf5 in the running process or another plugin-loading/integration issue, so that part is only a hypothesis. Reproducible: Always Steps to Reproduce: 1.discover update 2. 3. Actual Results: Received error after reboot Expected Results: Successful update
Upstream report: https://github.com/PackageKit/PackageKit/issues/1012
Plugin initialization is done by libdnf5, not PackageKit. So the issue belongs with dnf5. Moving.
> Steps to reproduce: > > 1. Have a Fedora 44 KDE installation with DNF5/libdnf5 5.4.3.0-2.fc44. > 2. Install the current Fedora updates through KDE Discover. I don't know what this step does. Does it install something, or not? > 3. Allow Discover to perform its offline update during reboot. > 4. The transaction upgrades DNF5/libdnf5 to 5.4.4.0-1.fc44 and installs > libdnf5-plugin-systemd-inhibit as a weak dependency. > 5. After returning to the desktop, Discover reports that the offline update > failed with the systemd-inhibit.conf plugin-loading error. What do you mean with "returning to the desktop"? There is no desktop to be returned to. An off-line update reboots the machine. There is a new desktop running with a new Discover process with a new libdnf5 and new libdnf5 plugins. I frankly do not know what Discover does and why it reports a failure. That needs to be explained by a Discover maintainer.
Thank you. I can clarify the reproduction steps. In step 2, by “Install the updates through KDE Discover” I mean using Discover to download/prepare the available updates and choosing its restart/install-offline-update workflow. The packages are not actually installed at the desktop at that point. The actual installation then occurs during the offline boot through PackageKit’s packagekit-offline-update.service / pk-offline-update. By “after returning to the desktop” in step 5, I mean after the offline updater has completed and rebooted the machine back into the normal graphical session. I understand that this is then a new Discover process using the newly installed libraries. However, the failure itself is recorded before that reboot, during the offline update. The journal from a clean Fedora 44 KDE reproduction shows: percentage 100% ... status finished ... update-packages transaction ... finished with failed writing failed results failed to update system: Cannot load libdnf plugin enabled from: /etc/dnf/libdnf5-plugins/00-systemd-inhibit.conf ... rebooting So Discover appears to be displaying the persisted PackageKit offline-update failure result after login, rather than generating a new plugin-loading failure from the new Discover process. I also reproduced this on a completely fresh stock Fedora 44 KDE installation. That installation started with libdnf5 5.4.1 rather than 5.4.3, so the reproduction does not appear to depend specifically on 5.4.3 → 5.4.4. On that clean KDE reproduction, the journal also shows the new plugin itself being successfully installed: op=install sw="libdnf5-plugin-systemd-inhibit-5.4.4.0-1.fc44.x86_64" ... res=success followed afterward by the plugin-loading failure above. For comparison, I also performed fresh Fedora 44 GNOME installations that ended up with the same dnf5, libdnf5, and libdnf5-plugin-systemd-inhibit 5.4.4 packages. Those updates completed through dnf5daemon-server without this failure.
I think the issue is here: https://github.com/PackageKit/PackageKit/blob/cf89311412a109098d186b238793368907c48d95/backends/dnf5/dnf5-backend-thread.cpp#L565 dnf5_setup_base() runs after trans.run() and creates a new libdnf5::Base. Its Base::setup() rescans enabled plugin configuration and loads plugins from disk. In this transaction, libdnf5 was upgraded to 5.4.4 and the new systemd-inhibit plugin, including its enabled configuration file, was installed as a weak dependency. The plugin uses IPlugin2_2, an API introduced by libdnf5 5.4.4. However, PackageKit is still the same process and still has the pre-5.4.4 libdnf5.so.2 loaded in memory. Recreating Base does not reload that shared library. Consequently, the new plugin is loaded against the old in-memory libdnf5 and fails because IPlugin2_2 is unavailable. Therefore, reinitializing plugins after a transaction that replaced libdnf5 or installed libdnf5 plugins is unsafe unless the backend process is restarted first.
That's a good explanation, but I still don't understand how that can happen during off-line update. At off-line update there is (should be) no PackageKit process running. Only DNF5. I suspect that the reporter actually installs new libdnf5 with the new plugin on-line, not off-line. Or the PackageKit has preinstalled some libdnf5 plugin which is loaded anytime when DNF5 is executed, including an off-line update. Nevertheless, libdnf5 should not reload plugins exactly for the problem Marek described.
The answer might be `/usr/lib/systemd/system/packagekit-offline-update.service`. PackageKit seems to have its own pk-offline-update process that handles offline upgrades.
(In reply to Marek Blaha from comment #5) > I think the issue is here: > > https://github.com/PackageKit/PackageKit/blob/ > cf89311412a109098d186b238793368907c48d95/backends/dnf5/dnf5-backend-thread. > cpp#L565 > > dnf5_setup_base() runs after trans.run() and creates a new libdnf5::Base. > Its Base::setup() rescans enabled plugin configuration and loads plugins > from disk. > > In this transaction, libdnf5 was upgraded to 5.4.4 and the new > systemd-inhibit plugin, including its enabled configuration file, was > installed as a weak dependency. The plugin uses IPlugin2_2, an API > introduced by libdnf5 5.4.4. > > However, PackageKit is still the same process and still has the pre-5.4.4 > libdnf5.so.2 loaded in memory. Recreating Base does not reload that shared > library. Consequently, the new plugin is loaded against the old in-memory > libdnf5 and fails because IPlugin2_2 is unavailable. > > Therefore, reinitializing plugins after a transaction that replaced libdnf5 > or installed libdnf5 plugins is unsafe unless the backend process is > restarted first. Would it be reasonable to just drop that part?
(In reply to Neal Gompa from comment #8) > > Therefore, reinitializing plugins after a transaction that replaced libdnf5 > > or installed libdnf5 plugins is unsafe unless the backend process is > > restarted first. > > Would it be reasonable to just drop that part? Either drop the post-transaction Base reinitialization, or treat failures during the Base refresh separately from failures of the transaction itself. Could PackageKit mark the DNF5 backend as requiring a service restart before accepting another request? I also considered handling this selectively in libdnf5, but there is a chicken-and-egg problem. The failure can occur during dlopen(plugin.so), before libdnf5 can call the plugin’s exported libdnf_plugin_get_api_version() function. Therefore, libdnf5 cannot reliably determine that this particular loading failure means “the plugin requires a newer API.” Supporting that would require advertising the required API version before dlopen(), e.g. "required_api_version=2.2" This could be stored in the plugin configuration or, preferably, in package-owned plugin metadata. An older libdnf5 could then skip only plugins requiring a newer API while keeping other plugin-loading failures fatal. Of course it would address only API version incompatibility and could not make arbitrary in-process plugin reloads after updating libdnf5 generally safe.
> I also considered handling this selectively in libdnf5, but there is a chicken-and-egg problem. My idea was adding a global variable to libdnf5, allowing loading the plugins only once. Effectively enabling plugins only in the first instance of libdnf5::Base in a process. Alternatively there could be a new Base::setup(, libdnf5::Base &old_base) which PackageKit would use to copy the iplugins from old_base into this Base instead of loading the plugins again.
(In reply to Marek Blaha from comment #9) > > Could PackageKit mark the DNF5 backend as requiring a service restart before > accepting another request? > Doing this would make PackageKit extremely slow, because it would do it for every request of any kind. I would really like to avoid that.
At least, I don't think it's possible to only trigger a restart for transaction requests and not require it for everything else.