Created attachment 1006301 [details] reproducer script Description of problem: (using rpmfusion repos) VirtualBox needs kernel modules to run. These modules are installed via meta package kmod-VirtualBox, which has kmod-VirtualBox-<kernelversion> (kernel version is part of the name) as dependency and this dependent package requires given kernel version. When three kernels are present on system (along with VirtualBox modules) and there is a kernel update, the oldest kernel (if not currently running) gets removed (and corresponding kmod-VirtualBox-<kernelversion> too). This is expected behavior and it is exactly what 'yum update' does. However, offline updates, which uses 'pkcon get-updates' (triggered from console or from gnome software center) ignores the kernel update for some reason. Everything gets updated, but kernel. Having VirtualBox installed and updating via offline updates (e.g. gnome software center) means you never receive newer kernels, but stay on the old ones all the time. That means you don't receive even security updates of the kernel. Version-Release number of selected component (if applicable): Fedora 21 Live PackageKit-1.0.4-1.fc21.x86_64 gnome-packagekit-3.14.2-1.fc21.x86_64 libsolv-0.6.4-3.fc21.x86_64 libhif-0.1.8-4.fc21.x86_64 librepo-1.7.13-1.fc21.x86_64 yum-3.4.3-153.fc21.noarch gnome-software-3.14.3-1.fc21.x86_64 I created shell script mimicking the VirtualBox packages to reproduce this on F21 Live installation. Its attached here. Simulating this with VirtualBox packages is much harder to reproduce, that's why I used fake packages to show this issue. Steps to Reproduce: 1. install Fedora F21 Live 2. boot it and download the reproducer script mentioned 3. run the script as root (and have a coffee since it's updating the system) 4. reboot (make sure to boot the latest kernel) 5. pkcon refresh force 6. compare output of the following: pkcon get-updates yum update gnome-software (Updates tab) 7. to-be-updated packages are different, yum wants to update the kernel, but pkcon (and gnome-software) does not Example of 'pkcon get-updates' output: Getting updates [=========================] Loading cache [=========================] Finished [=========================] Available devassistant-0.9.3-4.fc21.noarch (updates) DevAssistant - Making life easier for developers Available kmod-foo-0.1-2.x86_64 (foo) Dummy summary Example of 'yum update' output: ================================================================================ Package Arch Version Repository Size ================================================================================ Installing: kernel x86_64 3.19.1-201.fc21 updates 48 k kernel-core x86_64 3.19.1-201.fc21 updates 19 M kernel-modules x86_64 3.19.1-201.fc21 updates 17 M Updating: devassistant noarch 0.9.3-4.fc21 updates 306 k kmod-foo x86_64 0.1-2 foo 5.6 k Removing: kernel x86_64 3.17.4-301.fc21 @koji-override-0/$releasever 0.0 kernel-core x86_64 3.17.4-301.fc21 @koji-override-0/$releasever 40 M kernel-modules x86_64 3.17.4-301.fc21 @koji-override-0/$releasever 17 M Installing for dependencies: foo-3.19.1-201 x86_64 0.1-2 foo 5.7 k Removing for dependencies: foo-3.17.4-301 x86_64 0.1-1 @foo 0.0
I can confirm this issue, I've seen this for weeks on my F21 home laptop. I have VirtualBox installed there, and offline upgrades never upgrade my kernel (nor virtualbox modules), I always have to do it manually by using yum. This has been happening every week for the past ~two months. I asked Lukas to reproduce this with rpmfluff-made packages, because reproducing this with real virtualbox packages is difficult - it depends on the current state of two repos (fedora updates and rpmfusion updates) and sometimes they are not in sync. Rpmfluff allowed us to make the reproducer easy. There's one tiny difference though, this line in reproducer should use '=' instead of '>=' to mimick the virtualbox behavior exactly: foo2.add_requires("kernel-uname-r >= 3.19.1-201.fc21.x86_64") However, because by the time you try this, there can be a newer kernel update in updates repo, we settled for '>=' to make sure this reproducer still works in the future.
Does DNF handle it correctly?
Yep, DNF handles it correctly.
Created attachment 1006336 [details] Fedora 22 reproducer
I was able to reproduce this on Fedora 22 installed from 22 Alpha RC3 Live Workstation. Steps are the same as above, just use the 22 reproducer I attached. Version-Release number of selected component (if applicable): Fedora 22 Live Alpha RC3 PackageKit-1.0.5-1.fc22.x86_64 librepo-1.7.13-1.fc22.x86_64 libhif-0.1.8-5.fc22.x86_64 librepo-1.7.13-1.fc22.x86_64 yum-3.4.3-155.fc22.noarch gnome-software-3.14.3-1.fc21.x86_64
Proposed as a Blocker for 22-final by Fedora user lbrabec using the blocker tracking app because: I propose this as final blocker as it violates the (beta) criterion: Updates The installed system must be able to download and install updates with the default graphical package manager in all release-blocking desktops. Although its beta criterion, I propose this as final blocker - it is affected by only one package.
Not that my opinion counts for much, but I'm -1 to consider this a blocker, if it's only triggered by 3rd-party/non-fedora content. Is it reproducible from content from fedora-only repos?
Lukas, Richard asked for packagekitd --verbose log, can you please supply it? Thanks. (In reply to Rex Dieter from comment #7) > Not that my opinion counts for much, but I'm -1 to consider this a blocker, > if it's only triggered by 3rd-party/non-fedora content. I assume this could also be triggered by binary nvidia drivers, they use a similar structure to virtualbox packages. But we haven't tested it yet, it's more difficult to reproduce this with real packages than with fake ones, due to changing repo states. > > Is it reproducible from content from fedora-only repos? We will try to find out.
Created attachment 1006706 [details] packagekitd --verbose log
I was able to reproduce this with nvidia drivers from rpmfusion. Version-Release number of selected component (if applicable): Fedora 21 kernel-3.17.7-300.fc21.x86_64 kernel-3.18.8-201.fc21.x86_64 kernel-3.18.9-200.fc21.x86_64 kernel-core-3.17.7-300.fc21.x86_64 kernel-core-3.18.8-201.fc21.x86_64 kernel-core-3.18.9-200.fc21.x86_64 kernel-modules-3.17.7-300.fc21.x86_64 kernel-modules-3.18.8-201.fc21.x86_64 kernel-modules-3.18.9-200.fc21.x86_64 (downloaded and installed manually to ensure correct version) kmod-nvidia-3.17.7-300.fc21.x86_64-343.36-1.fc21.1.x86_64 kmod-nvidia-343.36-1.fc21.1.x86_64 xorg-x11-drv-nvidia-343.36-1.fc21.x86_64 xorg-x11-drv-nvidia-libs-343.36-1.fc21.x86_64 yum update output: ================================================================================ Package Arch Version Repository Size ================================================================================ Installing: kernel x86_64 3.19.1-201.fc21 updates 48 k kernel-core x86_64 3.19.1-201.fc21 updates 19 M kernel-modules x86_64 3.19.1-201.fc21 updates 17 M Updating: devassistant noarch 0.9.3-4.fc21 updates 306 k kmod-nvidia x86_64 1:346.47-2.fc21.1 rpmfusion-nonfree-updates 45 k xorg-x11-drv-nvidia x86_64 1:346.47-1.fc21 rpmfusion-nonfree-updates 4.0 M xorg-x11-drv-nvidia-libs x86_64 1:346.47-1.fc21 rpmfusion-nonfree-updates 16 M Removing: kernel x86_64 3.17.7-300.fc21 @/kernel-3.17.7-300.fc21.x86_64 0.0 kernel-core x86_64 3.17.7-300.fc21 @/kernel-core-3.17.7-300.fc21.x86_64 40 M kernel-modules x86_64 3.17.7-300.fc21 @/kernel-modules-3.17.7-300.fc21.x86_64 17 M Installing for dependencies: kmod-nvidia-3.19.1-201.fc21.x86_64 x86_64 1:346.47-2.fc21.1 rpmfusion-nonfree-updates 3.7 M Removing for dependencies: kmod-nvidia-3.17.7-300.fc21.x86_64 x86_64 1:343.36-1.fc21.1 @/kmod-nvidia-3.17.7-300.fc21.x86_64-343.36-1.fc21.1.x86_64 15 M pkcon get-updates output: Getting updates [=========================] Starting [=========================] Finished [=========================] Available devassistant-0.9.3-4.fc21.noarch (updates) DevAssistant - Making life easier for developers Available kmod-nvidia-1:346.47-2.fc21.x86_64 (rpmfusion-nonfree-updates) Metapackage which tracks in nvidia kernel module for newest kernel Available xorg-x11-drv-nvidia-1:346.47-1.fc21.x86_64 (rpmfusion-nonfree-updates) NVIDIA's proprietary display driver for NVIDIA graphic cards Available xorg-x11-drv-nvidia-libs-1:346.47-1.fc21.x86_64 (rpmfusion-nonfree-updates) Libraries for xorg-x11-drv-nvidia
Created attachment 1006785 [details] packagekitd --verbose (with nvidia drivers)
I didn't find any packages from fedora repo that could cause this problem. Packages from rpmfusion that will cause this problem: kmod-VirtualBox-3.17.4-301.fc21.x86_64 kmod-ndiswrapper-3.17.4-301.fc21.x86_64 kmod-nvidia-3.17.4-301.fc21.x86_64 kmod-nvidia-304xx-3.17.4-301.fc21.x86_64 kmod-staging-3.17.4-301.fc21.x86_64 kmod-wl-3.17.4-301.fc21.x86_64 kmod-xtables-addons-3.17.4-301.fc21.x86_64
(In reply to Lukas Brabec from comment #12) > Packages from rpmfusion that will cause this problem: > > kmod-VirtualBox-3.17.4-301.fc21.x86_64 > kmod-ndiswrapper-3.17.4-301.fc21.x86_64 > kmod-nvidia-3.17.4-301.fc21.x86_64 > kmod-nvidia-304xx-3.17.4-301.fc21.x86_64 > kmod-staging-3.17.4-301.fc21.x86_64 > kmod-wl-3.17.4-301.fc21.x86_64 > kmod-xtables-addons-3.17.4-301.fc21.x86_64 To be exact, we haven't tested all of them, just virtualbox and nvidia. But these are likely to exhibit the same problem, because they require a particular kernel version and have the same dependency structure (a triangle of kernel <-> metapackage foo <-> package foo-kernel-version).
Discussed at 2015-03-30 blocker review meeting: http://meetbot.fedoraproject.org/fedora-blocker-review/2015-03-30/f22-blocker-review.2015-03-30-16.04.log.txt . This is clearly an unfortunate bug, but it's triggered by a dependency structure which cannot exist in the Fedora repos by policy (kernel modules outside of the kernel .src.rpm are not allowed), and seems therefore very strongly a 3rd party repo issue. Fedora has a strong stance of not 'supporting' 3rdparty stuff like this, so we couldn't justify accepting it as a release blocker. It also appears to be possible to fix it with an update, as the update mechanism itself is not broken. It's therefore rejected as a blocker, but accepted as a freeze exception issue.
Hi, unfortunately we can't count on rpmfusion repo because we don't have it for F22 please read [1] for more information [1] https://ask.fedoraproject.org/en/question/54642/rpmfusion-on-f22-rawhide/?comment=66707#comment-66707
*** Bug 1204533 has been marked as a duplicate of this bug. ***
Please note that in bug 1204533 an independent reporter has confirmed that kmod-wl packages cause the same issue as virtualbox and nvidia packages.
I debugged this today and https://github.com/hughsie/PackageKit/pull/64 should have the fixes needed to make PackageKit match DNF behaviour with kernel updates.
(In reply to Kalev Lember from comment #18) > I debugged this today and https://github.com/hughsie/PackageKit/pull/64 > should have the fixes needed to make PackageKit match DNF behaviour with > kernel updates. Awesome news, thanks !
Created attachment 1036315 [details] Fedora 22 reproducer version 2 Kalev, thanks a lot. I've found out that the original script broke, because some bodhi updates got deleted and some koji builds as well. I have prepared an updated script for F22 that will hopefully last longer. I actually prepared it last week, but I was waiting for more kernel builds to appear in F22. That hasn't happened, so I just used one from F23 (but it should work the same), and I'm attaching the script now. It is super-easy to verify this bug with this script. Just install F22 workstation *Live* (has to be Live, because netinst would get you updated packages) into a VM, copy the reproducer script inside and run it as root. It will do everything necessary, then just reboot (necessary, to switch running kernel) and compare dnf vs packagekit. This is the current output: [root@localhost ~]# dnf clean all && dnf check-update Cleaning repos: foo Cleaning up Everything Foo Repo 141 MB/s | 773 kB 00:00 Last metadata expiration check performed 0:00:00 ago on Mon Jun 8 13:20:02 2015. kernel.x86_64 4.1.0-0.rc6.git2.1.fc23 foo kernel-core.x86_64 4.1.0-0.rc6.git2.1.fc23 foo kernel-modules.x86_64 4.1.0-0.rc6.git2.1.fc23 foo kmod-foo.x86_64 1-2 foo [root@localhost ~]# pkcon refresh force && pkcon get-updates Refreshing cache [=========================] Waiting for authentication [=========================] Finished [=========================] Getting updates [=========================] Finished [=========================] There are no updates available at this time. If you update packagekit, you can then test whether the functionality is fixed (or I'll do it, once there's a new F22 build for PK).
Thanks! I've just kicked off F22 builds with the fixes - testing would be much appreciated.
PackageKit-1.0.6-6.fc22 has been submitted as an update for Fedora 22. https://admin.fedoraproject.org/updates/PackageKit-1.0.6-6.fc22
(In reply to Kalev Lember from comment #21) > Thanks! I've just kicked off F22 builds with the fixes - testing would be > much appreciated. Well, we will need some time to verify this fixed updating from rpmfusion, but it definitely works now with our reproducer script, which is pretty awesome :o) [root@localhost ~]# pkcon refresh force && pkcon get-updates Refreshing cache [=========================] Waiting for authentication [=========================] Finished [=========================] Getting updates [=========================] Finished [=========================] Available kernel-4.1.0-0.rc6.git2.1.fc23.x86_64 (foo) The Linux kernel Available kernel-core-4.1.0-0.rc6.git2.1.fc23.x86_64 (foo) The Linux kernel Available kernel-modules-4.1.0-0.rc6.git2.1.fc23.x86_64 (foo) kernel modules to match the core kernel Available kmod-foo-1-2.x86_64 (foo) Dummy summary Available kmod-foo-4.1.0-0.rc6.git2.1-1-2.x86_64 (foo) Dummy summary
gnome-software-3.14.5-2.fc21, PackageKit-1.0.6-1.fc21, libhif-0.2.0-3.fc21, libappstream-glib-0.4.0-1.fc21 has been submitted as an update for Fedora 21. https://admin.fedoraproject.org/updates/FEDORA-2015-9570/gnome-software-3.14.5-2.fc21,PackageKit-1.0.6-1.fc21,libhif-0.2.0-3.fc21,libappstream-glib-0.4.0-1.fc21
Package PackageKit-1.0.6-6.fc22: * should fix your issue, * was pushed to the Fedora 22 testing repository, * should be available at your local mirror within two days. Update it with: # su -c 'yum update --enablerepo=updates-testing PackageKit-1.0.6-6.fc22' as soon as you are able to. Please go to the following url: https://admin.fedoraproject.org/updates/FEDORA-2015-9675/PackageKit-1.0.6-6.fc22 then log in and leave karma (feedback).
Clearing F22 accepted / nominated freeze exception status as F22 has shipped, per https://fedoraproject.org/wiki/BugZappers/HouseKeeping/Trackers . You may nominate as an Alpha, Beta or Final freeze exception for F23 if desired using the web application - https://qa.fedoraproject.org/blockerbugs/propose_bug (though it is not currently set up for F23) - or by marking the bug as blocking AlphaFreezeException, BetaFreezeException, or FinalFreezeException.
gnome-software-3.14.5-2.fc21, PackageKit-1.0.6-1.fc21, libhif-0.2.0-3.fc21, libappstream-glib-0.4.0-1.fc21 has been pushed to the Fedora 21 stable repository. If problems still persist, please make note of it in this bug report.
PackageKit-1.0.6-6.fc22 has been pushed to the Fedora 22 stable repository. If problems still persist, please make note of it in this bug report.