Bug 1430920

Summary: gnome-software graphical updates fail, pk-offline-update: no trigger, exiting
Product: [Fedora] Fedora Reporter: Chris Murphy <bugzilla>
Component: PackageKitAssignee: Richard Hughes <rhughes>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 26CC: awilliam, bugzilla, hobbes1069, johannbg, jonathan, klember, lnykryn, mruckman, msekleta, muadda, nb, pwhalen, rdieter, rhughes, robatino, smparrish, ssahani, s, systemd-maint, zbyszek
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: AcceptedBlocker AcceptedFreezeException
Fixed In Version: PackageKit-1.1.5-4.fc26 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-03-27 16:28:18 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1349185, 1349186    
Attachments:
Description Flags
journal debug none

Description Chris Murphy 2017-03-09 21:29:04 UTC
Description of problem:

Gnome Software and Shell initiated software updates aren't working, what happens is I get a reboot, a very brief startup and right at the point where I expect updates to start, I get another reboot without any updates applied.


Version-Release number of selected component (if applicable):
systemd-233-2.fc26.x86_64
gnome-software-3.23.91-1.fc26.x86_64


How reproducible:
Always


Steps to Reproduce:
1. Try to initiate a graphical update in GNOME
2.
3.

Actual results:

Two back to back reboots, and no updates applied between 1st and 2nd reboots.


Expected results:

Updates are applied after the 1st reboot.


Additional info:

Comment 1 Chris Murphy 2017-03-09 21:35:34 UTC
[    5.336603] localhost.localdomain systemd[1]: Reached target Offline System Update.
[    5.339436] localhost.localdomain systemd[1]: Starting Remove the Offline System Updates symlink...
[    5.343809] localhost.localdomain rm[733]: removed '/system-update'
[    5.346354] localhost.localdomain pk-offline-update[731]: no trigger, exiting


What is this rm[733] process initiated by? Is it clearing the flag before pk-offline-update can even see it?

Comment 2 Chris Murphy 2017-03-09 21:55:29 UTC
Created attachment 1261708 [details]
journal debug

If I boot with rd.debug systemd.log_level=debug, I get different results. Is this possibly a race? It still doesn't update the system, but I don't see

 pk-offline-update[731]: no trigger, exiting


And then there's these bogus statements...

[    6.746111] localhost.localdomain pk-offline-update[696]: sent msg to plymouth 'Installing updates; this could take a while...'
[    6.746251] localhost.localdomain pk-offline-update[696]: writing dummy results

Not exactly helpful...

[    6.873421] localhost.localdomain pk-offline-update[696]: status finished
[    6.873500] localhost.localdomain pk-offline-update[696]: writing failed results
[    6.873562] localhost.localdomain pk-offline-update[696]: failed to update system: Error calling StartServiceByName for org.freedesktop.PackageKit: The connection is closed


OK, unknown processes trying to do things?

[    6.873626] localhost.localdomain unknown[696]: Error while sending AddMatch() message: The connection is closed



Yeah that's it. Different result depending on whether I debug it or not.

Comment 3 Fedora Blocker Bugs Application 2017-03-09 23:18:01 UTC
Proposed as a Blocker for 26-beta by Fedora user chrismurphy using the blocker tracking app because:

 The installed system must be able to download and install updates with the default graphical package manager in all release-blocking desktops.

Comment 4 Zbigniew Jędrzejewski-Szmek 2017-03-12 01:40:08 UTC
This is probably caused by https://github.com/systemd/systemd/pull/4763. If pk-offline-update exited immediately, before it has finished it's job, that'd match the observed effects.

(Note: the upgrade script was always supposed to force a reboot on its own after it has finished the upgrade, but this wasn't properly enforced.)

Comment 5 Mike Ruckman 2017-03-13 18:55:03 UTC
Discussed in today's Blocker Review meeting. This is a clear violation of the following criterion: "The installed system must be able to download and install updates with the default graphical package manager in all release-blocking desktops."

Comment 6 Adam Williamson 2017-03-16 20:43:31 UTC
Just seen this same failure mode in an openQA test:

https://openqa.fedoraproject.org/tests/66104

Comment 7 Adam Williamson 2017-03-17 20:06:41 UTC
Also proposing as an Alpha FE; if we happen to get a fix for this it'd be good to have Alpha fixed, so folks installing from Alpha images can perform updates without falling back on dnf.

Comment 8 Kalev Lember 2017-03-21 07:31:54 UTC
Zbigniew, can you explain please what PackageKit should do differently? It looks like systemd is deleting /system-update before packagekit-offline-update.service even gets a chance to run?

Comment 9 Kalev Lember 2017-03-21 07:35:47 UTC
Also, are the patches in https://github.com/hughsie/PackageKit/pull/72 related to this?

Comment 10 Kalev Lember 2017-03-23 15:41:56 UTC
I can't see anything wrong in PackageKit. system-update-cleanup.service that is part of systemd is removing /system-update before the PackageKit offline updater gets a chance to run and this breaks the offline updater.

Reassigning back to systemd.

Comment 11 Zbigniew Jędrzejewski-Szmek 2017-03-23 20:13:57 UTC
https://github.com/hughsie/PackageKit/pull/72
https://github.com/rpm-software-management/dnf-plugins-extras/pull/89

(I know this is my fault, I should have checked the pk service file more carefully when updating the systemd units.)

Comment 12 Kalev Lember 2017-03-24 10:05:34 UTC
Thanks Zbigniew, that makes it work for me again. I've backported the last patch to PackageKit-1.1.5-4.fc26

Comment 13 Fedora Update System 2017-03-24 11:07:01 UTC
PackageKit-1.1.5-4.fc26 has been submitted as an update to Fedora 26. https://bodhi.fedoraproject.org/updates/FEDORA-2017-f6392bfef7

Comment 14 Fedora Update System 2017-03-24 14:53:53 UTC
PackageKit-1.1.5-4.fc26 has been pushed to the Fedora 26 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-f6392bfef7

Comment 15 Mike Ruckman 2017-03-24 20:07:49 UTC
This would be good to get in for Alpha.

+1 FE.

Comment 16 Paul Whalen 2017-03-24 20:09:06 UTC
+1 FE for Alpha

Comment 17 Nick Bebout 2017-03-24 20:34:49 UTC
+1 FE for Alpha

Comment 18 Adam Williamson 2017-03-24 20:35:46 UTC
That's +3/+4, marking as accepted.

Can folks test and karma the fix? It works for me, be good to have more karma. Thanks!

Comment 19 Fedora Update System 2017-03-27 16:28:18 UTC
PackageKit-1.1.5-4.fc26 has been pushed to the Fedora 26 stable repository. If problems still persist, please make note of it in this bug report.

Comment 20 Richard Shaw 2017-08-12 18:49:35 UTC
I'm currently trying to upgrade from Fedora 25 to 26 and I am running into this problem. It looks like only an updated package for Fedora 26 was released?

Comment 21 Richard Shaw 2017-08-12 21:06:50 UTC
Never mind, somehow my upgrade got interrupted and then it didn't know what to do. Error message could have been more helpful though...