Bug 1519543 - Fedora 27 offline updates (gnome-software intitiated) are failing
Summary: Fedora 27 offline updates (gnome-software intitiated) are failing
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: dnf-plugins-extras
Version: 27
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Jaroslav Mracek
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
: 1524459 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-11-30 21:12 UTC by Chris Murphy
Modified: 2018-01-09 16:46 UTC (History)
21 users (show)

Fixed In Version: dnf-plugins-extras-2.0.5-1.fc27 dnf-plugins-extras-2.0.5-1.fc26
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-12-16 12:33:59 UTC
Type: Bug


Attachments (Terms of Use)
journal.log (153.87 KB, text/x-vhdl)
2017-11-30 21:12 UTC, Chris Murphy
no flags Details
journaldebug.log (1.53 MB, text/x-vhdl)
2017-11-30 22:21 UTC, Chris Murphy
no flags Details
Journal (143.22 KB, text/x-vhdl)
2017-12-12 21:31 UTC, Alexandre Franke
no flags Details

Description Chris Murphy 2017-11-30 21:12:09 UTC
Created attachment 1361241 [details]
journal.log

Description of problem:


This is on a system upgraded a couple weeks ago with 'dnf system-upgrade' from Fedora 26 to Fedora 27.

I've receive a notification there are updatess, so I go into Gnome Software, choose Restart & Update from within Gnome-Software, it reboots and plymouth gives me the updating indication and status, and pretty much immediately I get a reboot.

Once rebooted, it's clear the updates have not been applied.

Version-Release number of selected component (if applicable):



How reproducible:


Steps to Reproduce:
1. Launch Gnome-Software and click Restart & Update
2. System reboots seemingly normally, and then:

Actual results:

Immediately reboots as the installing updates status appears in plymouth; does not apply updates.

Expected results:

Should apply updates.


Additional info:

Attaching complete journal.log for the prior boot (-b -1) which would be the pkofflineupdate boot that fails. This seems to be the relevant part but it's not enough information. Why does dnf and systemd want dnf-system-upgrade.service for a routine (weekly) non-upgrade update?

[    6.218141] f27h.localdomain pk-offline-update[815]: percentage 4%
[    6.254059] f27h.localdomain pk-offline-update[815]: percentage 6%
[    6.325118] f27h.localdomain dnf[813]: Error: use 'dnf system-upgrade reboot' to begin the upgrade
[    6.352470] f27h.localdomain systemd[1]: dnf-system-upgrade.service: Main process exited, code=exited, status=1/FAILURE
[    6.356239] f27h.localdomain systemd[1]: Failed to start System Upgrade using DNF.
[    6.356570] f27h.localdomain systemd[1]: dnf-system-upgrade.service: Unit entered failed state.
[    6.356777] f27h.localdomain systemd[1]: dnf-system-upgrade.service: Failed with result 'exit-code'.
[    6.356983] f27h.localdomain systemd[1]: Rebooting: service failed

Comment 1 Chris Murphy 2017-11-30 21:15:55 UTC
systemd-234-9.fc27.x86_64
gnome-software-3.26.3-1.fc27.x86_64
dnf-2.7.5-1.fc27.noarch

Comment 2 Chris Murphy 2017-11-30 22:21:55 UTC
Created attachment 1361323 [details]
journaldebug.log

Replace the useless log with one using systemd.log_level=debug.


[    6.876190] f27h.localdomain dnf[823]: Error: use 'dnf system-upgrade reboot' to begin the upgrade
[    6.904437] f27h.localdomain systemd[1]: systemd-journald.service: Received EPOLLHUP on stored fd 53 (stored), closing.
[    6.905020] f27h.localdomain systemd[1]: Received SIGCHLD from PID 823 (dnf).
[    6.905189] f27h.localdomain systemd[1]: Child 823 (dnf) died (code=exited, status=1/FAILURE)
[    6.905328] f27h.localdomain systemd[1]: dnf-system-upgrade.service: Child 823 belongs to dnf-system-upgrade.service
[    6.906227] f27h.localdomain systemd[1]: dnf-system-upgrade.service: Main process exited, code=exited, status=1/FAILURE
[    6.906471] f27h.localdomain systemd[1]: dnf-system-upgrade.service: About to execute: /usr/bin/rm -fv /system-update
[    6.906792] f27h.localdomain systemd[1]: dnf-system-upgrade.service: Forked /usr/bin/rm as 917
[    6.906984] f27h.localdomain systemd[1]: dnf-system-upgrade.service: Changed start -> stop-post



Well it looks to me like there's a conflict between dnf-system-upgrade.service and packagekit-offline-update.service. They are both running, but why?

[    6.292645] f27h.localdomain systemd[1]: dnf-system-upgrade.service: ConditionPathExists=/system-update succeeded.



Are both of these update methods using the same trigger? Very clearly gnome-software is setting /system-update symlink when I click Restart & Update. Huh...

Comment 3 Chris Murphy 2017-11-30 22:28:51 UTC
To be clear, gnome-software upon selecting and confirming Restart & Update, sets this symlink

/system-update -> /var/lib/PackageKit/prepared-update

This is intended for packagekit-offline-update.service, which does in fact start the update, as indicated by the progress in the log where it gets to 6%, at which point dnf injects itself into the update, and I guess it gets pissed because the symlink points to what it sees as a malformed payload, since the payload pointed to by the symlink is not a dnf system-upgrade download based payload.

Comment 4 Jaroslav Mracek 2017-12-05 11:35:13 UTC
Please can you provide dnf-plugins-extras version on your system?

Comment 5 Chris Murphy 2017-12-06 03:42:29 UTC
$ rpm -qa | grep dnf
python3-dnf-plugins-core-2.1.5-1.fc27.noarch
dnf-yum-2.7.5-1.fc27.noarch
dnf-conf-2.7.5-1.fc27.noarch
libdnf-0.11.1-1.fc27.x86_64
python3-dnf-plugins-extras-common-2.0.4-1.fc27.noarch
dnf-2.7.5-1.fc27.noarch
python3-dnf-2.7.5-1.fc27.noarch
dnf-plugins-core-2.1.5-1.fc27.noarch
python3-dnf-plugin-system-upgrade-2.0.4-1.fc27.noarch

Comment 6 Alexandre Franke 2017-12-12 21:31:41 UTC
Created attachment 1366975 [details]
Journal

Same problem here, attaching my journal output as asked on IRC by Kalev.

FWIW only one of my three machines has this issue and it’s a laptop, unlike the others.

Comment 7 Alexandre Franke 2017-12-12 21:32:15 UTC
*** Bug 1524459 has been marked as a duplicate of this bug. ***

Comment 8 Jaroslav Mracek 2017-12-13 08:45:16 UTC
I believe that the problem can be reproduce only with python3-dnf-plugin-system-upgrade-2.0.4-1.fc27.noarch . The commit https://github.com/rpm-software-management/dnf-plugins-extras/commit/cf33132f305fddf0f283e6227007f7fb7b18f44f should solve the issue.

Comment 9 Chris Murphy 2017-12-13 09:21:23 UTC
That version of python3-dnf-plugin-system-upgrade is stable on F26 and F27, so is seems like it'd affect Fedora 26 as well. If I understand this bug correctly, everyone who did their major version upgrade with dnf system-upgrade, cannot apply minor version updates using Gnome Software. Presumably they could work around this in the meantime by removing python3-dnf-plugin-system-upgrade?

Comment 10 Kalev Lember 2017-12-13 09:35:07 UTC
We're getting reports from multiple users that this has regressed in F26 as well now and broken updates done through gnome-software, e.g. https://mail.gnome.org/archives/gnome-software-list/2017-December/msg00000.html

Comment 11 Andrea Vai 2017-12-13 09:46:35 UTC
I am experiencing the same problem on Fedora 26: see also

https://mail.gnome.org/archives/gnome-software-list/2017-December/msg00000.html

gnome-software-3.24.3-3.fc26.x86_64

$ rpm -qa | grep dnf
python2-dnf-plugin-system-upgrade-2.0.4-1.fc26.noarch
dnf-conf-2.7.5-2.fc26.noarch
python3-dnf-2.7.5-2.fc26.noarch
python2-dnf-plugin-migrate-2.1.5-1.fc26.noarch
python2-dnf-2.7.5-2.fc26.noarch
python3-dnf-plugins-core-2.1.5-1.fc26.noarch
python3-dnf-plugins-extras-common-2.0.4-1.fc26.noarch
python3-dnf-plugin-system-upgrade-2.0.4-1.fc26.noarch
python2-dnf-plugins-core-2.1.5-1.fc26.noarch
python2-dnf-plugins-extras-common-2.0.4-1.fc26.noarch
dnf-2.7.5-2.fc26.noarch
dnf-plugins-core-2.1.5-1.fc26.noarch
libdnf-0.11.1-1.fc26.x86_64
dnf-yum-2.7.5-2.fc26.noarch

Let me know if I have to provide any further info / test result that may help

Comment 12 Jaroslav Mracek 2017-12-13 16:57:29 UTC
Tomorrow I will make a release of dnf-plugins-extras (dnf-system-upgrade) with patch that should solve it.

Comment 13 Fedora Update System 2017-12-14 09:38:26 UTC
dnf-plugins-extras-2.0.5-1.fc26 has been submitted as an update to Fedora 26. https://bodhi.fedoraproject.org/updates/FEDORA-2017-86e209f3c1

Comment 14 Fedora Update System 2017-12-14 09:38:43 UTC
dnf-plugins-extras-2.0.5-1.fc27 has been submitted as an update to Fedora 27. https://bodhi.fedoraproject.org/updates/FEDORA-2017-48b29b741c

Comment 15 Kalev Lember 2017-12-14 09:44:52 UTC
Thanks Jaroslav!

Comment 16 Andrea Vai 2017-12-14 10:12:00 UTC
Thank you. How can I try it out on Fedora 26? 

dnf install dnf-plugins-extras --enablerepo=updates-testing

says no matching argument.

Sorry for my being unskilled.

Comment 17 Jaroslav Mracek 2017-12-14 10:29:07 UTC
RPMs can be downloaded from https://koji.fedoraproject.org/koji/buildinfo?buildID=1009394 or just wait till it appears in updates-testing (I guess 24 to 48 hours).

Comment 18 Fedora Update System 2017-12-14 13:02:42 UTC
dnf-plugins-extras-2.0.5-1.fc27 has been pushed to the Fedora 27 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-48b29b741c

Comment 19 Fedora Update System 2017-12-14 13:34:18 UTC
dnf-plugins-extras-2.0.5-1.fc26 has been pushed to the Fedora 26 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-86e209f3c1

Comment 20 Fedora Update System 2017-12-16 12:33:59 UTC
dnf-plugins-extras-2.0.5-1.fc27 has been pushed to the Fedora 27 stable repository. If problems still persist, please make note of it in this bug report.

Comment 21 Andrea Vai 2017-12-18 08:08:17 UTC
(In reply to Jaroslav Mracek from comment #17)
> RPMs can be downloaded from
> https://koji.fedoraproject.org/koji/buildinfo?buildID=1009394 or just wait
> till it appears in updates-testing (I guess 24 to 48 hours).

I am sorry but I would ask for more instructions to test it, because still today the command

dnf install dnf-plugins-extras --enablerepo=updates-testing

says "no match for argument"

Is my configuration missing anything, or something else I don't undersand?

Comment 22 SirWeazel 2017-12-23 19:15:53 UTC
Is this getting pushed to the Fedora 26 Stable Repo also? I saw where it's in testing, and has been pushed to the 27 repo already.

Comment 23 Josh Poimboeuf 2018-01-01 16:46:25 UTC
(In reply to Andrea Vai from comment #21)
> I am sorry but I would ask for more instructions to test it, because still
> today the command
> 
> dnf install dnf-plugins-extras --enablerepo=updates-testing
> 
> says "no match for argument"
> 
> Is my configuration missing anything, or something else I don't undersand?

This worked for me:

dnf update --enablerepo=updates-testing python3-dnf-plugin-system-upgrade

Comment 24 Fedora Update System 2018-01-09 16:46:59 UTC
dnf-plugins-extras-2.0.5-1.fc26 has been pushed to the Fedora 26 stable repository. If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.