Bug 1717185 - stop job is running for PackageKit Daemon, holds up shutdown for 1m30s
Summary: stop job is running for PackageKit Daemon, holds up shutdown for 1m30s
Keywords:
Status: NEW
Alias: None
Product: Fedora
Classification: Fedora
Component: PackageKit
Version: 35
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Richard Hughes
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-06-04 20:40 UTC by Chris Murphy
Modified: 2021-11-04 17:57 UTC (History)
14 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:
Type: Bug


Attachments (Terms of Use)
shutdown log (1.99 MB, text/plain)
2019-06-04 20:43 UTC, Chris Murphy
no flags Details

Description Chris Murphy 2019-06-04 20:40:54 UTC
Description of problem:

On some reboots, PackageKit Daemon is holding up the shutdown. Looks like Gnome Software needs to be run, maybe even add or remove a package (not sure of the pattern, but merely logging in and rebooting doesn't appear to reproduce the problem).



Version-Release number of selected component (if applicable):


How reproducible:
Always, as described

Steps to Reproduce:
1. Login, launch gnome-software, add or remove something
2. Reboot
3.

Actual results:

Hang, systemd reports a stop job is running for PackageKitDaemon, times out after 1m30s.


Expected results:

PK should not hold up boot.

Additional info:

Comment 1 Chris Murphy 2019-06-04 20:43:10 UTC
Created attachment 1577276 [details]
shutdown log

https://freedesktop.org/wiki/Software/systemd/Debugging/#index2h1

followed the Shutdown Completes Eventually section to capture this; all I can see it shows is a 90s delay from the time systemd asks it to shutdown to the time it's forced to shutdown

Comment 2 Chris Murphy 2019-06-04 20:45:13 UTC
PackageKit-1.1.12-5.fc30.x86_64
python3-hawkey-0.31.0-3.fc31.x86_64

Those are the same versions Fedora 30 has (other than both are fc30), I don't have this problem there.

Comment 3 bugzilla.redhat.cvl.legacy 2019-07-09 14:33:36 UTC
Hey, I got the exact same problem. Problem is, I didn't really open gnome software or indeed anything. I just log in, shutdown, and there it stucks every time. Don't really know what kind of information you'd need.

Comment 4 skierpage 2019-07-17 23:03:16 UTC
This happens to me with Fedora KDE spin. I update software from the plasma "Software Updates" notification, which kicks off packagekitd, and I often update just before I close my laptop, shutting it down instead if the kernel has updated.

"PackageKit itself is a system activated daemon called packagekitd. Being system activated means that it's only being run when the user is using a text mode or graphical tool, and quits when it's no longer being used. This means we don't delay the boot sequence or session startup and don't consume memory when not being used." So it seems packagekitd should ordinarily not be running at system shutdown.

FWIW, I just got a notification about a few new packages and performed a "Software Updates", and a few minutes later packagekitd is still running. strace shows it's polling fd 4, 5,and 10. About 5 minutes later packagekitd process exited. So maybe packagekitd just takes several minutes to exit on its after you've installed software, and systemd shutdown can't hurry it up.

From the shutdown log attachment it seems systemd sends packagekitd stop-sigterm (not sure whether a DBus message or signal), the packagekit DBus interface continues to get or send additional PropertyChanged messages, then much later system shutdown gives up and "Killing process 1369 (packagekitd) with signal SIGKILL." There's an EOL'd Bug 1190915 "packagekitd does not respond to systemctl stop" that might be relevant but is short on details.

Comment 5 skierpage 2019-07-19 22:21:58 UTC
Shutdown and `systemctl stop packagekit` send SIGTERM to the process, then by default wait 90 seconds before sending SIGKILL. It doesn't look like PackageKit's pk-main.c responds to SIGTERM, but it does respond to Ctrl-C (SIGINT). However, the last time System Updates ran I tried sending SIGTERM and SIGINT to packagekitd after installation had apparently finished and neither seemed to make the process exit. If I manually start packagekit with `systemctl start packagekit` and send the packagekitd process SIGTERM it exits immediately, so maybe packagekitd is stuck in some state waiting for its DNF backend.

It seemed that the command-line utility `pkcon quit` immediately terminates the packagekitd process, so a possible fix/workaround is the packagekit.service could define "ExecStop=/usr/bin/pkcon quit" though that would add a dependency on pkcon.

I'm very much guessing, hope this helps.

Comment 6 Ben Cotton 2019-08-13 16:48:17 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 31 development cycle.
Changing version to '31'.

Comment 7 Tim Hawkins 2019-12-11 00:48:53 UTC
Is there any progress on this item, it is seriously irritating to have to wait for 90 seconds to shut down the workstation.

Comment 8 Martin Pitt 2020-02-04 09:57:44 UTC
A local workaround is to severely reduce the shutdown time for packagekit.service. I do that for cockpit now (https://github.com/cockpit-project/bots/pull/495/files), as this annoying to waste 1.5 minutes on each reboot.

Comment 9 Chris Murphy 2020-02-04 15:44:52 UTC
On reboot/shutdown should SIGTERM be issued to PackageKit? Is that happening or is it being ignored?

Comment 10 Martin Pitt 2020-02-04 16:04:45 UTC
@Chris: it is sent but ignored. Crude reproducer:


# pkcon get-updates &
# systemctl stop packagekit   # hangs

# systemctl status packagekit
● packagekit.service - PackageKit Daemon
   Loaded: loaded (/usr/lib/systemd/system/packagekit.service; static; vendo>
   Active: deactivating (stop-sigterm) since Tue 2020-02-04 11:03:42 EST; 21>
 Main PID: 1020 (packagekitd)
    Tasks: 6 (limit: 1137)
   Memory: 68.5M
      CPU: 1.680s
   CGroup: /system.slice/packagekit.service
           ├─1020 /usr/libexec/packagekitd
           └─3512 gpg-agent --homedir /var/cache/PackageKit/31/metadata/upda>

Feb 04 11:03:38 m1.cockpit.lan systemd[1]: Starting PackageKit Daemon...
Feb 04 11:03:38 m1.cockpit.lan PackageKit[1020]: daemon start
Feb 04 11:03:38 m1.cockpit.lan systemd[1]: Started PackageKit Daemon.
Feb 04 11:03:42 m1.cockpit.lan systemd[1]: Stopping PackageKit Daemon...

Comment 11 Sébastien Wilmet 2020-11-03 12:36:40 UTC
Still not fixed in Fedora 33.

Comment 12 Ben Cotton 2020-11-03 15:15:32 UTC
This message is a reminder that Fedora 31 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora 31 on 2020-11-24.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
Fedora 'version' of '31'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 31 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 13 skierpage 2020-11-03 22:56:53 UTC
(In reply to Sébastien Wilmet from comment #11)
> Still not fixed in Fedora 33.

Interesting. I hadn't experienced this problem with shutdown for months, my shutdowns with Fedora 32 were so fast that I barely see the console output after KDE exits, so I assumed something in Fedora 32 made it better in my case. (I did switch to command-line `sudo dnf --setopt=install_weak_deps=False update` to avoid PackageKit installing more packages than I want, but I'm pretty sure this was after I stopped having delayed shutdowns.) Martin Pitt's steps to reproduce in comment #10 don't reproduce for me on Fedora 33, I get: 

● packagekit.service - PackageKit Daemon
     Loaded: loaded (/usr/lib/systemd/system/packagekit.service; static)
     Active: inactive (dead)
...
Nov 03 14:47:09 fedlaptop PackageKit[64358]: get-updates transaction /6379_edbdcaaa from uid 1000 finished with suc>
Nov 03 14:47:22 fedlaptop systemd[1]: Stopping PackageKit Daemon...
Nov 03 14:47:22 fedlaptop systemd[1]: packagekit.service: Succeeded.
Nov 03 14:47:22 fedlaptop systemd[1]: Stopped PackageKit Daemon.
Nov 03 14:47:22 fedlaptop systemd[1]: packagekit.service: Consumed 3min 38.010s CPU time.

Comment 14 robin zlatic 2020-11-08 09:00:37 UTC
(In reply to Sébastien Wilmet from comment #11)
> Still not fixed in Fedora 33.

can confirm the bug appears in fedora 33.

Comment 15 Scott Beamer 2021-06-19 18:49:31 UTC
Still not fixed on Fedora 34.

Comment 16 Ben Cotton 2021-11-04 13:54:28 UTC
This message is a reminder that Fedora 33 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora 33 on 2021-11-30.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
Fedora 'version' of '33'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 33 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 17 Ben Cotton 2021-11-04 14:23:53 UTC
This message is a reminder that Fedora 33 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora 33 on 2021-11-30.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
Fedora 'version' of '33'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 33 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 18 Martin Pitt 2021-11-04 14:54:27 UTC
I still see it on F34 and F35.

Comment 19 Ben Cotton 2021-11-04 15:21:30 UTC
This message is a reminder that Fedora 33 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora 33 on 2021-11-30.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
Fedora 'version' of '33'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 33 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.


Note You need to log in before you can comment on or make changes to this bug.