Bug 1418886

Summary: Trying to start, stop, restart services with systemd fails and returns dbus timeout (g-dbus-error-quark, 20)
Product: Red Hat Enterprise Linux 7 Reporter: Joe Wright <jwright>
Component: systemdAssignee: systemd-maint
Status: CLOSED INSUFFICIENT_DATA QA Contact: qe-baseos-daemons
Severity: high Docs Contact:
Priority: unspecified    
Version: 7.3CC: ajipatil, alanm, ffutigam, jraising, jsynacek, mkolbas, nk5, ouimet, systemd-maint-list
Target Milestone: rcKeywords: Desktop, Reopened
Target Release: ---   
Hardware: x86_64   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-12-15 10:10:32 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1383699    

Description Joe Wright 2017-02-03 00:39:49 UTC
Description of problem:
Trying to start, stop, restart services with systemd fails and emits errors like:
Error getting authority: Error initializing authority: Error calling StartServiceByName for org.freedesktop.PolicyKit1: GDBus.Error:org.freedesktop.DBus.Error.TimedOut: Activation of org.freedesktop.PolicyKit1 timed out (g-dbus-error-quark, 20)

Version-Release number of selected component (if applicable):
- polkit-0.112-9.el7.x86_64
- systemd-219-30.el7_3.6.x86_64
- dbus-1.6.12-17.el7.x86_64
- kernel-3.10.0-514.6.1.el7.x86_64

How reproducible:
- unsure. Unable to reproduce in-house

Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:
- Permissions on tmp shadow and passwd are correct
- rpm verification ok

Based on the following logs it seems that the issue could be with systemd and not specific to polkit :

Jan 25 17:19:28 localhost systemd-logind: Failed to fully start up daemon: Connection timed out
Jan 25 17:19:28 localhost dbus[908]: [system] Failed to activate service 'org.freedesktop.systemd1': timed out
Jan 25 17:19:28 localhost dbus-daemon: dbus[908]: [system] Failed to activate service 'org.freedesktop.systemd1': timed out
Jan 25 17:19:28 localhost systemd: systemd-logind.service: main process exited, code=exited, status=1/FAILURE
Jan 25 17:19:28 localhost systemd: Failed to start Login Service.
Jan 25 17:19:28 localhost systemd: Unit systemd-logind.service entered failed state.
Jan 25 17:19:28 localhost systemd: systemd-logind.service failed.
Jan 25 17:19:28 localhost systemd: systemd-logind.service has no holdoff time, scheduling restart.
Jan 25 17:19:28 localhost systemd: Starting Login Service...

The line "Jan 25 17:19:28 localhost dbus[908]: [system] Failed to activate service 'org.freedesktop.systemd1': timed out" seems to be pointing to the issue.

On checking for issues reported for upstream, I could see systemd being responsible for the issue. Following are the upstream reported issues:

https://github.com/systemd/systemd/issues/2019
https://bugs.freedesktop.org/show_bug.cgi?id=93066
https://github.com/systemd/systemd/issues/2925

Comment 3 Lukáš Nykrýn 2017-02-06 10:53:21 UTC
I don't think that logind would block systemctl start/....
This seems to be a bigger problem:
Jan 25 17:19:28 localhost dbus-daemon: dbus[908]: [system] Failed to activate service 'org.freedesktop.systemd1': timed out

Comment 7 Daniel Ouimet 2017-12-19 19:06:30 UTC
I have the exact same problem with a server I just updated.
My version number are different (except for dbus is the same):
polkit-0.112-12.el7_3.x86_64
systemd-219-42.el7_4.4.x86_64
kernel-3.10.0-693.11.1.el7.x86_64

The problem seems to be polkit that can't start org.freedesktop.PolicyKit1
Here is a part of the start of polkit:
polkitd[139162]: Acquired the name org.freedesktop.PolicyKit1 on the system bus
systemd[1]: polkit.service start operation timed out. Terminating.
Failed to start Authorization Manager.

dbus and other services are not starting for this reason. I googled the problem and didn't see anything.

I am not using SELinux, so this is not the problem.
And I reinstall polkit just in case: yum reinstall polkit
And reinstall dbus too, just in case!
I checked that all the rules in /etc/polkit-1/ are exactly the same with exact same permission as 1 year ago. Done!

There is a bug in the way that polkit is started (is it in polkit? dbus? systemd?)

A strace of the start of polkit seems normal until at the end it loops on:
poll([{fd=5, events=POLLIN}, {fd=8, events=POLLIN}], 2, -1) = 1 ([{fd=5, revents=POLLIN}])
read(5, "\1\0\0\0\0\0\0\0", 16) = 8
with sometimes this line appears:
futex(0x560d2fca1d50, FUTEX_WAKE_PRIVATE, 1) = 1

So there is a real bug in the updates because it was working perfectly before the update.

Comment 8 Daniel Ouimet 2017-12-22 14:42:12 UTC
Problem solved!
I don't know if your problem was the same, but mine was that setroubleshoot user was absent from the /etc/passwd

setroubleshoot-server was complaining it could not be updated because the setroubleshoot group was there but not the user. So I removed the group, reinstall setroubleshoot with an update. After I restart by hand (with systemctl) all other failed deamon beginning with polkit, and voila!

Hope this helps you find your problem.

Daniel

Comment 9 nk5 2018-07-09 02:54:05 UTC
Anyone else pulling their hair out with this issue - run a quick pwck and see if anything is amiss. Due to a fat-finger search and replace, I managed to remove the home directory field for systemd-network and polkitd, making both accounts invalid and systemd/polkitd very upset.