Bug 789758

Summary:	systemd update leaves no options to reboot/shutdown
Product:	[Fedora] Fedora	Reporter:	Michal Jaegermann <michal>
Component:	systemd	Assignee:	systemd-maint
Status:	CLOSED RAWHIDE	QA Contact:	Fedora Extras Quality Assurance <extras-qa>
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	rawhide	CC:	dan.mashal, johannbg, metherid, mschmidt, notting, plautrba, ssorce, systemd-maint
Target Milestone:	---	Keywords:	Reopened
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	systemd-43-1.fc17	Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2013-08-26 07:35:09 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Michal Jaegermann 2012-02-12 20:40:59 UTC

Description of problem:

After systemd update happened the following shows up on attempts to reboot:

# reboot
Failed to open /dev/initctl: No such device or address
Failed to talk to init daemon.
# systemctl -f reboot
Failed to get D-Bus connection: Failed to connect to socket /run/systemd/private: No such file or directory
# systemctl -f daemon-reexec
Failed to get D-Bus connection: Failed to connect to socket /run/systemd/private: No such file or directory
# systemctl -f kexec
Failed to get D-Bus connection: Failed to connect to socket /run/systemd/private: No such file or directory

Indeed, in /run/systemd/ we have only these:

drwxr-xr-x 2 root root  40 Feb 12 11:51 ask-password
drwxr-xr-x 2 root root 100 Feb 12 11:51 journal
srw-rw-rw- 1 root root   0 Feb 12 12:21 notify
drwxr-xr-x 2 root root  60 Feb 12 12:39 seats
drwxr-xr-x 2 root root 100 Feb 12 13:01 sessions
-rw-r--r-- 1 root root   0 Feb 12 12:17 show-status
srw------- 1 root root   0 Feb 12 11:51 shutdownd
drwxr-xr-x 2 root root  40 Feb 12 11:51 system
drwxr-xr-x 2 root root  60 Feb 12 13:01 users

Process tables show both systemd and dbus running.

So what one can do here beyond pressing a power switch and what happens if that
power switch is on a remote machine?

Version-Release number of selected component (if applicable):
systemd-42-1.fc17.x86_64

How reproducible:
all the time until power switch is used

Expected results:
A way to reboot a machine after systemd version changed; in particular when the only access is remote.  Actually it should be a way even when dbus messed up.

Additional info:
This particular mess is from systemd-39-3.fc17->systemd-42-1.fc17 transition. OTOH I had this issue already a few times in the past but it was always together with some other troubles and I never looked closer.

After a power button was applied, which made fsck quite unhappy on reboot, /run/systemd/private socked did show up and 'reboot' is functional again.

Comment 1 Jóhann B. Guðmundsson 2012-02-12 21:08:18 UTC

Wondering if this is duplicate of 732845 ( closed duplicate of abrt bug wondering if that one actually was a duplicate ) particularly pay attention to commment 1

Comment 2 Michal Jaegermann 2012-02-12 21:54:50 UTC

(In reply to comment #1)
> Wondering if this is duplicate of 732845 ...

It does not seem to be.  As opposed to bug 732845 here everything seems to be running just fine (I noted in a description that systemd and dbus were present in process tables) only is of not much use. Abrt did not catch anything, nor I can find anything about crashing in logs.  The only unusual stuff recorded at the time when this upgrade was happening was:

 systemd[1]: [/usr/lib/systemd/system/systemd-journald.socket:26] Failed to parse numeric value, ignoring: 8M
 systemd[1]: [/usr/lib/systemd/system/syslog.socket:24] Failed to parse numeric value, ignoring: 8M
 systemd[1]: [/usr/lib/systemd/system/prefdm.service:21] Unknown lvalue 'IgnoreSIGPIPE' in section 'Service'. Ignoring.
 systemd[1]: [/usr/lib/systemd/system/getty@.service:30] Unknown lvalue 'IgnoreSIGPIPE' in section 'Service'. Ignoring.
 systemd[1]: [/usr/lib/systemd/system/emergency.service:27] Unknown lvalue 'IgnoreSIGPIPE' in section 'Service'. Ignoring.

Messages like those, or very similar, showed up in logs few times more while other updates were happening and stopped after a hard powerdown and reboot.

Comment 3 Michal Schmidt 2012-02-13 10:31:12 UTC

(In reply to comment #0)
> This particular mess is from systemd-39-3.fc17->systemd-42-1.fc17 transition.

This could be a result of the move of the systemd binary. daemon-reexec would fail in the old process.

Transitional issues like this are no big deal in Rawhide, but we should keep an eye on eventual F16->F17 yum upgrades which will be affected by this too. We can patch F16's systemd to try both the old and the new locations when reexecuting.

And FWIW, "sync; reboot -f" is likely to work fine even if systemd is dead. And it's safer than the power switch.

Comment 4 Michal Schmidt 2012-02-13 22:46:07 UTC

*** Bug 790210 has been marked as a duplicate of this bug. ***

Comment 5 Michal Jaegermann 2012-02-13 23:02:26 UTC

(In reply to comment #3)
 
> And FWIW, "sync; reboot -f" is likely to work fine even if systemd is dead. And
> it's safer than the power switch.

Not only that but it also works on from a remote login.  I messed up a system on purpose by killing dbus-daemon.  All "usual suspects" refused to work with "Failed to get D-Bus connection" messages.  No surprise here. Still
"sync; reboot -f" did work.  Thanks!  It says on 'man reboot' in -f description: "Don't contact the init system" but somehow I missed implications.

Comment 6 Michal Schmidt 2012-02-16 10:37:57 UTC

Lennart fixed the upgrade issue in systemd-43-1.fc17 by adding a compatibility symlink. The old systemd process will be able to reexec /bin/systemd.

Comment 7 Dan Mashal 2013-08-24 02:46:24 UTC

Getting this on rawhide:

[root@Fedora ~]# systemctl reboot
Failed to get D-Bus connection: Failed to connect to socket /run/systemd/private: Connection refused


[root@Fedora ~]# reboot
Failed to open /dev/initctl: No such device or address
Failed to talk to init daemon.
[root@Fedora ~]# init 6
Failed to open /dev/initctl: No such device or address
Failed to talk to init daemon.

Comment 8 Michal Schmidt 2013-08-26 07:35:09 UTC

Dan, this looks like systemd has crashed. Please looks for a core dump or for an assertion failure in the journal. Please open a new BZ if you can provide some information like that.
It is not likely to be related to the original problem discussed in this BZ, so I'm closing it again.