Bug 1420753

Summary: "systemd-udevd.service: Failed at step ADDRESS_FAMILIES spawning" for PowerPC
Product: [Fedora] Fedora Reporter: Michel Normand <normand>
Component: systemdAssignee: systemd-maint
Status: CLOSED RAWHIDE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: high Docs Contact:
Priority: unspecified    
Version: 26CC: dan, gmarr, hannsj_uhl, johannbg, lnykryn, msekleta, muadda, normand, rjones, robatino, ssahani, s, systemd-maint, zbyszek
Target Milestone: ---   
Target Release: ---   
Hardware: powerpc   
OS: Linux   
Whiteboard: RejectedBlocker AcceptedFreezeException
Fixed In Version: systemd-233-2.fc26 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-03-02 20:10:08 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1071880, 1349185    
Attachments:
Description Flags
rawhide_20170208_dracut_failure.log none

Description Michel Normand 2017-02-09 13:20:06 UTC
Created attachment 1248811 [details]
rawhide_20170208_dracut_failure.log

Description of problem: Unable to boot PowerPC Rawhide iso
at least since 20170131 compose.
(no failure on 20170125)

Still same problem with 20170208 compose.

How reproducible: try to install ppc64le guest with netiso rawhide.
https://kojipkgs.fedoraproject.org/compose/rawhide/latest-Fedora-Rawhide/compose/Server/ppc64le/iso/

Actual results:
the install failed in dracut with failed lines like following ones:
===
[FAILED] Failed to start Journal Service.
[   11.663906] systemd[536]: systemd-udevd.service: Failed at step ADDRESS_FAMILIES spawning /usr/lib/systemd/systemd-udevd: File exists
===

Comment 1 Fedora Blocker Bugs Application 2017-02-09 15:51:59 UTC
Proposed as a Blocker for 26-alpha by Fedora user michelmno using the blocker tracking app because:

 Unable to do a simple install ppc64/ppc64le Rawhide since compose 20170131 (worked on 20170125)

Comment 2 Zbigniew Jędrzejewski-Szmek 2017-02-12 22:51:09 UTC
It's beeing worked on. I think current systemd git works OK, but some seccomp filters are disabled on PPC and other non-amd64 arches. Systemd 233 is to be released soon (ETA ~ a week or so), and should fix this.

Comment 3 Geoffrey Marr 2017-02-13 20:06:22 UTC
Discussed during the 2017-02-13 blocker review meeting: [1]

The decision was made to classify this bug as a RejectedBlocker and an AcceptedFreezeException as the current information we have on this bug leads us to believe that this only affects non-release-blocking arches (ppc). However this bug is accepted as a FreezeException as it is a major bug for non-blocking releases.

Also, as fixing/changing anything related to systemd is considered hazardous, QE would like to request a backport of the fix, and not a new systemd release build.

[1] https://meetbot.fedoraproject.org/fedora-blocker-review/2017-02-13/f26-blocker-review.2017-02-13-18.01.txt

Comment 4 Michel Normand 2017-02-14 15:42:30 UTC
Note that a similar bug was raised on Debian 853755
supposed to be solved by systemd 232.16 as per url
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=853755#89

Assuming this is the same problem,
are we able to have such level in fedora ?

Comment 5 Richard W.M. Jones 2017-02-27 20:46:49 UTC
Is there a workaround to get the machine to boot again?

Comment 6 Richard W.M. Jones 2017-02-27 20:52:35 UTC
To answer my own question, going back to a Fedora kernel
from the GRUB menu seems to have worked
(faulty kernel: 4.11.0-0.rc0.git2.1.fc26.ppc64
working kernel: 4.8.8-300.fc25.ppc64)

Comment 7 Richard W.M. Jones 2017-02-27 20:53:03 UTC
(In reply to Richard W.M. Jones from comment #6)
> To answer my own question, going back to a Fedora kernel
                                                   ^ 25

Comment 8 Richard W.M. Jones 2017-02-27 21:56:57 UTC
No in fact that didn't work - it just seems to have failed
slightly later on, but with the same fundamental problem of
"Failed at step ADDRESS_FAMILIES ..."

Comment 9 Dan Horák 2017-02-27 22:27:23 UTC
boot with init=/bin/sh and then
sed -i.bak -e 's/RestrictAddressFamilies/#RestrictAddressFamilies/g' /usr/lib/systemd/system/*.service

AFAIK the bad system unit files are present in the new initrds, so booting old initrd (pre-systemd-232) "into" updated local units should work. I used this (no warranty) for recovering a rawhide s390 guest after hitting https://github.com/systemd/systemd/issues/4575 which should be the same root cause (buggy seccomp support).

Comment 10 Fedora End Of Life 2017-02-28 11:13:51 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 26 development cycle.
Changing version to '26'.

Comment 11 Michel Normand 2017-03-02 12:14:39 UTC
(In reply to Dan Horák from comment #9)
> boot with init=/bin/sh and then
> sed -i.bak -e 's/RestrictAddressFamilies/#RestrictAddressFamilies/g'
> /usr/lib/systemd/system/*.service
> 
> AFAIK the bad system unit files are present in the new initrds, so booting
> old initrd (pre-systemd-232) "into" updated local units should work. I used
> this (no warranty) for recovering a rawhide s390 guest after hitting
> https://github.com/systemd/systemd/issues/4575 which should be the same root
> cause (buggy seccomp support).

Because there is no visible progress in solving of this bug in system for wa while and because we are not able to boot any iso for PowerPC rawhide (still failure with compose 20170228)

Is it easier to have the bypass suggested by Dan to be integrated for Rawhide and F26 build for PowerPC ?

Comment 12 Dan Horák 2017-03-02 14:19:42 UTC
systemd 233 that should fix the issue was released yesterday, no build in Fedora yet, but I expect one in the next hours.

Comment 13 Zbigniew Jędrzejewski-Szmek 2017-03-02 20:10:08 UTC
Should be fixed now. Let's see.

Comment 14 Michel Normand 2017-03-14 08:26:57 UTC
correction has been validated on Rawhide f27 compose 20170310
BUT not yet on f26 compose 20170313 still blocked by bug #1426796.