Bug 1572953

Summary: systemd-tmpfiles-setup-dev fails, boot stalls, CPU 100%, system unusable, after dnf update
Product: [Fedora] Fedora Reporter: Paul DeStefano <prd-fedora>
Component: systemdAssignee: Ray Strode [halfline] <rstrode>
Status: CLOSED EOL QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 29CC: alexl, jmv1, john.j5live, lnykryn, matthias_haase, mclasen, msekleta, renault, rhughes, rstrode, sergio, s, systemd-maint, zbyszek
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-11-27 22:37:40 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
journalctl -xb none

Description Paul DeStefano 2018-04-29 07:38:33 UTC
Description of problem:
Just did dnf upgrade, now system boot stalls for a long time at Started User Manger for UID 42, eventually launches GDM, which never shows up. Cannot login, cannot switch to virtual console, system unusable.

During boot, many services say they are starting multiple times.

Comment 1 Paul DeStefano 2018-04-29 07:56:28 UTC
I can see that two units are failing:

systemd-tmpfiles-setup-dev
systemd-tmp-files-setup

Comment 2 Paul DeStefano 2018-04-29 07:58:51 UTC
systemd-tmpfiles-setup-dev error says: 

Unable to fix SELinux secruty context of /dev/cuse: Permission Denied.

Comment 3 Paul DeStefano 2018-04-29 08:04:24 UTC
I see this on both my VM and my netbook.

Comment 4 Paul DeStefano 2018-04-29 08:08:13 UTC
CPU is pegged at 100% after "Started Virtualization daemon. Dispatcher Service ...es.ce.ansactions..." which is clearly corrupted output.

Comment 5 Couret Charles-Antoine 2018-04-29 19:18:32 UTC
I have the same issue with GDM but I can have access to virtual console (only if I don't touch any key during startup and if I don't try to go to virtual console 1).

On journad I can read:

gdm: Could not start command '/usr/libexec/gdm-session-worker': too many files opened
gdm: Glib: g_child_watch_add_full: assertion 'pid > 0' failed
gdm: Child process XXXXXX was already dead
gdm: Unable to kill session worker process

And some events from systemd to say that the service GDM was stopped finally.

Comment 6 Couret Charles-Antoine 2018-04-29 19:22:06 UTC
So not sure that systemd was the root cause of your main issue.
I got other errors above those I quoted:

gdm: /usr/libexec/gmd-wayland-session: symbol lookup error: /lib64/libcairo.so.2: undefined symbol: FT_Done_MM_Var
gdm: Glib: g_variant_new_string: assertion 'string != NULL' failed

And some invalid SELinux contexts for systemd-logind

Comment 7 Paul DeStefano 2018-04-29 20:41:45 UTC
(In reply to Couret Charles-Antoine from comment #6)
> So not sure that systemd was the root cause of your main issue.
> I got other errors above those I quoted:
> 
> gdm: /usr/libexec/gmd-wayland-session: symbol lookup error:
> /lib64/libcairo.so.2: undefined symbol: FT_Done_MM_Var
> gdm: Glib: g_variant_new_string: assertion 'string != NULL' failed
> 
> And some invalid SELinux contexts for systemd-logind

I don't see any of these errors.  I only see AVC errors for wayland stuff.

I only see some of your errors from comment #5, but not all.

Do you see systemd-tmpfiles errors?

It doesn't sound like you have the same symptoms as me.  You should open another bug report.  Could be the same thing, but could be completely different.

Comment 8 Paul DeStefano 2018-05-03 03:55:01 UTC
I am able to boot single and start the network.  Using this, I was able to run dnf upgrade and get a new SElinux policy.  After reboot, I get the exact same behavior: CPU pegged after several critical service failures.  Two that stand out are

Create static device nodes in /dev
Create volatile files and directories

I don't understand what could cause these failures.

So, again, it's just me experiencing this?  That defies logic.  I just did a dnf upgrade.  I'm so confused.

Comment 9 Paul DeStefano 2018-05-03 04:03:36 UTC
Created attachment 1430416 [details]
journalctl -xb

I recovered this from the system.

Comment 10 Matthias Haase 2018-05-03 06:06:16 UTC
(In reply to Paul DeStefano from comment #0)
> Description of problem:
> Just did dnf upgrade, now system boot stalls for a long time at Started User
> Manger for UID 42, eventually launches GDM, which never shows up. Cannot
> login, cannot switch to virtual console, system unusable.
> 
> During boot, many services say they are starting multiple times.

Can confirm this exactly on a Lenovo G50-70 after updating from Fedora 27 to Fedora 28. Boot stucks and gdm tries to startup multiple times without success.

Have updated manually to the 203 build of 4.16 kernel from bodhi. This fixes the initial delay on boot as described in known bug section for Fedora 28. But the gdm failure is still there. From ash I can access and everything seems ok.

It's strange really.

Comment 11 Matthias Haase 2018-05-03 08:07:56 UTC
It sounds stupid but - this morning without any change in software - gdm starts up as expected.

So I think this may be time stamp related on /var/lib/gdm/.ICEauthority or similar, I have no exact idea. However I can exclude hardware issues.

Comment 12 Paul DeStefano 2018-05-04 04:11:54 UTC
Interesting.

Comment 13 Paul DeStefano 2018-05-04 04:15:27 UTC
Clearly, your issue was not this one.  I've tried many times to get this system to boot and it will not.

If I can get anyone anything that would help diagnose this problem, just tell me what to do.  My system really isn't usable.  I can initiate updates, but that hasn't helped.

Comment 14 Paul DeStefano 2018-05-04 08:29:46 UTC
systemd-tmpfiles-setup-dev specifically reports:

Unable to fix SELinux security context of /dev/snd/seq: Permision denied

Comment 15 Paul DeStefano 2018-05-08 00:31:33 UTC
Update seems to have resolved this issue.

Comment 16 Jan Kurik 2018-08-14 10:04:45 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 29 development cycle.
Changing version to '29'.

Comment 17 Sergio Basto 2018-09-04 03:57:56 UTC
(In reply to Paul DeStefano from comment #7)
> (In reply to Couret Charles-Antoine from comment #6)
> > So not sure that systemd was the root cause of your main issue.
> > I got other errors above those I quoted:
> > 
> > gdm: /usr/libexec/gmd-wayland-session: symbol lookup error:
> > /lib64/libcairo.so.2: undefined symbol: FT_Done_MM_Var
> > gdm: Glib: g_variant_new_string: assertion 'string != NULL' failed
> > 
> > And some invalid SELinux contexts for systemd-logind


undefined symbol: FT_Done_MM_Var is caused by freetype-freeworld 

Fix for me was "dnf remove freetype-freeworld".

Comment 18 Ben Cotton 2019-10-31 19:14:44 UTC
This message is a reminder that Fedora 29 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora 29 on 2019-11-26.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
Fedora 'version' of '29'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 29 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 19 Ben Cotton 2019-11-27 22:37:40 UTC
Fedora 29 changed to end-of-life (EOL) status on 2019-11-26. Fedora 29 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.