Bug 839736

Summary: After update to systemd-44-13 or newer, logging in via gdm gets no console session, sound doesn't work, Gnome Session uses >100% CPU
Product: [Fedora] Fedora Reporter: Eric Smith <spacewar>
Component: systemdAssignee: Michal Schmidt <mschmidt>
Status: CLOSED NEXTRELEASE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 17CC: johannbg, lnykryn, lpoetter, metherid, mschmidt, msekleta, notting, plautrba, systemd-maint
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-10-07 22:46:41 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
systemd-tmpfiles status with systemd-44-17 (failed)
none
systemd-tmpfiles status with systemd-44-17 with patches 405-407 removed (good)
none
avc and systemd messages from /var/log/messages with selinux enforcing
none
avc and systemd messages from /var/log/messages with selinux permissive none

Description Eric Smith 2012-07-12 16:59:15 UTC
Description of problem:

Did a fresh install of F17. GDM login sessions worked fine. Sound worked. Gnome Shell CPU utilization was low when idle.

Did a "yum update" which installed 420 new and updated packages.  After rebooting, GDM logins did not get a session as listed by "systemd-loginctl", no sound devices other than null were available, and Gnome Shell CPU utiliation was >100%.


Version-Release number of selected component (if applicable):

systemd-44-13 or newer


How reproducible:
100% on my machine


Steps to Reproduce:
1. Do a fresh install of F17
2. Do a "yum update" to get systemd-44-13 or newer
3. Reboot
4. Log in via gdm
  

Actual results:

User has no login session as listed by "systemd-loginctl".
Sound doesn't work, with no devices other than null listed by sound settings or pactl.
Gnome Shell continuously uses >100% CPU (on multicore system).


Expected results:

User will have login session as listed by "systemd-loginctl".
Sound devices available normally.
Gnome Shell CPU utilization should be under 15% when idle.


Additional info:

The missing sound devices were the first noticed symptom.  At first I thought this might be bug #815413 as listed on the "Common F17 bugs" wiki page.  I followed the directions in that bug, but the PAM config files were actually correct.

I found bug #838651 which also involved sound not working on F17 with similar sound hardware, so I thought I might be experiencing the same problem.  After discussion in the comments on that bug, with a lot of help from Brendan Jones, it became apparent that my problem is not the same as that of Pablo Iranzo Gómez, the initial reporter of that bug.

I also found bug #812624 about Gnome Shell constantly using >100% of CPU, but again apparently my problem has a different root cause that than of the original reporter and other people experiencing that bug.

Through trial and error I determined that the bug was introduced by updating systemd.  By trying various systemd builds from Bodhi and Koji, I determined that systemd-44-12 did not exhibit this problem, and that systemd-44-13 did.

The systemd-44-13 update has the same base sources as systemd-44-12, with new patches 339-411.  I rebuilt eight times from the 44-13 SRPM doing a binary search and determined that building with patches 339-404 resulted in a working system, but building with 405 fails.  After some help by email from Michal Schmidt, I found that I can rebuild systemd-44-17 with patches 405-407 omitted, and the system works.  Switching to the offical systemd-44-17 with the full set of patches consistently causes this issue, and switching to my onofficial build of systemd-44-17 with patches 405-407 omitted consistently makes everything work properly.

Comment 1 Eric Smith 2012-07-12 17:11:20 UTC
Created attachment 597856 [details]
systemd-tmpfiles status with systemd-44-17 (failed)

Comment 2 Eric Smith 2012-07-12 17:12:08 UTC
Created attachment 597857 [details]
systemd-tmpfiles status with systemd-44-17 with patches 405-407 removed (good)

Comment 3 Eric Smith 2012-07-14 07:19:50 UTC
I've found that setting /etc/sysconfig/selinux to set permissive rather than enforcing and using the normal systemd-44-17 does result in getting a login session, sound working, etc.

There are var/log/messages entries for AVC denials and systemd errors.  I'll attached edited log files for both the permissive (successful) and enforcing (failure) cases.  I don't know what to make of these, other than to note that apparently when enforcing, systemd-logind reports that it can't create /run/user, which presumably is the reason that a user session can't be created.

Comment 4 Eric Smith 2012-07-14 07:20:48 UTC
Created attachment 598226 [details]
avc and systemd messages from /var/log/messages with selinux enforcing

Comment 5 Eric Smith 2012-07-14 07:21:18 UTC
Created attachment 598227 [details]
avc and systemd messages from /var/log/messages with selinux permissive

Comment 6 Michal Schmidt 2012-07-24 11:41:35 UTC
The problem is that your /usr/local is a symlink. The SELinux policy does not take this possibility into account. The 'filesystem' package provides it as a directory. You should not modify your system this way. Or if you do, make sure you can do the necessary SELinux policy changes too. You could use a bind mount instead of a symlink to avoid the SELinux issue.

Patch 0405 changes the behaviour of systemd-tmpfiles slightly. When opening the configuration directories, any other error than ENOENT now causes the whole thing to fail. This was an unexpected change that I can fix.

Comment 7 Eric Smith 2012-07-24 16:03:23 UTC
Interesting!  Thanks for looking at this.

I always set up /usr/local and /opt as symlinks to /home/local and /home/opt so that they're on a filesystem that doesn't get wiped when I do an upgrade or reinstall.

I barely know my way around SELinux, and certainly don't know how to fix the policy properly.  I'll look into using bind mount as you suggest.

Comment 8 Michal Schmidt 2012-07-24 21:37:43 UTC
(In reply to comment #6)
> Patch 0405 changes the behaviour of systemd-tmpfiles slightly. When opening
> the configuration directories, any other error than ENOENT now causes the
> whole thing to fail. This was an unexpected change that I can fix.

Fixed upstream:
http://cgit.freedesktop.org/systemd/systemd/commit/?id=578ac0604e6c10b267f73e114bc2215aa3f6619a

Comment 9 Fedora Update System 2012-09-20 19:55:43 UTC
systemd-190-1.fc18 has been submitted as an update for Fedora 18.
https://admin.fedoraproject.org/updates/systemd-190-1.fc18

Comment 10 Fedora Update System 2012-09-22 06:36:57 UTC
Package systemd-191-2.fc18, rtkit-0.11-3.fc18:
* should fix your issue,
* was pushed to the Fedora 18 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing systemd-191-2.fc18 rtkit-0.11-3.fc18'
as soon as you are able to, then reboot.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2012-14581/rtkit-0.11-3.fc18,systemd-191-2.fc18
then log in and leave karma (feedback).

Comment 11 Fedora Update System 2012-09-28 00:17:20 UTC
Package glibc-2.16-17.fc18, systemd-192-1.fc18, selinux-policy-3.11.1-23.fc18, rtkit-0.11-3.fc18:
* should fix your issue,
* was pushed to the Fedora 18 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing glibc-2.16-17.fc18 systemd-192-1.fc18 selinux-policy-3.11.1-23.fc18 rtkit-0.11-3.fc18'
as soon as you are able to, then reboot.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2012-14581/selinux-policy-3.11.1-23.fc18,rtkit-0.11-3.fc18,systemd-192-1.fc18,glibc-2.16-17.fc18
then log in and leave karma (feedback).

Comment 12 Fedora Update System 2012-10-01 20:09:36 UTC
Package glibc-2.16-17.fc18, rtkit-0.11-3.fc18, systemd-193-1.fc18:
* should fix your issue,
* was pushed to the Fedora 18 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing glibc-2.16-17.fc18 rtkit-0.11-3.fc18 systemd-193-1.fc18'
as soon as you are able to, then reboot.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2012-14581/rtkit-0.11-3.fc18,systemd-193-1.fc18,glibc-2.16-17.fc18
then log in and leave karma (feedback).

Comment 13 Fedora Update System 2012-10-12 16:46:45 UTC
systemd-44-20.fc17 has been submitted as an update for Fedora 17.
https://admin.fedoraproject.org/updates/systemd-44-20.fc17