Bug 1043212
| Summary: | something creating /var/run/nologin | ||
|---|---|---|---|
| Product: | [Fedora] Fedora | Reporter: | Jason Haar <jhaar> |
| Component: | systemd | Assignee: | systemd-maint |
| Status: | CLOSED ERRATA | QA Contact: | Fedora Extras Quality Assurance <extras-qa> |
| Severity: | high | Docs Contact: | |
| Priority: | high | ||
| Version: | 19 | CC: | adrien-xx-redhatbz, aheadley, ajschult784, aliakc, andriy.ushakov, atontti+rh, aumicka, awilliam, beton, bloch, bugzilla.redhat.com, bugzilla, carasin.berlogue, cjs, colin, cpanceac, crokett, danielkza2, david, djh, dopey, edgar.hoch, ed.greshko, edwinh, error, frank, fredericg_99, gilboad, grosser.meister.morti, hakan_duran, hebert.soares, hill-robert, horsley1953, hvtaifwkbgefbaei, hytekk, icon, ignatenko, ikke, jeremy, jhaar, jmatthew, jochen, johan.heikkila, johannbg, jokeyrhyme, jss, jtfjdehf, j, katmeef, kmf, lars, leho, lhn, lnykryn, maciej.kycler, maciek.borzecki, madko, marcus, maurizio.antillon, mcatanzaro+wrong-account-do-not-cc, mmckinst, mrsam, msekleta, mykola.dvornik, nekohayo, next.little.owl, nicku, nuno.ponte, pablo.iranzo, paul, plautrba, pnemade, raphael.brandis, rdieter, redhat, rmainz, rohara, rolf, schmidtm, sergio.pasra, systemd-maint, thomas.mey, tmraz, tra, twaugh, vmukhame, vpavlin, vskcode, wendellcraigbaker, wonder_6908, zbyszek |
| Target Milestone: | --- | Keywords: | CommonBugs, Reopened |
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | https://fedoraproject.org/wiki/Common_F20_bugs#systemd-nologin-creation | ||
| Fixed In Version: | systemd-208-14.fc20 | Doc Type: | Bug Fix |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2015-01-12 19:52:09 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
|
Description
Jason Haar
2013-12-15 04:05:05 UTC
I just rebooted and it happened again. Even after waiting 20minutes before logging in via gdm, I cannot due to this issue. /var/run/nologin contains System is booting up. See pam_nologin(8) I have had to hack a service together to continually delete that file to let me use the system at all :-( PAM is not creating the file at all. I have no idea what creates the file on your system. But given the system shutdown is handled by systemd I am reassigning it there for further investigation. It's some daemon/service that's creating this. Jason do you have proftpd installed by anychance? simply running "systemd-tmpfiles --create" will create that file so I think we are dealing with two bugs here one that something calls systemd-tmpfiles --create without path and two that systemd-tmpfiles creates that file if run without path *** Bug 1043598 has been marked as a duplicate of this bug. *** I see three facets of this bug: 1. "systemd-tmpfiles --create" breaks a running system. In principle this is technically correct, but an innocuous sounding command wreaks havoc. We should fix this. I'm assinging this part to myself. 2. Something calls "systemd-tmpfiles --create". We still need to find out what, and update that. Hopefully Jason or Jean-François can help us. 3. screensaver uses pam_nologin. I'm not sure if this is good or bad. Probably not a bug. Upstream commit: http://cgit.freedesktop.org/systemd/systemd/commit/?id=c4708f1323. /run/nologin will not be created when systemd-tmpfiles --create is run anymore. This change should be easy to backport. (In reply to Jason Haar from comment #1) > I just rebooted and it happened again. Even after waiting 20minutes before > logging in via gdm, I cannot due to this issue. /var/run/nologin contains > > System is booting up. See pam_nologin(8) > > I have had to hack a service together to continually delete that file to let > me use the system at all :-( I've had an identical case: "System is booting up. See pam_nologin(8)". I hard reset, during which the boot process went into selinux relabeling. After that my selinux config (disabled) is ignored: sestatus: SELinux status: enabled SELinuxfs mount: /sys/fs/selinux SELinux root directory: /etc/selinux Loaded policy name: targeted Current mode: permissive Mode from config file: disabled Policy MLS status: enabled Policy deny_unknown status: allowed Max kernel policy version: 28 Left it like this (wouldn't know how to change, as my config is ignored), and did not have the /var/run/nologin issue since. Note that this happened after applying libselinux packages: Updated libselinux-2.1.13-19.fc20.i686 @?fedora Updated libselinux-2.1.13-19.fc20.x86_64 @?fedora Update 2.2.1-4.fc20.i686 @updates Update 2.2.1-4.fc20.x86_64 @updates Updated libselinux-devel-2.1.13-19.fc20.x86_64 @?fedora Update 2.2.1-4.fc20.x86_64 @updates Updated libselinux-python-2.1.13-19.fc20.x86_64 @?fedora Update 2.2.1-4.fc20.x86_64 @updates Updated libselinux-utils-2.1.13-19.fc20.x86_64 @?fedora Update 2.2.1-4.fc20.x86_64 @updates Ah, OK, I think I've got it. It looks like /etc/cron.daily/prelink is running at 3PM local time, and as part of its execution it does a "telinit u", which causes systemd to re-exec itself. In doing so, it effectively does its reboot things and calls systemd-tmpfiles, creating /var/run/nologin. I haven't completely confirmed this, but certainly running telinit u puts the system into a very strange state. Running telinit u does not create /run/nologin for me. According to the manpage, all it does is execute systemctl daemon-reload and I don't see why should that execute systemd-tmpfiles.
It's only a guess that this bug is that something is executing systemd-tmpfiles --create. I don't see where this was documented. All it appears to be is that it's a guess. So, changing what the --create option does, and replacing it with a different option, from my viewpoint, is not really a guaranteed fix for this bug. All that's known here is that:
1) systemd-tmpfiles --create gets executes as part of a normal system boot, from systemd-tmpfiles-setup.service
2) Sometimes /run/nofiles remains after boot finishes.
Hunting around, it appears that /run/nofiles is supposed to be removed by systemd-user-sessions.service
# systemctl status -l systemd-user-sessions.service
systemd-user-sessions.service - Permit User Sessions
Loaded: loaded (/usr/lib/systemd/system/systemd-user-sessions.service; static)
Active: active (exited) since Sat 2014-01-04 09:56:10 EST; 12h ago
Docs: man:systemd-user-sessions.service(8)
Main PID: 729 (code=exited, status=0/SUCCESS)
CGroup: /system.slice/systemd-user-sessions.service
Jan 04 09:56:10 monster.email-scan.com systemd[1]: Starting Permit User Sessions...
Jan 04 09:56:10 monster.email-scan.com systemd[1]: Started Permit User Sessions.
Still, I ended up with /run/nologin blocking logins, ten hours later.
I think it's just as likely is that systemd-user-sessions for some reason fails to remove /run/nologin, than something else randomly executing systemd-tmpfiles.
To clarify my comment -- it was ten hours later that, by chance, I discovered that I run /run/nofiles blocking ssh access. I can't say with 100% confidence, but I'm fairly certain that the log's timestamp matches the timestamp of the /run/nologin file, suggesting that the bug is not spurious execution of systemd-tmpfiles, so changing what its --create option does probably is not going to do much good, here. I haven't seen a recurrence of the problem since disabling prelink, so there's at least a correlation even if there's no clear causal connection. When I look in my logs after a telinit u it looks like systemd is doing everything it would on a reboot. What would affect telinit u's behaviour? But perhaps there's multiple causes for this symptom, and they've yet to be identified? I've started to experience it yesterday latest packages upgraded: Jan 06 19:34:20 Updated: poppler-0.24.3-3.fc20.x86_64 Jan 06 19:34:21 Updated: gnutls-3.1.18-3.fc20.x86_64 Jan 06 19:34:22 Updated: gnutls-dane-3.1.18-3.fc20.x86_64 Jan 06 19:34:22 Updated: 1:cups-filesystem-1.7.0-9.fc20.noarch Jan 06 19:34:23 Updated: 1:cups-libs-1.7.0-9.fc20.x86_64 Jan 06 19:34:24 Updated: grep-2.16-1.fc20.x86_64 Jan 06 19:34:25 Updated: ntpdate-4.2.6p5-18.fc20.x86_64 Jan 06 19:34:26 Updated: ntp-4.2.6p5-18.fc20.x86_64 Jan 06 19:34:29 Updated: 1:cups-1.7.0-9.fc20.x86_64 Jan 06 19:34:30 Updated: gnutls-utils-3.1.18-3.fc20.x86_64 Jan 06 19:34:30 Updated: poppler-glib-0.24.3-3.fc20.x86_64 Jan 06 19:34:31 Updated: poppler-qt-0.24.3-3.fc20.x86_64 Jan 06 19:34:32 Updated: poppler-utils-0.24.3-3.fc20.x86_64 Jan 06 19:34:37 Updated: bjnplugin-2.4.128.5-1.x86_64 Jan 06 19:34:38 Updated: dnf-0.4.10-1.fc20.noarch Jan 06 19:34:39 Updated: hwdata-0.259-1.fc20.noarch Jan 06 19:34:41 Updated: kexec-tools-2.0.4-18.fc20.x86_64 Jan 06 19:34:41 Updated: 1:cups-libs-1.7.0-9.fc20.i686 Jan 06 19:34:42 Updated: gnutls-3.1.18-3.fc20.i686 Jan 07 22:04:24 Updated: libkkc-common-0.3.2-1.fc20.noarch Jan 07 22:04:26 Updated: libkkc-0.3.2-1.fc20.x86_64 Jan 07 22:04:27 Updated: krb5-libs-1.11.3-38.fc20.x86_64 Jan 07 22:04:29 Updated: krb5-workstation-1.11.3-38.fc20.x86_64 Jan 07 22:04:32 Updated: ibus-kkc-1.5.19-1.fc20.x86_64 Jan 07 22:04:34 Updated: setroubleshoot-plugins-3.0.59-1.fc20.noarch Jan 07 22:04:35 Updated: 1:net-snmp-libs-5.7.2-16.fc20.x86_64 Jan 07 22:04:36 Updated: libbluray-0.5.0-2.fc20.x86_64 Jan 07 22:04:40 Updated: rubygem-rhc-1.18.2-1.fc20.noarch Jan 07 22:04:47 Updated: 2:postfix-2.10.2-3.fc20.x86_64 Jan 07 22:04:56 Updated: qtwebkit-2.3.3-3.fc20.x86_64 Jan 07 22:05:02 Updated: qtwebkit-2.3.3-3.fc20.i686 Jan 07 22:05:04 Updated: krb5-libs-1.11.3-38.fc20.i686 Just an FYI.... Other distros have seen similar earlier on.... https://bugzilla.novell.com/show_bug.cgi?id=811793 I tried disabling /etc/cron.daily/prelink as per some comments above and it did seem to work for a while. However, I reinstalled my system yesterday from scratch (previously it was upgraded from FC19) and the issue is happening again, but there's no /etc/cron.daily/prelink, so it can't be the culprit of this. I had the same problem today when trying to unlock my session at 3:30pm... Here is my journalctl log at 3:00pm local time (14:00 UTC) Jan 12 14:00:01 localhost.localdomain systemd[1]: Starting Session 479 of user root. Jan 12 14:00:01 localhost.localdomain systemd[1]: Started Session 479 of user root. Jan 12 14:00:01 localhost.localdomain systemd[1]: Starting Session 480 of user root. Jan 12 14:00:01 localhost.localdomain systemd[1]: Started Session 480 of user root. Jan 12 14:00:01 localhost.localdomain CROND[27146]: (root) CMD (wget XXXX) Jan 12 14:00:01 localhost.localdomain CROND[27147]: (root) CMD (wget XXXX) Jan 12 14:00:01 localhost.localdomain kernel: traps: polkitd[19481] general protection ip:7f61367f6022 sp:7fff55c657b0 error:0 in libmozjs-17.0.so[7f61366b7000+3b3000] Jan 12 14:00:01 localhost.localdomain gnome-session[3409]: PolicyKit daemon disconnected from the bus. Jan 12 14:00:01 localhost.localdomain gnome-session[3409]: We are no longer a registered authentication agent. The wget command is custom, and added by myself so it should have nothing to do with the problem, except that there is a crash at that time... I hope this'll help. Frederic. I'm experiencing a similar problem with a different build but with mostly the same patchset as fedora's systemd. For me, the trigger seems to be enabling or disabling a sysvinit script using systemctl. For whatever reason, the bus call to reload systemd after shelling out to chkconfig seems to cause a some kind of failed reload and leave things in this bad state with the /run/nologin file. systemctl --system daemon-reload <-- works fine systemctl disable --no-reload somesysvinitscript <-- works fine systemctl disable somesysvinitscript <-- fails and creates a /run/nologin file. Can someone check to see if this same trigger case affects fedora? NB: As a side note, there is also a cosmetic bug when enabling a sysvinit script via systemctl which warns that there is no [Install] section on the native unit (which does not exist). I've sent a patch to (hopefully) resolve this upstream already. In case this is the same cause as ours, here are the patches that fixed it for me: chkconfig: http://svnweb.mageia.org/packages/cauldron/chkconfig/current/SOURCES/chkconfig-1.3.61-no-systemd-reload.patch?revision=566487&view=markup&pathrev=566487 systemd: http://svnweb.mageia.org/packages/cauldron/systemd/current/SOURCES/0513-systemctl-Do-not-attempt-native-calls-for-enable-dis.patch?revision=566488&view=markup&pathrev=566488 and http://svnweb.mageia.org/packages/cauldron/systemd/current/SOURCES/0514-systemctl-Ensure-the-no-reload-and-no-redirect-optio.patch?revision=566488&view=markup&pathrev=566488 Fingers crossed it's the same issue for you guys. Small update to yesterday's post. As well as the above, another trigger case seems to be the following: systemctl --no-block daemon-reload; systemctl --no-block daemon-reload or systemctl daemon-reexec; systemctl daemon-reexec or telinit u; telinit u Basically just repeating either command in very quick succession. NOrmally daemon-reload will wait for a reply before exiting, but --no-block allows the second command to start sooner. The other commands naturally don't wait for replies, so don't need the --no-block option. Any of the above triggers the problem every time for me, but I suspect you may need a reasonably fast machine to do so. Obviously these are only examples. Some kind of tests on Fedora boxes would be really appreciated :) systemd-208-11.fc20 has been submitted as an update for Fedora 20. https://admin.fedoraproject.org/updates/systemd-208-11.fc20 Started experiencing this tonight. I logged in, ran yum upgrade, then locked my screen, when I came back I had this error. Here are the packages I updated on my system tonight. Jan 15 19:51:07 Updated: device-mapper-multipath-libs-0.4.9-56.fc20.x86_64 Jan 15 19:51:08 Updated: firewalld-0.3.9-1.fc20.noarch Jan 15 19:51:08 Updated: kpartx-0.4.9-56.fc20.x86_64 Jan 15 19:51:08 Updated: device-mapper-multipath-0.4.9-56.fc20.x86_64 Jan 15 19:51:08 Updated: firewall-config-0.3.9-1.fc20.noarch Jan 15 19:51:08 Updated: whois-5.1.1-1.fc20.x86_64 Jan 15 19:51:08 Updated: cifs-utils-6.3-1.fc20.x86_64 Jan 15 19:51:11 Updated: gnome-software-3.10.4-2.fc20.x86_64 Jan 15 19:51:11 Updated: 2:microcode_ctl-2.1-2.fc20.x86_64 Jan 15 19:51:11 Updated: gnome-disk-utility-3.10.0-2.fc20.x86_64 Jan 15 19:51:11 Updated: nettle-2.7.1-3.fc20.x86_64 ran yum history undo and the problem seems to have stopped. I will wait a day or so before attempting to upgrade again. Package systemd-208-11.fc20: * should fix your issue, * was pushed to the Fedora 20 testing repository, * should be available at your local mirror within two days. Update it with: # su -c 'yum update --enablerepo=updates-testing systemd-208-11.fc20' as soon as you are able to. Please go to the following url: https://admin.fedoraproject.org/updates/FEDORA-2014-0902/systemd-208-11.fc20 then log in and leave karma (feedback). Can someone please try my trigger cases posted in comment 20? I can still trigger this problem easily via these mechanisms using an almost identical build to the latest fedora update, so it would be very good to get more feedback for upstream (I've got a few threads going on the ML there) I looked further as to why prelink was a trigger case. We have pretty much the same prelink package in Mageia also. Looking at the cronjob it has a "telinit u" call, but the binary src/main.c invokes "/sbin/init U" which does the same thing. So we basically have two daemon-reexecs in rapid succession, thus the trigger code in comment 20 holds true. The simplest way to avoid the crash from prelink would appear to be to simply remove the (apparently redundant) call to "telinit u" from the cronjob. I did that in a patch to our prelink package here: http://svnweb.mageia.org/packages?view=revision&revision=566667 The full fix is obviously to prevent the race in the first place, but this should let things carry on for now. Happened here too, F20, after "yum upgrade" /var/run/nologin had been created. Moved the file out of the way and tried Colin's tests: all 3 of them each recreated that file. Also, "runlevel" comes back with "unknown", although it should say "N 5" or something. And "yum update --enablerepo=updates-testing systemd-208-11.fc20" comes back with "No package systemd-208-11.fc20 available". Currently systemd-208-9.fc20 is installed. Why the NEEDINFO, are logfiles needed? Installing systemd directly from koji did the trick, now /var/run/login is no longer created when running Colin's tests. However, "runlevel" and "who -r" gets confused and still prints "unknown" after reloading systemd. For the record, installation was done with: $ sudo yum install http://kojipkgs.fedoraproject.org//packages/systemd/208/11.fc20/i686/systemd-208-11.fc20.i686.rpm http://kojipkgs.fedoraproject.org//packages/systemd/208/11.fc20/i686/systemd-libs-208-11.fc20.i686.rpm http://kojipkgs.fedoraproject.org//packages/systemd/208/11.fc20/i686/libgudev1-208-11.fc20.i686.rpm http://kojipkgs.fedoraproject.org//packages/systemd/208/11.fc20/i686/systemd-python-208-11.fc20.i686.rpm I've come into this a little late, but I've been bitten by this a few times as well, especially after yum update. Now knowing it is caused by things like systemctl daemon-reload, I can see that is in the preinstall and postinstall scripts for initscripts. I'm sure it is in other RPMs as well. (In reply to Christian Kujau from comment #27) > And "yum update --enablerepo=updates-testing systemd-208-11.fc20" comes back > with "No package systemd-208-11.fc20 available". Currently > systemd-208-9.fc20 is installed. Yes, I unpushed the update because there were regressions which I didn't have the time to properly investigate. It should be back in a few days. > Why the NEEDINFO, are logfiles needed? It's not needed anymore. The reasons are clear. I had this happen to my home system at 4:02 this morning, and I see that the logrotate apparently happened at 4:02 as well (at least 4:02 are the first entries in /var/log/messages). (In reply to Tom Horsley from comment #31) > I had this happen to my home system at 4:02 this morning, and I see that the > logrotate apparently happened at 4:02 as well (at least 4:02 are the first > entries in /var/log/messages). This is likely due to the prelink daily cron job which presumably runs at this time too. It will trigger to daemon-reexec's as noted above in comment 26. s/trigger to daemon-reexec's/trigger two daemon-reexec's/ (sorry fingers faster than brain!) I have prelinking turned off in my /etc/sysconfig/prelink file, so unless it is running side effects anyway even though it is turned off, it shouldn't be prelink doing it. (or maybe it no longer pays attention to /etc/sysconfig/prelink?) I did temporarily fix this by making a /etc/cron.daily/zzzzzz-nologin script that does an rm -rf /var/run/nologin so it gets removed as the last thing in the daily cron jobs that wind up creating it. (In reply to Tom Horsley from comment #35) > I did temporarily fix this by making a /etc/cron.daily/zzzzzz-nologin script > that does an rm -rf /var/run/nologin so it gets removed as the last thing in > the daily cron jobs that wind up creating it. Note that the /run/nologin file is not the only side effect of this problem. The general state of systemd is basically corrupted so it's definitely better to avoid the problem if at all possible. Chances are it's something that calls "systmectl daemon-reexec", "telinit u", or "init U" in quick succession. Tracking it down can be a pain. I replaced telinit with a script that wrote out a log file with pstree output so I could see clearly what run it. Anyway, the real solution is to fix the underlying problem! Still running systemd-208-11.fc20 for a few days now and it fixed actually two issues here: oftentimes when I open my laptop and it wakes up from suspend-to-ram only the blue fedora-logo would be displayed but no login window. The only way to get back into the system was to switch to a text console (alt-ctrl-f2), login as root and then I would sometimes see that dreaded "System is booting up" message, most probably caused by this very bug. But with systemd-208-11.fc20 both the b0rked resume-from-suspend and the "System is booting up" message hasn't appeared ever since. Will systemd-208-11.fc20 (or some later version) be pushed out again? Another possible side effect happening maybe due to the same root problem is bug 1057614 where I report my usb mouse gets reset every night. Even more stuff was completely borked this morning. I found rsyslogd using 100% of the cpu. I found journalctl --verify reporting corrupted files (Hurrah for binary formal logs!). Rebooting didn't fix anything. I had to boot into fed 19 temporarily, remove the /var/log/journal/yadda-yadda directory, then boot back into f20 to finally get a system that functioned. This is NOT caused by prelink, by the way, because I changed the cron.daily prelink script to exit on the 1st line, so something else is wreaking all the havoc here. I suggest reverting back to fedora 19's systemd until massive fixes are made. Look! Bug 1057883 is probably also related! What fun! I find "yum update" can screw up the system just like cron.daily One possibly relevant tidbit: My desktop at work is a near identical fedora 20 system, but I haven't seen nearly as many problems there. My system at home gets screwed up incessantly. One big difference is the hardware. I have an ssd system disk at home (not to mention a very fast cpu), and at work, I have an older dell with slow disk and not very fast cpu. If there is indeed some sort of race condition, then maybe that explains why it happens on my home system so much. My old and slow desktop at work has definitely survived several cron.daily cycles without me seeing any of these problems on it. Just as a (probably useless) datapoint, I tried modifying both logrotate and prelink in /etc/cron.daily to just have "exit 0" as their first line, and my system still goes funny in the head every day at cron.daily time. I originally opened this ticket. Yesterday I reinstalled my system. Originally I had installed FC20-beta which then upgraded itself through to F20. This time I installed afresh, based on the F20 ISO downloaded yesterday (and immediately fully upgraded before even using). I'm amazed to say how several applications present themselves quite differently (I would have thought they'd be identical). The nologin problem immediately re-appeared the very first time I suspended and unsuspended my laptop. It's extremely annoying - 99% of users will only be able to work around it by rebooting. I'm finding even Ctr-Alt-F2 doesn't work reliably any more (compared with the F18 system I had on the same hardware previously) In any case, I've solved it by putting a service on the system that runs a while loop to remove /var/run/nologin every 10 seconds. A comment above said this should have been fixed 7+ days ago, well I can say that is not the case :-( New data: Running a "yum update" which did no updates of any kind still put my system into the funny state. That reminded me that I have my own yum plugin called after-yum-hook that runs a script to fix various things yum updates often put back into a state I don't like. Among those things were updates to various libvirt packages always re-enabling the ksm and ksmtuned service, so one of my after yum hook scripts does this: systemctl disable ksm.service systemctl disable ksmtuned.service As soon as I run that script, the system goes funny in the head. My USB mouse is reset, systemd login1 stops working and times out all the time, etc., etc. The interesting thing is that those services apparently don't even exist in fedora 20, but running those commands in my after yum hook sends my system into a nose dive. I'll comment out those lines and see if the system behaves better after cron.daily tonight... Definitely a race condition. I rebooted to get my system normal again, and tried this: systemctl disable supercalifragilistic.service Nothing funny happened. So next I tried this: systemctl disable supercalifragilistic.service ; systemctl disable expiallydosius.service Ka-boom! Running two disable commands in quick succession blowed my system up. As soon as I ran the commands, this stuff started spewing to the log: Jan 26 19:24:46 zooty alsactl[2366]: alsactl 1.0.27.2 daemon started Jan 26 19:24:46 zooty bluetoothd[2370]: Bluetooth daemon 5.13 Jan 26 19:24:46 zooty alsactl[2366]: alsactl daemon stopped Jan 26 19:24:46 zooty kernel: [ 152.258106] EXT4-fs (sdb2): re-mounted. Opts: (null) Jan 26 19:24:46 zooty systemd-journald[2374]: File /var/log/journal/ce1d14df659e254dad8570fc5d633b2b/system.journal corrupted or uncleanly shut down, renaming and replacing. Jan 26 19:24:46 zooty alsactl[563]: alsactl daemon stopped Jan 26 19:24:46 zooty systemd-journald[2374]: Vacuuming done, freed 0 bytes Jan 26 19:24:46 zooty bluetoothd[2370]: Starting SDP server Jan 26 19:24:46 zooty bluetoothd[2370]: binding L2CAP socket: Address already in use Jan 26 19:24:46 zooty bluetoothd[2370]: Server initialization failed Jan 26 19:24:46 zooty bluetoothd[2370]: Bluetooth management interface 1.3 initialized Jan 26 19:24:46 zooty bluetoothd[2370]: l2cap_bind: Address already in use (98) Jan 26 19:24:46 zooty bluetoothd[2370]: Failed to listen on control channel Jan 26 19:24:46 zooty bluetoothd[2370]: input-hid: Operation not permitted (1) Jan 26 19:24:46 zooty bluetoothd[2370]: l2cap_bind: Address already in use (98) Jan 26 19:24:46 zooty bluetoothd[2370]: network-panu: Invalid argument (22) Jan 26 19:24:46 zooty bluetoothd[2370]: l2cap_bind: Address already in use (98) Jan 26 19:24:46 zooty bluetoothd[2370]: network-gn: Invalid argument (22) Jan 26 19:24:46 zooty bluetoothd[2370]: l2cap_bind: Address already in use (98) Jan 26 19:24:46 zooty bluetoothd[2370]: network-nap: Invalid argument (22) Jan 26 19:24:46 zooty bluetoothd[2370]: l2cap_bind: Address already in use (98) Jan 26 19:24:46 zooty bluetoothd[2370]: avrcp-controller: Protocol not supported (93) Jan 26 19:24:46 zooty dbus[2434]: [system] Activating systemd to hand-off: service name='org.freedesktop.hostname1' unit='dbus-org.freedesktop.hostname1.service' Jan 26 19:24:46 zooty dbus-daemon: dbus[2434]: [system] Activating systemd to hand-off: service name='org.freedesktop.hostname1' unit='dbus-org.freedesktop.hostname1.service' Jan 26 19:24:46 zooty systemd-journald[2374]: Received request to flush runtime journal from PID 1 Jan 26 19:24:46 zooty bluetoothd[2370]: l2cap_bind: Address already in use (98) Jan 26 19:24:46 zooty bluetoothd[2370]: audio-avrcp-target: Protocol not supported (93) Jan 26 19:24:46 zooty dbus-daemon: dbus[2434]: [system] Activating systemd to hand-off: service name='org.freedesktop.ColorManager' unit='colord.service' Jan 26 19:24:46 zooty dbus[2434]: [system] Activating systemd to hand-off: service name='org.freedesktop.ColorManager' unit='colord.service' Jan 26 19:25:01 zooty dbus[2434]: [system] Activating systemd to hand-off: service name='org.freedesktop.login1' unit='dbus-org.freedesktop.login1.service' ... I concur with the issue with rsyslogd. I have also seen it using 100% of the CPU. I was trying to find out why fail2ban wasn't working, and it seems that nothing is getting logged into /var/log/secure anymore.. That seems to be because rsyslogd was spinning its wheels.. Apparently it doesn't even require a fast system to reproduce the problem in commend #45. I just tried it on my system at work, and as soon as I issued the two disable commands, this spewed into /var/log/messages: Jan 27 07:25:53 tomh systemd: Reloading. Jan 27 07:25:53 tomh systemd: [/usr/lib/systemd/system/rtkit-daemon.service:32] Unknown lvalue 'ControlGroup' in section 'Service' Jan 27 07:25:54 tomh systemd: Mounted Configuration File System. Jan 27 07:25:54 tomh systemd: Started CUPS Printing Service. Jan 27 07:25:54 tomh systemd: Created slice Root Slice. Jan 27 07:25:54 tomh systemd: Activated swap /dev/disk/by-label/SWAP-sda5. Jan 27 07:25:54 tomh systemd: Found device ST3500630AS. Jan 27 07:25:54 tomh systemd: Mounted /caliban. Jan 27 07:25:54 tomh systemd: Mounted /mainboot. Jan 27 07:25:54 tomh systemd: Mounted /fedora19. Jan 27 07:25:54 tomh systemd: Mounted /. Jan 27 07:25:54 tomh systemd: Started File System Check on /dev/disk/by-label/FEDORA19. Jan 27 07:25:54 tomh systemd: Found device ST3500630AS. Jan 27 07:25:54 tomh systemd: Started File System Check on /dev/disk/by-label/MAINBOOT. Jan 27 07:25:54 tomh systemd: Found device ST3500630AS. Jan 27 07:25:54 tomh systemd: Started File System Check on /dev/disk/by-label/CALIBAN. Jan 27 07:25:54 tomh systemd: Found device ST3500630AS. Jan 27 07:25:54 tomh systemd: Reached target Sound Card. Jan 27 07:25:54 tomh systemd: Found device ST3500630AS. Jan 27 07:25:54 tomh systemd: Found device ST3500630AS. Jan 27 07:25:54 tomh systemd: Found device ST3500630AS. Jan 27 07:25:54 tomh systemd: Found device ST3500630AS. Jan 27 07:25:54 tomh systemd: Found device ST3500630AS. Jan 27 07:25:54 tomh systemd: Found device ST3500630AS. Jan 27 07:25:54 tomh systemd: Found device ST3500630AS. Jan 27 07:25:54 tomh systemd: Found device ST3500630AS. Jan 27 07:25:54 tomh systemd: Found device ST3500630AS. Jan 27 07:25:54 tomh systemd: Found device ST3500630AS. Jan 27 07:25:54 tomh systemd: Found device ST3500630AS. Jan 27 07:25:54 tomh systemd: Found device ST3500630AS. Jan 27 07:25:54 tomh systemd: Found device ST3500630AS. Jan 27 07:25:54 tomh systemd: Found device ST3500630AS. Jan 27 07:25:54 tomh systemd: Found device ST3500630AS. Jan 27 07:25:54 tomh systemd: Found device ST3500630AS. Jan 27 07:25:54 tomh systemd: Activated swap /dev/disk/by-id/ata-ST3500630AS_9QG7J060-part5. Jan 27 07:25:54 tomh systemd: Activated swap /dev/disk/by-uuid/fff9de2e-683c-4784-a40a-acd4e70627c5. Jan 27 07:25:54 tomh systemd: Activated swap /dev/sda5. Jan 27 07:25:54 tomh systemd: Reloading. Jan 27 07:25:54 tomh systemd: [/usr/lib/systemd/system/rtkit-daemon.service:32] Unknown lvalue 'ControlGroup' in section 'Service' Jan 27 07:25:54 tomh systemd: Mounted Configuration File System. Jan 27 07:25:54 tomh systemd: Started CUPS Printing Service. Jan 27 07:25:54 tomh systemd: Created slice Root Slice. Jan 27 07:25:54 tomh systemd: Activated swap /dev/disk/by-label/SWAP-sda5. Jan 27 07:25:54 tomh systemd: Found device ST3500630AS. Jan 27 07:25:54 tomh systemd: Mounted /caliban. Jan 27 07:25:54 tomh systemd: Mounted /mainboot. Jan 27 07:25:54 tomh systemd: Mounted /fedora19. Jan 27 07:25:54 tomh systemd: Mounted /. Jan 27 07:25:54 tomh systemd: Started File System Check on /dev/disk/by-label/FEDORA19. Jan 27 07:25:54 tomh systemd: Found device ST3500630AS. Jan 27 07:25:54 tomh systemd: Started File System Check on /dev/disk/by-label/MAINBOOT. Jan 27 07:25:54 tomh systemd: Found device ST3500630AS. Jan 27 07:25:54 tomh systemd: Started File System Check on /dev/disk/by-label/CALIBAN. Jan 27 07:25:54 tomh systemd: Found device ST3500630AS. Jan 27 07:25:54 tomh systemd: Reached target Sound Card. Jan 27 07:25:54 tomh systemd: Found device ST3500630AS. Jan 27 07:25:54 tomh systemd: Found device ST3500630AS. Jan 27 07:25:54 tomh systemd: Found device ST3500630AS. Jan 27 07:25:54 tomh systemd: Found device ST3500630AS. Jan 27 07:25:54 tomh systemd: Found device ST3500630AS. Jan 27 07:25:54 tomh systemd: Found device ST3500630AS. Jan 27 07:25:54 tomh systemd: Found device ST3500630AS. Jan 27 07:25:54 tomh systemd: Found device ST3500630AS. Jan 27 07:25:54 tomh systemd: Found device ST3500630AS. Jan 27 07:25:54 tomh systemd: Found device ST3500630AS. Jan 27 07:25:54 tomh systemd: Found device ST3500630AS. Jan 27 07:25:54 tomh systemd: Found device ST3500630AS. Jan 27 07:25:54 tomh systemd: Found device ST3500630AS. Jan 27 07:25:54 tomh systemd: Found device ST3500630AS. Jan 27 07:25:54 tomh systemd: Found device ST3500630AS. Jan 27 07:25:54 tomh systemd: Found device ST3500630AS. Jan 27 07:25:54 tomh systemd: Activated swap /dev/disk/by-id/ata-ST3500630AS_9QG7J060-part5. Jan 27 07:25:54 tomh systemd: Activated swap /dev/disk/by-uuid/fff9de2e-683c-4784-a40a-acd4e70627c5. Jan 27 07:25:54 tomh systemd: Activated swap /dev/sda5. Jan 27 07:26:01 tomh systemd: Starting Session 6143 of user tweety. Jan 27 07:26:01 tomh systemd: Started Session 6143 of user tweety. Jan 27 07:27:01 tomh systemd: Starting Session 6144 of user tweety. Jan 27 07:27:01 tomh systemd: Started Session 6144 of user tweety. Jan 27 07:28:01 tomh systemd: Starting Session 6145 of user tweety. Jan 27 07:28:01 tomh systemd: Started Session 6145 of user tweety. (In reply to Colin Guthrie from comment #36) > (In reply to Tom Horsley from comment #35) > > I did temporarily fix this by making a /etc/cron.daily/zzzzzz-nologin script > > that does an rm -rf /var/run/nologin so it gets removed as the last thing in > > the daily cron jobs that wind up creating it. > > Note that the /run/nologin file is not the only side effect of this problem. > The general state of systemd is basically corrupted so it's definitely > better to avoid the problem if at all possible. > > Chances are it's something that calls "systmectl daemon-reexec", "telinit > u", or "init U" in quick succession. Tracking it down can be a pain. I > replaced telinit with a script that wrote out a log file with pstree output > so I could see clearly what run it. Anyway, the real solution is to fix the > underlying problem! Is it possible that one of the recent updates in F20 is calling one of these commands in post/preun/postun? Just a thought. (In reply to Jason Haar from comment #43) > I originally opened this ticket. Yesterday I reinstalled my system. > Originally I had installed FC20-beta which then upgraded itself through to > F20. This time I installed afresh, based on the F20 ISO downloaded yesterday > (and immediately fully upgraded before even using). I'm amazed to say how > several applications present themselves quite differently (I would have > thought they'd be identical). > > The nologin problem immediately re-appeared the very first time I suspended > and unsuspended my laptop. It's extremely annoying - 99% of users will only > be able to work around it by rebooting. I'm finding even Ctr-Alt-F2 doesn't > work reliably any more (compared with the F18 system I had on the same > hardware previously) I can confirm this. I've been able to recreate this problem numerous times now. Fresh F20 install, yum update, and the /run/nologin is created. I've had no luck figuring our which update is responsible for this. > In any case, I've solved it by putting a service on the system that runs a > while loop to remove /var/run/nologin every 10 seconds. > > A comment above said this should have been fixed 7+ days ago, well I can say > that is not the case :-( Agree. I'm not suggesting that the fix to systemd is wrong, but I am not convinced it will fix the problem. The updated systemd package would have to be installed prior to the offending update, would it not? (In reply to Ryan O'Hara from comment #48) > (In reply to Colin Guthrie from comment #36) > > (In reply to Tom Horsley from comment #35) > > > I did temporarily fix this by making a /etc/cron.daily/zzzzzz-nologin script > > > that does an rm -rf /var/run/nologin so it gets removed as the last thing in > > > the daily cron jobs that wind up creating it. > > > > Note that the /run/nologin file is not the only side effect of this problem. > > The general state of systemd is basically corrupted so it's definitely > > better to avoid the problem if at all possible. > > > > Chances are it's something that calls "systmectl daemon-reexec", "telinit > > u", or "init U" in quick succession. Tracking it down can be a pain. I > > replaced telinit with a script that wrote out a log file with pstree output > > so I could see clearly what run it. Anyway, the real solution is to fix the > > underlying problem! > > Is it possible that one of the recent updates in F20 is calling one of these > commands in post/preun/postun? Just a thought. OK. Now I see that this is precisely what is being asked in comment #6. After digging through all the updates that were applied, I found only one that called 'systemd-tmpfiles --create' in POSTIN:
# rpm -q --queryformat "%{POSTIN}\n" samba-common
/sbin/ldconfig
/usr/bin/systemd-tmpfiles --create /usr/lib/tmpfiles.d/samba.conf
if [ -d /var/cache/samba ]; then
mv /var/cache/samba/netsamlogon_cache.tdb /var/lib/samba/ 2>/dev/null
mv /var/cache/samba/winbindd_cache.tdb /var/lib/samba/ 2>/dev/null
rm -rf /var/cache/samba/
ln -sf /var/cache/samba /var/lib/samba/
fi
Could samba-common be the offending package?
(In reply to Ryan O'Hara from comment #51) > After digging through all the updates that were applied, I found only one > that called 'systemd-tmpfiles --create' in POSTIN: > > # rpm -q --queryformat "%{POSTIN}\n" samba-common > /sbin/ldconfig > /usr/bin/systemd-tmpfiles --create /usr/lib/tmpfiles.d/samba.conf > if [ -d /var/cache/samba ]; then > mv /var/cache/samba/netsamlogon_cache.tdb /var/lib/samba/ 2>/dev/null > mv /var/cache/samba/winbindd_cache.tdb /var/lib/samba/ 2>/dev/null > rm -rf /var/cache/samba/ > ln -sf /var/cache/samba /var/lib/samba/ > fi > > Could samba-common be the offending package? Ignore this. I am fairly confident that the package that is responsible for creating /run/nologin is the updated 'initscripts' package. This took a fair bit of debugging to track down. I just installed a fresh F20 system and *only* updated initscripts. This caused the /run/nologin to be created. Whether the bug here is the creation of the file vs the failure to remove it is a different matter. Could someone please sanity check this? Run 'yum update initscripts' on a freshly installed F20 machine and see if /run/nologin file is created. I think comment #44 is a key piece of information pointing to an actual bug here (that may sometimes be triggered by things rpm scripts do). Running two "systemctl disable" commands in quick succession *always* triggers all the mysterious things getting reset behavior (even when the services mentioned don't exist). Running them one at a time with a nice pause in between has no mysterious side effects, so it isn't anything disable is doing by itself, it is some kind of race that confuses everything. (In reply to Tom Horsley from comment #53) > I think comment #44 is a key piece of information pointing to an actual bug > here. Not really. The problem has been identified some time back as a serialization race in systemd (see my comment #20 through #26), the only things that are really adding to the discussion now is finding all the things that can trigger this. We already know that things like prelink will trigger a "telinit u" and run "init U" thus trigger this bug; we know that enabling or disabling services will trigger reloads, especially in combination with unknown or sysvinit scripts (and as a result, updating packages that specially do enables/disables or daemon-reloads are affected - initscripts package does two of these things so it pretty much guaranteed to trigger the bug!) So really the problem is pretty well understood now. All that remains is to actually fix it which is something I'll be looking at on Thursday/Friday. I guess I just find it more peculiar that two commands which should do absolutely nothing manage to trigger this race which winds up resetting the universe. (In reply to Tom Horsley from comment #55) > I guess I just find it more peculiar that two commands which should do > absolutely nothing manage to trigger this race which winds up resetting the > universe. Yeah, systemctl seems to always reload systemd on enable/disable operations even if there are no units to work with... I'll add this to my list of fixes (upstream has a change which avoids the reload if only sysvinit scripts are used assuming that chkconfig will trigger the reload, but it doesn't avoid the reload if it's a noop on invalid native units - nor does it give any error. I'll try and address this). >Yeah, systemctl seems to always reload systemd on enable/disable operations even >if there are no units to work with...
But in that case, wouldn't just a single disable command cause the reload? Why does it only happen when there are two disables in a row?
(In reply to Tom Horsley from comment #57) > >Yeah, systemctl seems to always reload systemd on enable/disable operations even >if there are no units to work with... > > But in that case, wouldn't just a single disable command cause the reload? > Why does it only happen when there are two disables in a row? Read comment #20 again. There is a race problem here. Reloading systemd once in isolation is fine and works (it does seem to trigger Type=oneshot services again which seems wrong to me but that's another issue). Reloading twice in quick succession - e.g. when the first reload is still being processed is what triggers the bug. As I mentioned above, the prelink cronjob *does* call reload twice (actually it calls reexec but the race is the same there), once in the cron script itself via calling "telinit u" and a second time from inside the prelink binary itself which shells out to "init U" (which is the same thing). That's why it triggers it. In the case of enabling sysvinit scripts, it's triggered because both chkconfig (which systemctl shells out to) and systemctl itself both do a reload. If you used chkconfig directly for sysvinit scripts you'd only get one reload and thus not trigger the problem. This is all explained in the comments above. >Reloading twice in quick succession - e.g. when the first reload is still being >processed is what triggers the bug.
Ah! I think I finally have my brain wrapped around the idea now :-). Thanks.
I didn't expect that the real fix for this problem had anything to do with the initscripts package. My hope was that trying to isolate the package that was triggering the problem would help others get past this until a proper fix is available. That said, I ran another test today where I updated a newly installed F20 system but excluded the initscripts package. The same problem occured, so obviously running 'yum update' has the potential to trigger this problem in numerous ways. Can we at least set this back to ASSIGNED instead of ON_QA? This is a serious problem. After this happened I was left with no way to login. The switch to virtual console is also not working, and I have ssh to root disabled. I decided to push the reset button and ended up with a corrupted file system! I then reinstalled fedora 20 and started a yum update. I went away and left it running, when I came back to check how far it was, I was once again locked out! In addition to the yum update I had a small set of yum install running in another terminal (waiting for the yum update to finish of course). I don't know if it is reproducible, like described. The lockout has happened to me several time during the last couple of days. I'm having the same exact issue as everyone else. I installed Fedora 20 from the ISO I downloaded from your website. I didn't update or anything for months. I never had the issue. Today I updated all my software via terminal "yum update" and now I'm having "System is booting up. See "pam_nologin(8)"." This only happens due to being idle and the screen locking. This is a huge issue for me because I use Fedora 20 mainly for security. I have very private protected health information and have to abide by HIPAA standards. The only work around I could find (which really isn't a work around) is to "turn off" screen lock in privacy settings. This is a huge issue for me since I NEED to lock my screen due to information stored on the computer. The only way to get past the message is to manually power off the computer and log back in. So for now I'm leaving the computer powered off until there's a fix or might possibly look at other distributions. I'm unable to ssh to my newly installed fedora: port 22: Connection refused. If I ever get in again I will remember to enable ssh ): I had the same problem trying to ssh into a remote server. I kept getting the response msg: System is booting up. See pam_nologin(8) Connection closed by xxx.xxx.xxx.xxx I had no problems with fedora 18. I deleted the file as mentioned above /var/run/nologin and then created the script /etc/cron.daily/zzzzzz-nologin which deletes the file. so after 14 hrs I can still ssh into the remote server. While this takes care of the symptom of the problem, it is not the final solution. I'm hitting this issue as well and I ask you to please work on it as fast as you can since it locks me out of my PC at least once a day, and has locked me out remotely at least once. A nice bug which brings adrenalin back to my life: Will I log in to my computer today? Surprisingly, only one of four computers is having this issue. I am also having this problem on two different F20 installations. Yum update creates the file, if yum updates anything. Some package installations via yum create the file, some do not. I created a cron job to check for the file daily. I will also set up an alias for yum update to run the update then delete that file. I am affected by this bug. My system forcibly kicked me out and their is no way to log-in. Is there a fix by downgrading or upgrading to some version of affected package. I don't want to be locked out of my system when I am doing important work. Just for some public feedback, I spent some time looking into this issue on Thursday and found that the problem appeared to be caused by a patch introduced in the previous update of systemd but said patch was reverted in the latest one (which I *think* is no longer available until this issue is resolved?) Anyway, as some people who did the upgrade still reported problems, I'm wondering if those reports are actually bogus and simply that the problem occurred *during* that update, and hasn't actually reoccurred *after* that upgrade? Anyway, all my reproduction cases could not trigger it for me any further. I sent a more detailed analysis to one of the Fedora maintainers, so there will hopefully be some movement on this bug shortly (although he is still of the opinion that there could still be an underlying problem that is much harder to trigger). You may be correct Colin I saw your comment and did a sudo yum update. After the update I don't see the file in /var/run and was able to lock the screen and unlock it with no issues. So it appears that the problem happens in release version of F20, but subsequent updates fix it. I used the CD to do the install. I wonder if it happens with net-based? While I was logged in to a Fedora 20 x86_64 machine using ssh I found the file -rw-r--r--. 1 root root 40 1. Feb 16:21 /run/nologin I searched in /var/log/yum.log for this date and time and found the following entries: Feb 01 16:21:05 Aktualisiert: glibc.x86_64 2.18-12.fc20 Feb 01 16:21:11 Aktualisiert: glibc-common.x86_64 2.18-12.fc20 Feb 01 16:21:12 Aktualisiert: pango.x86_64 1.36.1-2.fc20 Feb 01 16:21:13 Aktualisiert: clutter.x86_64 1.16.2-3.fc20 Feb 01 16:21:14 Aktualisiert: glibc-headers.x86_64 2.18-12.fc20 Feb 01 16:21:15 Aktualisiert: glibc-devel.x86_64 2.18-12.fc20 Feb 01 16:21:16 Aktualisiert: pango-devel.x86_64 1.36.1-2.fc20 Feb 01 16:21:16 Aktualisiert: libtirpc.x86_64 0.2.4-1.0.fc20 Feb 01 16:21:17 Aktualisiert: corosynclib.x86_64 2.3.3-1.fc20 Feb 01 16:21:18 Aktualisiert: corosync.x86_64 2.3.3-1.fc20 Feb 01 16:21:18 Aktualisiert: NetworkManager-glib.x86_64 1:0.9.9.0-28.git20131003.fc20 Feb 01 16:21:20 Aktualisiert: NetworkManager.x86_64 1:0.9.9.0-28.git20131003.fc20 Feb 01 16:21:21 Aktualisiert: nfs-utils.x86_64 1:1.2.9-3.0.fc20 Feb 01 16:21:25 Aktualisiert: clutter-devel.x86_64 1.16.2-3.fc20 Feb 01 16:21:27 Aktualisiert: glibc-static.x86_64 2.18-12.fc20 Feb 01 16:21:29 Aktualisiert: clutter-doc.x86_64 1.16.2-3.fc20 Feb 01 16:21:30 Aktualisiert: nscd.x86_64 2.18-12.fc20 Feb 01 16:21:30 Aktualisiert: freeglut.x86_64 2.8.1-3.fc20 Feb 01 16:21:31 Aktualisiert: rubygem-daemon_controller.noarch 1.1.8-1.fc20 Feb 01 16:21:33 Aktualisiert: glibc.i686 2.18-12.fc20 Feb 01 16:21:33 Aktualisiert: glibc-devel.i686 2.18-12.fc20 Feb 01 16:21:34 Aktualisiert: pango.i686 1.36.1-2.fc20 This happens only on one of three Fedora 20 hosts (the other hosts still run Fedora 19 because of this problem). I don't know if this will help to solve the problem. Edgar, Have you updated it since it was installed? I installed an F20 system Saturday from the live imaage on a USB key. After the first update on the installation, I experienced the issue. I removed the nologin file. I tried a yum update to recreate it again Saturday, but yum had nothing to update, so I figured it would only happen if yum actually updated anything. I saw Colin's update this morning so did another update. Among other updates I got a new kernel. After this morning's update before rebooting, I could not find the file. I tested locking/unlocking the screen and sudo several times with no issues - neither of these worked with that nologin file in place. So I believe Colin is correct. I am going to try installing from net to see if the bug is encountered there. Presumably you get the latest files that don't have the patch that causes the issues. (In reply to Colin Guthrie from comment #69) > Just for some public feedback, I spent some time looking into this issue on > Thursday and found that the problem appeared to be caused by a patch > introduced in the previous update of systemd but said patch was reverted in > the latest one (which I *think* is no longer available until this issue is > resolved?) Can you be specific about what versions of systemd you're talking about? (In reply to David Green from comment #72) > Edgar, Have you updated it since it was installed? My system noted in comment #71 was installed at January, 18, and has an uptime of 15 day currently. At install time all available updates was installed. yum-updatesd is running and configured to check and install updates every 12 hours. So updates are installed automatically when they are available. The systems runs with kernel-3.12.7-300.fc20.x86_64 since I haven't rebooted it in the meantime. systemd is systemd-208-9.fc20.x86_64, it wasn't updated since the installation. Now, half an hour ago, yum-updatesd has installed (updated) 30 packages, among these are the new kernel kernel-3.12.9-301.fc20.x86_64 . I have removed (renamed) /run/nologin before I have written comment #71. /run/nologin was not created again since this time. Just because some updates were applied and /run/nologin wasn't created doesn't mean this problem has been fixed. Not every yum update will trigger this. Can we agree on that? (In reply to Colin Guthrie from comment #69) > and found that the problem appeared to be caused by a patch > introduced in the previous update of systemd but said patch was reverted in > the latest one (which I *think* is no longer available until this issue is > resolved?) Hi Colin, please, what patch are you talking about? IMHO, there has been no systemd update since I observed this behaviour (systemd-208-9.fc20.x86_64) and it was not a Fata Morgana. Surprisingly, I am not able to reproduce it any more using a combination of above mentioned consecutive commands. Something has changed, but I have no clue what. Why is this bug treated so lightly? Effectively, a yum update results in a denial of service the moment screensaver kicks in, and this has been both known and unfixed for over a month and a half. This should definitely have a much higher severity level. Atom CPU => spreads events out more: $ stat /run/nologin File: ‘/run/nologin’ Size: 40 Blocks: 8 IO Block: 4096 regular file Device: 12h/18d Inode: 28998 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Context: system_u:object_r:systemd_logind_var_run_t:s0 Access: 2014-02-06 08:31:09.817640011 +1100 Modify: 2014-02-06 08:31:09.817640011 +1100 Change: 2014-02-06 08:31:09.817640011 +1100 /var/log/yum.log: Feb 06 08:30:53 Updated: anaconda-20.25.16-1.fc20.i686 Feb 06 08:31:00 Updated: 1:nfs-utils-1.2.9-3.0.fc20.i686 Feb 06 08:31:13 Updated: 1:NetworkManager-openvpn-0.9.9.0-0.1.git20140128.fc20.i686 Feb 06 08:31:15 Updated: python-pycurl-7.19.3-1.fc20.i686 *** Bug 1055708 has been marked as a duplicate of this bug. *** I can confirm this bug when will it be fixed? I just experienced this bug myself. It caused sudo to hang a long time before it prints the password prompt and caused a lot (all?) KDE/Qt programs to hang. For dolphin I saw in the shell that it tried to talk to a dbus daemon (something about UDisks2) but the dbus call did timeout! A dbus call that timeouts seems very broken to me. After I deleted /run/nologin and rebooted the system behaved normal again. I'm pretty sure Bug 1057883, Bug 1057811, Bug 1057618, and Bug 1057614 are all caused by the same race condition manifesting in different ways. Guten Abend! I just ran into this issue myself a few minutes ago. Just came back from dinner and wanted to unshield the system by providing my password. Sadly I got a yellow text telling me that the system is still booting. I tried to switch to console by pressing CTRL+ALT+F2 but the graphic corrupted (I recall that I ran into this a few days ago). I switched back with CTRL+ALT+F1 and found myself in a corrupt graphical shield (I wasn't able to see much but I was able to see the normal shield behaviour when pressing ESC). I decided to hibernate the notebook (I recall that I have done this a few days ago as well) when I came back from hibernation the "Shield" issue is gone and I found myself within gnome-shell (without the need to enter anything). I share the opition of comment #82 and belive that several different bugs are related but manifested in different ways. I even opened a different bugreport a few days ago but assumed that I had a wrong "keyboard layout" (US) set up accidently rather than my (DE) layout. Now that I run into the same behaviour again today (same looks, same behaviour, same situation). Maybe you might have a look at this one as well. I bet once this is solved this also solves the other issue. https://bugzilla.redhat.com/show_bug.cgi?id=1061148 I'm affected by this bug also. Fortunately /run/nologin has not reappeared in a week or so, but now there is another problem, which is most probably related. When clicking Switch User, you end up at the lock screen, and it is not possible for other users to log in. You must log out first, before the other user is able to log in. This happens on my workstation and laptop, both with Fedora 20 with latest updates. Johan: if /var/run/nologin does not exist, it is not the same problem. Sorry for not answering some questions directed at me earlier. As I've mentioned previously I'm not using the fedora builds, so cannot really give a meaningful answer to the "what version are you using" question. I'm testing code, not packages. Regarding my findings I've found that the patch: http://pkgs.fedoraproject.org/cgit/systemd.git/tree/0140-systemd-add-a-start-job-for-all-units-specified-with.patch?h=f20 Is the one that causes all the problems. I know this was released as part of a previous fedora update. Since then the patch has been reverted: http://pkgs.fedoraproject.org/cgit/systemd.git/tree/0146-Revert-systemd-add-a-start-job-for-all-units-specifi.patch?h=f20 I believe this was also issued as a fedora update but then that update was pulled. When we last talked, Zbigniew reckoned there was still a race, but in my testing, all the "easy" and reproducible ways I found to make it happen stopped working when I also reverted the patch. Here is some analysis I sent via mail: [with the patch applied] > e.g. I found that reloading systemd triggered my alsa-restore.service > unit. After some digging it seems that this is due to my sound device > having a Wants=sound.target and then sound.target then pulled in those jobs. > > systemctl show sys-devices-pci0000:00-0000:00:1b.0-sound-card0.device | > grep Wants > Wants=sound.target > > > As the patch calls manager_add_job() with override == true, I think I'm > right in saying that this triggers a start of sound.target even if it's > already started, which re-triggers the alsa-restore stuff etc. etc. > > > > On my machine, all the devices I have show the following Wants: > [root@jimmy systemd]# for dev in $(systemctl --full | grep "\.device"| > cut -d' ' -f1); do systemctl show $dev | grep Want; done > Wants=systemd-backlight > Wants=sound.target > Wants=bluetooth.target > Wants=sys-fs-fuse-connections.mount > > > Going one step further, these wanted units want: > > [root@jimmy systemd]# for want in $(for dev in $(systemctl --full | grep > "\.device"| cut -d' ' -f1); do systemctl show $dev | grep Wants; done | > cut -d'=' -f2); do systemctl show $want | grep Wants; done; > Wants=osspd.service > Wants=bluetooth.service > > > Any one more step: > [root@jimmy systemd]# systemctl show osspd.service | grep Wants > [root@jimmy systemd]# systemctl show bluetooth.service | grep Wants > Wants=system.slice > > And: > [root@jimmy systemd]# systemctl show system.slice | grep Wants > Wants=-.slice > > > > Now, I'm not sure of the semantics here, but all these "Wants" deps > certainly certainly tallies with the jobs shown in my log (attached just > for reference). > > Now I'm not 100% sure of the exact triggering mechanisms, but as it all > goes back to -.slice (via bluetooth.service) I can see it all bubbling > back up again. Ultimately causes systemd-tmpfiles-setup.service to be > started and then /run/nologin is written out as a side effect. Hope this helps. I don't want to "spam" the bug much further as I cannot really comment on specific fedora "builds", but if anyone does want to ask any specific questions, feel free. (In reply to Adam Williamson from comment #85) > Johan: if /var/run/nologin does not exist, it is not the same problem. OK, thanks! I'll try to find out. This happened in the last few days, so I haven't looked at it yet. I will report it (in a separate bug of course) if I find something useful. Just wanted to say it just happened again. After a sudo yum update /run/nologin exists. This happened this morning and it was a really obnoxious presentation. I was logged in, performed an update via yum in a terminal and walked away. Came back and the screen was locked (as to be expected, I was away for a while). Couldn't unlock my gnome session because of pam_nologin. I dropped to a virtual terminal and tried to rm /var/run/nologin which worked fine. ctrl-F1 back to the X console running and all I got was the plymouth overlay. No X screen. I couldn't ctrl-alt-Fx back to any virtual consoles and ended up having to hold the power button to shut off and lost everything on my running gnome session. this is the list of packages updated this morning: Feb 14 07:38:12 Updated: 2:libwbclient-4.1.4-1.fc20.x86_64 Feb 14 07:38:13 Updated: krb5-libs-1.11.5-2.fc20.x86_64 Feb 14 07:38:15 Updated: gstreamer1-1.2.3-1.fc20.x86_64 Feb 14 07:38:16 Updated: gstreamer1-plugins-base-1.2.3-1.fc20.x86_64 Feb 14 07:38:18 Updated: gstreamer1-plugins-good-1.2.3-1.fc20.x86_64 Feb 14 07:38:19 Updated: sqlite-3.8.3-1.fc20.x86_64 Feb 14 07:38:22 Updated: python-libs-2.7.5-10.fc20.x86_64 Feb 14 07:38:23 Updated: python-2.7.5-10.fc20.x86_64 Feb 14 07:38:25 Updated: 2:samba-libs-4.1.4-1.fc20.x86_64 Feb 14 07:38:26 Updated: 2:samba-common-4.1.4-1.fc20.x86_64 Feb 14 07:38:27 Updated: 2:libsmbclient-4.1.4-1.fc20.x86_64 Feb 14 07:38:28 Updated: 2:samba-winbind-modules-4.1.4-1.fc20.x86_64 Feb 14 07:38:28 Updated: 2:samba-winbind-4.1.4-1.fc20.x86_64 Feb 14 07:38:29 Updated: librepo-1.5.2-2.fc20.x86_64 Feb 14 07:38:30 Updated: libreport-filesystem-2.1.12-2.fc20.x86_64 Feb 14 07:38:31 Updated: libreport-2.1.12-2.fc20.x86_64 Feb 14 07:38:32 Updated: libreport-python-2.1.12-2.fc20.x86_64 Feb 14 07:38:32 Updated: abrt-libs-2.1.12-2.fc20.x86_64 Feb 14 07:38:33 Updated: libreport-web-2.1.12-2.fc20.x86_64 Feb 14 07:38:34 Updated: libreport-plugin-ureport-2.1.12-2.fc20.x86_64 Feb 14 07:38:35 Updated: abrt-2.1.12-2.fc20.x86_64 Feb 14 07:38:36 Updated: abrt-retrace-client-2.1.12-2.fc20.x86_64 Feb 14 07:38:37 Updated: abrt-python-2.1.12-2.fc20.x86_64 Feb 14 07:38:37 Updated: abrt-addon-python-2.1.12-2.fc20.x86_64 Feb 14 07:38:38 Updated: abrt-addon-ccpp-2.1.12-2.fc20.x86_64 Feb 14 07:38:39 Updated: abrt-dbus-2.1.12-2.fc20.x86_64 Feb 14 07:38:39 Updated: abrt-addon-xorg-2.1.12-2.fc20.x86_64 Feb 14 07:38:40 Updated: abrt-plugin-bodhi-2.1.12-2.fc20.x86_64 Feb 14 07:38:41 Updated: libreport-plugin-bugzilla-2.1.12-2.fc20.x86_64 Feb 14 07:38:42 Updated: libreport-plugin-kerneloops-2.1.12-2.fc20.x86_64 Feb 14 07:38:42 Updated: abrt-addon-kerneloops-2.1.12-2.fc20.x86_64 Feb 14 07:38:43 Updated: abrt-addon-pstoreoops-2.1.12-2.fc20.x86_64 Feb 14 07:38:44 Updated: abrt-addon-vmcore-2.1.12-2.fc20.x86_64 Feb 14 07:38:44 Updated: libreport-plugin-reportuploader-2.1.12-2.fc20.x86_64 Feb 14 07:38:45 Updated: libreport-gtk-2.1.12-2.fc20.x86_64 Feb 14 07:38:46 Updated: libreport-plugin-logger-2.1.12-2.fc20.x86_64 Feb 14 07:38:46 Updated: abrt-gui-libs-2.1.12-2.fc20.x86_64 Feb 14 07:38:47 Updated: abrt-gui-2.1.12-2.fc20.x86_64 Feb 14 07:38:48 Updated: libreport-fedora-2.1.12-2.fc20.x86_64 Feb 14 07:38:48 Updated: smc-fonts-common-6.0-7.fc20.noarch Feb 14 07:38:49 Updated: libsrtp-1.4.4-10.20101004cvs.fc20.x86_64 Feb 14 07:38:50 Updated: gstreamer1-plugins-bad-free-1.2.3-1.fc20.x86_64 Feb 14 07:38:51 Updated: gstreamer1-plugins-bad-free-extras-1.2.3-1.fc20.x86_64 Feb 14 07:38:53 Updated: smc-meera-fonts-6.0-7.fc20.noarch Feb 14 07:38:53 Updated: abrt-desktop-2.1.12-2.fc20.x86_64 Feb 14 07:38:54 Updated: libreport-cli-2.1.12-2.fc20.x86_64 Feb 14 07:38:54 Updated: libreport-newt-2.1.12-2.fc20.x86_64 Feb 14 07:38:55 Updated: python-librepo-1.5.2-2.fc20.x86_64 Feb 14 07:38:56 Updated: 2:samba-winbind-clients-4.1.4-1.fc20.x86_64 Feb 14 07:38:56 Updated: 2:samba-client-4.1.4-1.fc20.x86_64 Feb 14 07:38:57 Updated: 2:samba-4.1.4-1.fc20.x86_64 Feb 14 07:38:58 Updated: tkinter-2.7.5-10.fc20.x86_64 Feb 14 07:38:59 Updated: gstreamer1-plugins-good-extras-1.2.3-1.fc20.x86_64 Feb 14 07:39:00 Updated: krb5-workstation-1.11.5-2.fc20.x86_64 Feb 14 07:39:01 Updated: krb5-devel-1.11.5-2.fc20.x86_64 Feb 14 07:39:02 Updated: libgadu-1.12.0-0.3.rc2.fc20.x86_64 Feb 14 07:39:03 Updated: libgsf-1.14.29-1.fc20.x86_64 Feb 14 07:39:06 Updated: cmake-2.8.12.2-2.fc20.x86_64 Feb 14 07:39:07 Updated: lohit-gujarati-fonts-2.92.2-1.fc20.noarch Feb 14 07:39:08 Updated: xemacs-filesystem-21.5.34-5.fc20.noarch Feb 14 07:39:09 Updated: libdwarf-20140131-2.fc20.x86_64 Feb 14 07:39:09 Updated: less-458-6.fc20.x86_64 Feb 14 07:39:10 Updated: gnome-shell-extension-pidgin-0-0.12.git1a254319ea.fc20.x86_64 Feb 14 07:39:11 Updated: krb5-libs-1.11.5-2.fc20.i686 Feb 14 07:39:12 Updated: gstreamer1-1.2.3-1.fc20.i686 Feb 14 07:39:13 Updated: sqlite-3.8.3-1.fc20.i686 In the list above I see "samba*" as update. Someone earlier wrote something about having received "samba*" as updates as well. I recall that I updated samba as well when this error happened. But this means nothing since I had this issue before, without any samba updates. Maybe packagekitd, which runs in the background and automagically receives updates and want you to reboot so the updates get installed. This would be a probably explaination "to set some flag" so the new reboot will install the updates ? This is all just blank theory not meant to be offensive. So to fill in the missing bits of Fedora history: 1. F20 released with systemd-208-9.fc20 (around 2013-12-13) 2. The patch Colin identifies as potentially ameliorating the issue, 0146-Revert-systemd-add-a-start-job-for-all-units-specifi.patch , was added to the Fedora package git repo on 2014-01-14 3. systemd-208-11.fc20, incorporating that patch, was submitted to updates-testing on 2014-01-15 - https://admin.fedoraproject.org/updates/FEDORA-2014-0902/systemd-208-11.fc20 4. It was quickly withdrawn from updates-testing, because Harald complained about two bugs unrelated to this one (https://bugzilla.redhat.com/show_bug.cgi?id=1053315 and https://bugzilla.redhat.com/show_bug.cgi?id=1023820 ) Basically, most people are still likely using the systemd F20 originally shipped with, which we know can trigger this problem. If you happened to get systemd-208-11 while it was briefly in updates-testing you may find this issue is ameliorated, though you may hit the other issues Harald complains about. If you think those other issues aren't likely to be a major problem for you, you can grab 208-11 from http://koji.fedoraproject.org/koji/buildinfo?buildID=491024 and see if it helps out with this bug. I'll ask the systemd devs to take a look at these three issues as a priority and see if they can't get a new build out to address them all. It's worth noting the 'severity' and 'priority' fields are not necessarily particularly important to Fedora bugs; there is no project-wide mechanism affected by these statuses at all, they really only exist for the convenience of each individual components' maintainers, and many maintainers don't bother using them at all for whatever reason. So the fact that they're unset or set to defaults for a bug doesn't necessarily mean anything about how important the maintainers consider the bug. You can only really trust those fields to mean anything if they've been set to a non-default value by one of the package maintainers. [aakcaagac@localhost ~]$ rpm -qa | grep -i "^systemd" systemd-208-9.fc20.i686 systemd-python-208-9.fc20.i686 systemd-libs-208-9.fc20.i686 [aakcaagac@localhost ~]$ Will try the -11 one and report back. Looks basically the same for me: $ rpm -qa | grep -i "^systemd" systemd-libs-208-9.fc20.x86_64 systemd-devel-208-9.fc20.x86_64 systemd-208-9.fc20.x86_64 systemd-libs-208-9.fc20.i686 Then there is the version of systemd that fedora 19 had, which had none of these problems at all. Any chance we could revert back to that one till there is a real fix? No. You really can't just go around playing component bingo like that. My laptop, running mageia 4 with all latest updates, has seen two cases of this nologin bug. Deleting /var/run/pam_nologin from the root account (btw, I was able to log into the root account from the text console, though it took quite a while) still did not let me log in as a regular user. I had to reboot in both cases. Having read all above posts, as well as the syslog messages, I am wondering why on earth it is necessary for systemd to reload the services to make the system look like a rebooting machine? It happened every few hours yesterday on my computer. I think this is the root of this bug, and definitely need to be fixed. It's not really necessary, no. We already know what's going on with this bug, and that -11 - while it doesn't technically fix it - should prevent it happening in almost all cases. The problem we have is that -11 causes another bug which we don't really want to ship. I've talked to Zbigniew today, and the plan is to produce a -12 build with both the patch that mostly deals with this bug, and something to deal with the other bug - #1053315 . Once we have a build with those two things in it, it should be good enough to go out. The whole area of systemd ATM is somewhat complicated by the fact that what F20 has is systemd-208 with a whole boatload of backported patches, but not *everything* that's in current systemd master, because there have been some rather drastic changes since 208 that we don't want to destabilize F20 with. It makes it a bit tricky to keep track of what's changing where and what's applicable to the F20 build and what's different. But we're dealing with it. (In reply to Ali Akcaagac from comment #83) > I decided to hibernate the notebook (I recall that I have > done this a few days ago as well) when I came back from hibernation the > "Shield" issue is gone and I found myself within gnome-shell (without the > need to enter anything). Congrats, this bug made Phoronix [1] thanks to this comment. Bypassing the screen shield is very serious. Probably this should be tracked in a separate bug against gdm? [1] http://www.phoronix.com/scan.php?page=news_item&px=MTYwMzg Sorry for the noise, that's Bug #1061148. Curse my "deal with exactly one tab at a time" browsing behavior! (In reply to Adam Williamson from comment #97) > The whole area of systemd ATM is somewhat complicated by the fact that what > F20 has is systemd-208 with a whole boatload of backported patches, but not > *everything* that's in current systemd master, because there have been some > rather drastic changes since 208 that we don't want to destabilize F20 with. I wonder what you mean by "destabilize" hear. IMO, a system that forces reboot without letting the user first finish what they are doing IS already far too unstable. Last night I was updating some packages, and the Plymouth screen just popped up again and I had no way to get back to anything that was going on in my X session.. I somehow doubt that any bugfix would destabilize it any further. (In reply to Jukka Lahtinen from comment #100) > I somehow doubt that any bugfix would destabilize it any further. I just got reminded about a novel written by Mary Shelley. The Monster created by Victor Frankenstein ran out of control once it was given life. :) Jukka: please read in context. At no point have I suggested not fixing this bug. (In reply to Adam Williamson from comment #102) Anyway, I updated to systemd-208-11 using the link mentioned in comment 91. I definitely DON'T WANT any uncontrolled system lockouts. I wasn't really under the impression anyone *did*. I do wonder sometimes why people find it necessary to say "I don't like bugs". I don't think anyone likes bugs. Well, except QA professionals, they pay our mortgages. :P Another update, another screen lock. It seems to me that some of the update scripts is creating (or trigger system to create) nologin file. As someone noticed above, samba packages were updated this time. My update list (from yum history):
Loaded plugins: langpacks, refresh-packagekit
Transaction ID : 106
Begin time : Sun Feb 16 09:18:39 2014
Begin rpmdb : 2512:923c0a8ffee72ecdd5f19b1937c22793146a7f9e
End time : 09:21:33 2014 (174 seconds)
End rpmdb : 2513:8675cebc3f0afaf1e3d4584970d94d82985c7ecb
User : Maciej Kycler <mkycler>
Return-Code : Success
Command Line : update
Transaction performed with:
Installed rpm-4.11.1-7.fc20.x86_64 installed
Installed yum-3.4.3-132.fc20.noarch @updates
Installed yum-metadata-parser-1.1.4-9.fc20.x86_64 installed
Packages Altered:
Updated abrt-2.1.12-1.fc20.x86_64 @updates
Update 2.1.12-2.fc20.x86_64 @updates
Updated abrt-addon-ccpp-2.1.12-1.fc20.x86_64 @updates
Update 2.1.12-2.fc20.x86_64 @updates
Updated abrt-addon-kerneloops-2.1.12-1.fc20.x86_64 @updates
Update 2.1.12-2.fc20.x86_64 @updates
Updated abrt-addon-pstoreoops-2.1.12-1.fc20.x86_64 @updates
Update 2.1.12-2.fc20.x86_64 @updates
Updated abrt-addon-python-2.1.12-1.fc20.x86_64 @updates
Update 2.1.12-2.fc20.x86_64 @updates
Updated abrt-addon-vmcore-2.1.12-1.fc20.x86_64 @updates
Update 2.1.12-2.fc20.x86_64 @updates
Updated abrt-addon-xorg-2.1.12-1.fc20.x86_64 @updates
Update 2.1.12-2.fc20.x86_64 @updates
Updated abrt-dbus-2.1.12-1.fc20.x86_64 @updates
Update 2.1.12-2.fc20.x86_64 @updates
Updated abrt-desktop-2.1.12-1.fc20.x86_64 @updates
Update 2.1.12-2.fc20.x86_64 @updates
Updated abrt-gui-2.1.12-1.fc20.x86_64 @updates
Update 2.1.12-2.fc20.x86_64 @updates
Updated abrt-gui-libs-2.1.12-1.fc20.x86_64 @updates
Update 2.1.12-2.fc20.x86_64 @updates
Updated abrt-libs-2.1.12-1.fc20.x86_64 @updates
Update 2.1.12-2.fc20.x86_64 @updates
Updated abrt-plugin-bodhi-2.1.12-1.fc20.x86_64 @updates
Update 2.1.12-2.fc20.x86_64 @updates
Updated abrt-python-2.1.12-1.fc20.x86_64 @updates
Update 2.1.12-2.fc20.x86_64 @updates
Updated abrt-retrace-client-2.1.12-1.fc20.x86_64 @updates
Update 2.1.12-2.fc20.x86_64 @updates
Updated argyllcms-1.6.2-1.fc20.x86_64 ?
Update 1.6.3-1.fc20.x86_64 @updates
Updated bodhi-client-0.9.5-3.fc20.noarch @?fedora
Update 0.9.8-1.fc20.noarch @updates
Updated gnuplot-4.6.3-5.fc20.x86_64 @fedora
Update 4.6.3-6.fc20.x86_64 @updates
Updated gnuplot-common-4.6.3-5.fc20.x86_64 @fedora
Update 4.6.3-6.fc20.x86_64 @updates
Updated gstreamer1-1.2.2-1.fc20.x86_64 @updates
Update 1.2.3-1.fc20.x86_64 @updates
Updated gstreamer1-plugins-bad-free-1.2.2-1.fc20.x86_64 @updates
Update 1.2.3-1.fc20.x86_64 @updates
Updated gstreamer1-plugins-base-1.2.2-1.fc20.x86_64 @updates
Update 1.2.3-1.fc20.x86_64 @updates
Updated gstreamer1-plugins-good-1.2.2-2.fc20.x86_64 @updates
Update 1.2.3-1.fc20.x86_64 @updates
Updated krb5-libs-1.11.3-39.fc20.i686 @updates
Updated krb5-libs-1.11.3-39.fc20.x86_64 @updates
Update 1.11.5-2.fc20.i686 @updates
Update 1.11.5-2.fc20.x86_64 @updates
Updated less-458-5.fc20.x86_64 ?
Update 458-6.fc20.x86_64 @updates
Updated libgadu-1.12.0-0.2.rc1.fc20.x86_64 @updates
Update 1.12.0-0.3.rc2.fc20.x86_64 @updates
Updated libgsf-1.14.28-1.fc20.x86_64 @?fedora
Update 1.14.29-1.fc20.x86_64 @updates
Updated libreport-2.1.12-1.fc20.x86_64 @updates
Update 2.1.12-2.fc20.x86_64 @updates
Updated libreport-fedora-2.1.12-1.fc20.x86_64 @updates
Update 2.1.12-2.fc20.x86_64 @updates
Updated libreport-filesystem-2.1.12-1.fc20.x86_64 @updates
Update 2.1.12-2.fc20.x86_64 @updates
Updated libreport-gtk-2.1.12-1.fc20.x86_64 @updates
Update 2.1.12-2.fc20.x86_64 @updates
Updated libreport-plugin-bugzilla-2.1.12-1.fc20.x86_64 @updates
Update 2.1.12-2.fc20.x86_64 @updates
Updated libreport-plugin-kerneloops-2.1.12-1.fc20.x86_64 @updates
Update 2.1.12-2.fc20.x86_64 @updates
Updated libreport-plugin-logger-2.1.12-1.fc20.x86_64 @updates
Update 2.1.12-2.fc20.x86_64 @updates
Updated libreport-plugin-reportuploader-2.1.12-1.fc20.x86_64 @updates
Update 2.1.12-2.fc20.x86_64 @updates
Updated libreport-plugin-ureport-2.1.12-1.fc20.x86_64 @updates
Update 2.1.12-2.fc20.x86_64 @updates
Updated libreport-python-2.1.12-1.fc20.x86_64 @updates
Update 2.1.12-2.fc20.x86_64 @updates
Updated libreport-web-2.1.12-1.fc20.x86_64 @updates
Update 2.1.12-2.fc20.x86_64 @updates
Updated libsmbclient-2:4.1.3-2.fc20.x86_64 @updates
Update 2:4.1.4-1.fc20.x86_64 @updates
Dep-Install libsrtp-1.4.4-10.20101004cvs.fc20.x86_64 @updates
Updated libwbclient-2:4.1.3-2.fc20.x86_64 @updates
Update 2:4.1.4-1.fc20.x86_64 @updates
Updated lohit-gujarati-fonts-2.5.3-2.fc20.noarch @?fedora
Update 2.92.2-1.fc20.noarch @updates
Updated mariadb-1:5.5.34-3.fc20.x86_64 @updates
Update 1:5.5.35-3.fc20.x86_64 @updates
Updated mariadb-libs-1:5.5.34-3.fc20.x86_64 @updates
Update 1:5.5.35-3.fc20.x86_64 @updates
Updated mariadb-server-1:5.5.34-3.fc20.x86_64 @updates
Update 1:5.5.35-3.fc20.x86_64 @updates
Updated numpy-1:1.8.0-3.fc20.x86_64 ?
Update 1:1.8.0-4.fc20.x86_64 @updates
Updated php-5.5.8-1.fc20.x86_64 @updates
Update 5.5.9-1.fc20.x86_64 @updates
Updated php-cli-5.5.8-1.fc20.x86_64 @updates
Update 5.5.9-1.fc20.x86_64 @updates
Updated php-common-5.5.8-1.fc20.x86_64 @updates
Update 5.5.9-1.fc20.x86_64 @updates
Updated php-mbstring-5.5.8-1.fc20.x86_64 @updates
Update 5.5.9-1.fc20.x86_64 @updates
Updated php-mysqlnd-5.5.8-1.fc20.x86_64 @updates
Update 5.5.9-1.fc20.x86_64 @updates
Updated php-pdo-5.5.8-1.fc20.x86_64 @updates
Update 5.5.9-1.fc20.x86_64 @updates
Updated php-pgsql-5.5.8-1.fc20.x86_64 @updates
Update 5.5.9-1.fc20.x86_64 @updates
Updated php-process-5.5.8-1.fc20.x86_64 @updates
Update 5.5.9-1.fc20.x86_64 @updates
Updated php-xml-5.5.8-1.fc20.x86_64 @updates
Update 5.5.9-1.fc20.x86_64 @updates
Updated python-2.7.5-9.fc20.x86_64 @?fedora
Update 2.7.5-10.fc20.x86_64 @updates
Updated python-libs-2.7.5-9.fc20.x86_64 @?fedora
Update 2.7.5-10.fc20.x86_64 @updates
Updated python3-3.3.2-8.fc20.x86_64 @?fedora
Update 3.3.2-9.fc20.x86_64 @updates
Updated python3-libs-3.3.2-8.fc20.x86_64 @?fedora
Update 3.3.2-9.fc20.x86_64 @updates
Updated samba-client-2:4.1.3-2.fc20.x86_64 @updates
Update 2:4.1.4-1.fc20.x86_64 @updates
Updated samba-common-2:4.1.3-2.fc20.x86_64 @updates
Update 2:4.1.4-1.fc20.x86_64 @updates
Updated samba-libs-2:4.1.3-2.fc20.x86_64 @updates
Update 2:4.1.4-1.fc20.x86_64 @updates
Updated samba-winbind-2:4.1.3-2.fc20.x86_64 @updates
Update 2:4.1.4-1.fc20.x86_64 @updates
Updated samba-winbind-clients-2:4.1.3-2.fc20.x86_64 @updates
Update 2:4.1.4-1.fc20.x86_64 @updates
Updated samba-winbind-modules-2:4.1.3-2.fc20.x86_64 @updates
Update 2:4.1.4-1.fc20.x86_64 @updates
Updated sed-4.2.2-5.fc20.x86_64 @?fedora
Update 4.2.2-6.fc20.x86_64 @updates
Updated slowhttptest-1.5-3.fc20.x86_64 @?fedora
Update 1.6-1.fc20.x86_64 @updates
Updated smc-fonts-common-6.0-2.fc20.noarch @?fedora
Update 6.0-7.fc20.noarch @updates
Updated smc-meera-fonts-6.0-2.fc20.noarch @?fedora
Update 6.0-7.fc20.noarch @updates
Updated sqlite-3.8.2-1.fc20.i686 @updates
Updated sqlite-3.8.2-1.fc20.x86_64 @updates
Update 3.8.3-1.fc20.i686 @updates
Update 3.8.3-1.fc20.x86_64 @updates
Updated tcpreplay-4.0.0-2.fc20.x86_64 @updates
Update 4.0.3-1.fc20.x86_64 @updates
Updated xemacs-filesystem-21.5.34-4.fc20.noarch @?fedora
Update 21.5.34-5.fc20.noarch @updates
history info
No it's not triggered by samba or any of the packages in either list posted here. I just set up a test system and "yum reinstall <packages>" to see whether the file pops up or not. We already know what is causing the bug. We do not need any more data. Thanks. systemd-208-13.fc20 has been submitted as an update for Fedora 20. https://admin.fedoraproject.org/updates/systemd-208-13.fc20 *** Bug 1057883 has been marked as a duplicate of this bug. *** systemd-208-14.fc20 has been submitted as an update for Fedora 20. https://admin.fedoraproject.org/updates/systemd-208-14.fc20 Package systemd-208-14.fc20: * should fix your issue, * was pushed to the Fedora 20 testing repository, * should be available at your local mirror within two days. Update it with: # su -c 'yum update --enablerepo=updates-testing systemd-208-14.fc20' as soon as you are able to. Please go to the following url: https://admin.fedoraproject.org/updates/FEDORA-2014-2690/systemd-208-14.fc20 then log in and leave karma (feedback). I have installed systemd-208-14.fc20 on three systems. On a KVM machine - it seems to be ok. On a FUJITSU ESPRIMO P910, standalone, no monitor - it seems to be ok. On a FUJITSU ESPRIMO P920, nis client, nfs client, two monitors - strange: it has rsyslogd running using 100% cpu time. On the last listed machine I found the following kind of messages in the output of "journalctl -e": Feb 19 01:50:01 myhost.example.com dbus[1007]: [system] Rejected send message, 3 matched rules; type="method_call", sender=":1.75" (uid=42 pid=2270 comm="/usr/libexec/mission-control-5 ") interface="org.freedesktop.Networ Feb 19 01:50:01 myhost.example.com dbus[1007]: [system] Rejected send message, 3 matched rules; type="method_call", sender=":1.72" (uid=42 pid=2192 comm="gnome-shell --mode=gdm ") interface="org.freedesktop.NetworkManager Feb 19 01:50:01 myhost.example.com dbus-daemon[1007]: dbus[1007]: [system] Rejected send message, 3 matched rules; type="method_call", sender=":1.72" (uid=42 pid=2192 comm="gnome-shell --mode=gdm ") interface="org.freedes Feb 19 01:50:01 myhost.example.com dbus-daemon[1007]: dbus[1007]: [system] Rejected send message, 3 matched rules; type="method_call", sender=":1.75" (uid=42 pid=2270 comm="/usr/libexec/mission-control-5 ") interface="org These kind of messages was generated at boot time and about every 10 minutes (minute 00, 10, 20, 30, 40, 50 - may be caused by a cron job?), each time about 4 of each messages are generated consecutively with the same time stamp but they are not in a fixed order. I will possibly reinstall this machine to eleminate some inconsistency that may have occured during the problems with /run/nologin. I am not sure how I will rate the new version of systemd. I tend to think that it will be better then the previous one because it contains fixes for some obstructive bugs. I cannot tell if the /run/nologin problem is solved, because the problem appears randomly over a longer time. Until now no /run/nologin was created. I have tried the example in comment #45 but nothing happend. >On a FUJITSU ESPRIMO P920, nis client, nfs client, two monitors - strange: it
>has rsyslogd running using 100% cpu time.
That could be leftover from a corrupted journal from the previous systemd. I know I found when it got in that state, I almost always had to remove the old systemd journal and start over from scratch to get syslog back to normal.
(In reply to Edgar Hoch from comment #112) > On a FUJITSU ESPRIMO P920, nis client, nfs client, two monitors - strange: > it has rsyslogd running using 100% cpu time. This is probably a libsystemd-journal bug, not related to the changes in systemd-208-14. Would it be possible for you to upload the whole /var/log/journal/<machine-id> directory? If it doesn't contain sensitive information, that is. If you prefer to not to do it publicly, you can send upload it somewhere and just send me the link privately, or I can provide some space for you to upload it to if that's easier. > On the last listed machine I found the following kind of messages in the > output of "journalctl -e": > > Feb 19 01:50:01 myhost.example.com dbus[1007]: [system] Rejected send > message, 3 matched rules; type="method_call", sender=":1.75" (uid=42 > pid=2270 comm="/usr/libexec/mission-control-5 ") > interface="org.freedesktop.Networ > Feb 19 01:50:01 myhost.example.com dbus[1007]: [system] Rejected send > message, 3 matched rules; type="method_call", sender=":1.72" (uid=42 > pid=2192 comm="gnome-shell --mode=gdm ") > interface="org.freedesktop.NetworkManager This is (some part of) gdm attempting to communicate with NetworkManager. You might want to file a bug against one of those components. I have seen this before, but never got to the bottom of it. I just installed 208-14, then rebooted to get into a clean state and ran yum update to install lots of other pending updates, and something went wacko during the yum update. My mouse got reset again, and I find all this nonsense in /var/log/messages in between a batch of yum log entries: ... Feb 18 20:40:12 zooty yum[2392]: Updated: ibus-anthy-1.5.5-1.fc20.x86_64 Feb 18 20:40:13 zooty yum[2392]: Updated: rpm-python-4.11.2-1.fc20.x86_64 Feb 18 20:40:15 zooty yum[2392]: Updated: wireshark-1.10.5-3.fc20.x86_64 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/LNXSYSTM:00/LNXPWRBN:00/input/input1 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/LNXSYSTM:00/LNXPWRBN:00/input/input1/event1 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/LNXSYSTM:00/device:00/PNP0A08:00/LNXVIDEO:00/input/input5 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/LNXSYSTM:00/device:00/PNP0A08:00/LNXVIDEO:00/input/input5/event2 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/LNXSYSTM:00/device:00/PNP0C0C:00/input/input0 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/LNXSYSTM:00/device:00/PNP0C0C:00/input/input0/event0 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:14.0/usb3 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:03.0/sound/card0/input8 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:03.0/sound/card0/input10 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:03.0/sound/card0/input9 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:03.0/sound/card0/input10/event5 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:14.0/usb3/3-0:1.0 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:14.0/usb3/3-8 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:14.0/usb3/3-5 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:14.0/usb3/3-13 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:14.0/usb3/3-6 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:03.0/sound/card0/input8/event7 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:14.0/usb3/3-5/3-5:1.0 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:14.0/usb3/3-13/3-13:1.0 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:14.0/usb3/3-14 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:14.0/usb3/3-13/3-13:1.0/0003:051D:0002.0001 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:14.0/usb4 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:03.0/sound/card0/input9/event6 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:14.0/usb3/3-6/3-6:1.0 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:14.0/usb3/3-14/3-14:1.0 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:14.0/usb4/4-0:1.0 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:1a.0/usb1 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:14.0/usb3/3-6/3-6.3 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:14.0/usb3/3-6/3-6.1 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:14.0/usb3/3-14/3-14:1.0/usbmisc/lp0 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:1b.0/sound/card1/input14 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:14.0/usb3/3-8/3-8:1.2 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:1b.0/sound/card1/input12 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:14.0/usb3/3-8/3-8:1.1 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:1b.0/sound/card1/input13 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:1b.0/sound/card1/input11 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:1b.0/sound/card1/input15 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:14.0/usb3/3-8/3-8:1.0 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:14.0/usb4/4-6 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:14.0/usb3/3-6/3-6.1/3-6.1:1.0 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:1b.0/sound/card1/input12/event12 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:1b.0/sound/card1/input15/event9 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:14.0/usb3/3-6/3-6.3/3-6.3:1.0 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:14.0/usb3/3-6/3-6.1/3-6.1.3 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:1a.0/usb1/1-0:1.0 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:14.0/usb4/4-6/4-6:1.0 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:1b.0/sound/card1/input11/event13 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:14.0/usb3/3-6/3-6.3/3-6.3:1.0/0003:047D:1020.0002 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:1b.0/sound/card1/input14/event10 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:1b.0/sound/card1/input16 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:1a.0/usb1/1-1 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:14.0/usb3/3-6/3-6.3/3-6.3:1.0/input/input6 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:1a.0/usb1/1-1/1-1:1.0 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:14.0/usb3/3-6/3-6.1/3-6.1.3/3-6.1.3:1.0 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:14.0/usb3/3-6/3-6.1/3-6.1.3/3-6.1.3:1.1 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:14.0/usb3/3-6/3-6.1/3-6.1.3/3-6.1.3:1.0/0003:046D:C318.0003 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:14.0/usb3/3-6/3-6.3/3-6.3:1.0/input/input6/event3 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:14.0/usb3/3-6/3-6.1/3-6.1.3/3-6.1.3:1.1/0003:046D:C318.0004 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:14.0/usb3/3-6/3-6.1/3-6.1.3/3-6.1.3:1.1/usbmisc/hiddev1 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:1b.0/sound/card1/input13/event11 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:14.0/usb3/3-6/3-6.1/3-6.1.3/3-6.1.3:1.1/input/input18 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:1d.0/usb2 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:1d.0/usb2/2-0:1.0 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:1d.0/usb2/2-1 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:14.0/usb3/3-6/3-6.1/3-6.1.3/3-6.1.3:1.0/input/input17 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1:1.0 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:14.0/usb3/3-6/3-6.3/3-6.3:1.0/input/input6/mouse0 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:14.0/usb3/3-6/3-6.1/3-6.1.3/3-6.1.3:1.1/input/input18/event15 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:1b.0/sound/card1/input16/event8 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/pci0000:00/0000:00:14.0/usb3/3-6/3-6.1/3-6.1.3/3-6.1.3:1.0/input/input17/event14 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/platform/eeepc-wmi/input/input7 Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/virtual/input/mice Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: treating change event as add on /sys/devices/platform/eeepc-wmi/input/input7/event4 Feb 18 20:40:16 zooty yum[2392]: Updated: evolution-data-server-3.10.4-1.fc20.x86_64 Feb 18 20:40:16 zooty yum[2392]: Updated: php-cli-5.5.9-1.fc20.x86_64 Feb 18 20:40:17 zooty yum[2392]: Updated: cmake-2.8.12.2-2.fc20.x86_64 Feb 18 20:40:17 zooty yum[2392]: Updated: gnutls-c++-3.1.20-3.fc20.x86_64 ... (In reply to Tom Horsley from comment #113) > >On a FUJITSU ESPRIMO P920, nis client, nfs client, two monitors - strange: it > >has rsyslogd running using 100% cpu time. > > That could be leftover from a corrupted journal from the previous systemd. I > know I found when it got in that state, I almost always had to remove the > old systemd journal and start over from scratch to get syslog back to normal. Thanks for the hint. Yes, it seems that the journal files was corruped. After stopping the services rsyslogd and journald I have renamed the directory in /var/log/journal/ to a new name and restarted these services. Then rsyslogd was using very few cpu time. Nethertheless, I have started a new Fedora 20 installation on this computer. (In reply to Zbigniew Jędrzejewski-Szmek from comment #114) > (In reply to Edgar Hoch from comment #112) > > On a FUJITSU ESPRIMO P920, nis client, nfs client, two monitors - strange: > > it has rsyslogd running using 100% cpu time. > This is probably a libsystemd-journal bug, not related to the changes in > systemd-208-14. > Would it be possible for you to upload the whole > /var/log/journal/<machine-id> > directory? If it doesn't contain sensitive information, that is. If you > prefer > to not to do it publicly, you can send upload it somewhere and just send me > the > link privately, or I can provide some space for you to upload it to if > that's easier. I will send you a private email. > > On the last listed machine I found the following kind of messages in the > > output of "journalctl -e": > > > > Feb 19 01:50:01 myhost.example.com dbus[1007]: [system] Rejected send > > message, 3 matched rules; type="method_call", sender=":1.75" (uid=42 > > pid=2270 comm="/usr/libexec/mission-control-5 ") > > interface="org.freedesktop.Networ > > Feb 19 01:50:01 myhost.example.com dbus[1007]: [system] Rejected send > > message, 3 matched rules; type="method_call", sender=":1.72" (uid=42 > > pid=2192 comm="gnome-shell --mode=gdm ") > > interface="org.freedesktop.NetworkManager > This is (some part of) gdm attempting to communicate with NetworkManager. You > might want to file a bug against one of those components. I have seen this > before, > but never got to the bottom of it. I have filed bug #1066781 . (In reply to Tom Horsley from comment #115) > Feb 18 20:40:15 zooty yum[2392]: Updated: wireshark-1.10.5-3.fc20.x86_64 > Feb 18 20:40:15 zooty upowerd: (upowerd:1882): UPower-Linux-WARNING **: > treating change event as add on My system at work got similar resets of everything by upowerd, and they happened after the wireshark update as well. I also notice this weird stuff later in the log: Feb 19 07:40:15 tomh iprdump[563]: Failed to find ipr ioa 8 for iprdump Feb 19 07:40:35 tomh iprdump[563]: Failed to find ipr ioa 0 for iprdump Feb 19 07:40:55 tomh iprdump[563]: Failed to find ipr ioa 1 for iprdump ... Since iprdump seems to have something to do with an IBM Power Linux RAID adapter and I don't have one of those, it isn't surprising it can't find things, but why is it even looking? Backing up to the start of all this. I ran "systemctl -f reboot" after the install of systemd from updates-testing, and my system hung forever with the last stuff on the console that had been there at boot and no indication of any kind of activity. I had to power cycle it to get it back. (In reply to Tom Horsley from comment #118) > Backing up to the start of all this. I ran "systemctl -f reboot" after the > install of systemd from updates-testing, and my system hung forever with the > last stuff on the console that had been there at boot and no indication of > any kind of activity. I had to power cycle it to get it back. And I've just run "systemctl -f reboot" on a clean system newly booted from scratch, it it hangs there as well, In fact the --force option seems to be exactly the opposite of what is documented :-). I don't see that the iprdump stuff is at all likely to be connected to the update. Check what package it comes from, whether that package contains an initscript, and if it's enabled. wireshark does "/usr/bin/udevadm trigger" in its %post. I'm pretty sure that what causes all your upower output. See if it does that if you run that command manually. Perhaps the wireshark package should redirect the output to /dev/null, but I doubt systemd is doing anything wrong. Does the -f reboot thing happen only with the new systemd, or does it happen with the old one too? I'm sure I used systemctl -f reboot on the old systemd and it really rebooted (got into the habit of using -f since I figured it might help me avoid hanging forever on stop jobs for user units as in bug 1023820). I did eventually discover the udevadm trigger in the wireshark scripts, but it is exceedingly annoying that it would have side effects like making the system think I removed my mouse and plugged it in again. Hard to believe that is the way it ought to be working. I know my mouse never got reset on yum updates in fedora 19 and there must have been a wireshark update in there somewhere :-). I have those upowerd messages in the journal after running a 'udevadm trigger' on a clean out-of-the-box F20 install with systemd 208-9. They are not regressions in 208-14. 'sudo systemctl -f reboot' works for me (triggers a more-or-less instant reboot) with both -9 and -14. *** Bug 1063706 has been marked as a duplicate of this bug. *** *** Bug 1057966 has been marked as a duplicate of this bug. *** *** Bug 1057811 has been marked as a duplicate of this bug. *** *** Bug 1048487 has been marked as a duplicate of this bug. *** *** Bug 961707 has been marked as a duplicate of this bug. *** (In reply to Adam Williamson from comment #122) > 'sudo systemctl -f reboot' works for me (triggers a more-or-less instant > reboot) with both -9 and -14. I just tried systemctl -f reboot at home (my work system was the one hanging), and at home it doesn't hang forever, but it did take 10 or 15 seconds longer than a reboot without the -f, definitely not near instantaneous. I've got rhgb turned off on the boot line on both systems - it isn't trying to wait for the graphical boot to finish is it? (In reply to Tom Horsley from comment #128) > I've got rhgb turned off on the boot line on both systems - > it isn't trying to wait for the graphical boot to finish is it? No, it shouldn't. 10-15 s could be correct: if there are some services getting shut down, journal being closed, etc. It probably depends on how much stuff you have running and the speed of the hardware. systemd-208-14.fc20 has been pushed to the Fedora 20 stable repository. If problems still persist, please make note of it in this bug report. so i've upgraded all my system. systemd is now at 208-14. the problem is if i start the computer and leave it alone for a while (maybe 30 seconds is enough, but for me it was like several minutes), when i came back and try to login, i have no mouse or keyboard control over gdm. /var/run/nologin is not there, so this may be another bug. if this is the case, let me know and i will create another ticket. if it's the same bug, please reopen this. the machine is thinkpad x61, but i've seen things like this on my desktops, before the systemd upgrade. there i still had the mouse, though. (In reply to cornel panceac from comment #131) > so i've upgraded all my system. systemd is now at 208-14. the problem is if > i start the computer and leave it alone for a while (maybe 30 seconds is > enough, but for me it was like several minutes), when i came back and try to > login, i have no mouse or keyboard control over gdm. /var/run/nologin is not > there, so this may be another bug. if this is the case, let me know and i > will create another ticket. if it's the same bug, please reopen this. the > machine is thinkpad x61, but i've seen things like this on my desktops, > before the systemd upgrade. there i still had the mouse, though. Is there anything interesting in journalctl output? This sounds like a different issue. Happens to Dell XPS 17 as well. (In reply to Zbigniew Jędrzejewski-Szmek from comment #132) > (In reply to cornel panceac from comment #131) > > so i've upgraded all my system. systemd is now at 208-14. the problem is if > > i start the computer and leave it alone for a while (maybe 30 seconds is > > enough, but for me it was like several minutes), when i came back and try to > > login, i have no mouse or keyboard control over gdm. /var/run/nologin is not > > there, so this may be another bug. if this is the case, let me know and i > > will create another ticket. if it's the same bug, please reopen this. the > > machine is thinkpad x61, but i've seen things like this on my desktops, > > before the systemd upgrade. there i still had the mouse, though. > > Is there anything interesting in journalctl output? This sounds like a > different issue. no, nothing interesting there. on second try though, the mouse came up, but the keyboard, apparently not. actually, if i select the only one user (or the "Not listed" entry), i can type just fine. but, on my multiuser systems, i can go up and down on the user list using keyboard before the gdm screensaver blacks the screen, but not after. alos, on this single user system, i can not select the only user listed by pressing enter, after the gdm screensaver. I've seen gdm's behaviour change after screen blanking kicks in, yeah. that's definitely nothing to do with this bug. Please do file it separately (well, check it hasn't been filed already first). *** Bug 1051876 has been marked as a duplicate of this bug. *** (In reply to Fedora Update System from comment #130) > systemd-208-14.fc20 has been pushed to the Fedora 20 stable repository. If > problems still persist, please make note of it in this bug report. Running systemd-204-18.fc19.x86_64 here on F19. Running "/usr/bin/systemd-tmpfiles --create" (without other arguments) results in a /run/nologin Reopening the bug. (In reply to Rolf Fokkens from comment #137) > (In reply to Fedora Update System from comment #130) > > systemd-208-14.fc20 has been pushed to the Fedora 20 stable repository. If > > problems still persist, please make note of it in this bug report. > > Running systemd-204-18.fc19.x86_64 here on F19. Running > "/usr/bin/systemd-tmpfiles --create" (without other arguments) results in a > /run/nologin > > Reopening the bug. This is expected behaviour. Running "touch /run/nologin" will also create that file ;) FYI, there is a patch to hide a few very specific boot-time only tmpfiles bits behind another switch that is typically not passed in, but unless the above command is being triggered by scripts, this doesn't seem to be any bug here specifically. After having suffered from this myself, I agree it now is expected behaviour for me too. But befora that I did not expect this at all. This message is a notice that Fedora 19 is now at end of life. Fedora has stopped maintaining and issuing updates for Fedora 19. It is Fedora's policy to close all bug reports from releases that are no longer maintained. Approximately 4 (four) weeks from now this bug will be closed as EOL if it remains open with a Fedora 'version' of '19'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora 19 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged change the 'version' to a later Fedora version prior this bug is closed as described in the policy above. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete. This was fixed with systemd-208-14.fc20 and re-opened erroneously, methinks. agreed. Time to reopen this bug again.
ll /var/run/nologin
-rw-r--r-- 1 root root 42 Dec 18 16:22 /var/run/nologin
cat /etc/redhat-release
Fedora release 26 (Twenty Six)
rpm -q systemd
systemd-233-7.fc26.x86_64
I believe the problem is that systemd-user-sessions service is completing *BEFORE* the systemd-tmpfiles-setup service. So /var/run/login is being created by the tmpfiles service, after user sessions service has exited. And thus, no-one except root can login to the system
See below: user sessions service exits at 16:22:22, but the tmpfiles service exits 3 seconds later, at 16:22:25 :
● systemd-user-sessions.service - Permit User Sessions
Loaded: loaded (/usr/lib/systemd/system/systemd-user-sessions.service; static; vendor preset: disabled)
Drop-In: /etc/systemd/system/systemd-user-sessions.service.d
└─systemd-tmpfiles-setup.conf
Active: active (exited) since Mon 2017-12-18 16:22:22 ACDT; 1h 24min ago
● systemd-tmpfiles-setup.service - Create Volatile Files and Directories
Loaded: loaded (/usr/lib/systemd/system/systemd-tmpfiles-setup.service; static; vendor preset: disabled)
Active: active (exited) since Mon 2017-12-18 16:22:25 ACDT; 1h 24min ago
Obviously... this is no good.
Someone needs to fix the ordering & dependencies on these services. systemd-user-sessions.service must run *AFTER* systemd-tmpfiles-setup.service
The drop-in you may notice there on my user-sessions service was an attempt to force user-sessions to run after the tmpfiles service. But it obviously doesn't work. --------------- cat /etc/systemd/system/systemd-user-sessions.service.d/systemd-tmpfiles-setup.conf [Unit] #Requires=systemd-tmpfiles-setup.service After=systemd-tmpfiles-setup.service --------------- If i have the "Requires" dependency, user-sessions service fails with a dependency error. If I remove the Requires, leaving just After, the drop-in makes no difference, tmpfiles-setup still completes before user-sessions. Brilliant... *not*. Sorry, mistake there in my last reply, I should've said tmpfiles-setup still completes *after* user-sessions. Anyway, some further invesigation shows systemd is trying to execute tmpfiles-setup before user-sessions, but, tmpfiles-setup is failing the first time it runs, due to some utterly nonsensical errors. This causes user-sessions to fail due to its dependency on tmpfiles-setup. And then, later-on, user-session runs again, and succeeds, bit user-session is *not* re-run. journalctl --boot=0 | grep -e user-session -e tmpfile | grep -ve audit -ve dbus Dec 18 18:05:17 Il-Duce systemd-tmpfiles[763]: "/var/cache" already exists and is not a directory. Dec 18 18:05:17 Il-Duce systemd-tmpfiles[763]: Failed to create directory or subvolume "/var/cache/man": No such file or directory Dec 18 18:05:17 Il-Duce systemd[1]: systemd-tmpfiles-setup.service: Main process exited, code=exited, status=1/FAILURE Dec 18 18:05:17 Il-Duce systemd[1]: systemd-user-sessions.service: Job systemd-user-sessions.service/start failed with result 'dependency'. Dec 18 18:05:17 Il-Duce systemd[1]: systemd-tmpfiles-setup.service: Unit entered failed state. Dec 18 18:05:17 Il-Duce systemd[1]: systemd-tmpfiles-setup.service: Failed with result 'exit-code'. Dec 18 18:05:21 Il-Duce systemd-tmpfiles[2863]: "/var/cache" already exists and is not a directory. So anyway, I have several questions about this: 1) How does systemd-tmpfiles(-setup) get the ridiculous idea /var/cache exists but is not a directory? Is this running so early there is initramfs or something mounted on /var/cache, so it's a mountpoint? Is that the problem? 2) Whether /var/cache is a directory, or a mountpoint, there should be no problem creating /var/cache/man. It certainly should not fail with "No such file or directory", because we've confirmed /var/cache does exist, although systemd claims it is not a directory. If it's a mountpoint, we should be able to create /var/cache/man. 3) When tmpfiles runs the second time, and succeeds, why does this not cause user-sessions to re-run, and succeed. If this happened, then /var/run/nologin would be removed, and *I would not be here wasting everyone's time*. Finally, I note, that when bootup has finished, /var/cache does exits, it is not a mountpoint, and /var/cache/man exists too. So whatever is going on here, it is a massive ballache. Sorry... another mistake in last post... I should've said: "And then, later-on, tmpfiles-setup runs again, and succeeds, but user-session is *not* re-run." Bah. Just realised what's happened. I moved /var/cache to another partition, and made /var/cache a symlink. So we have: Dec 18 18:05:17 Il-Duce systemd[1]: Starting Create Volatile Files and Directories... Dec 18 18:05:17 Il-Duce systemd-tmpfiles[763]: "/var/cache" already exists and is not a directory. Dec 18 18:05:17 Il-Duce systemd-tmpfiles[763]: Failed to create directory or subvolume "/var/cache/man": No such file or directory Dec 18 18:05:17 Il-Duce systemd[1]: systemd-tmpfiles-setup.service: Main process exited, code=exited, status=1/FAILURE Dec 18 18:05:17 Il-Duce systemd[1]: Failed to start Create Volatile Files and Directories. ... Dec 18 18:05:18 Il-Duce systemd[1]: Mounted /data-ssd. What sort of garbage is this... systemd is so retarded it can't mount filesystems before it starts to create its tmpfiles & perform its other tasks? Solved by making /var/cache a bind mount. Weak. I've got enough bind mounts already, I shouldn't need to be making more of them just to resolve issues like this. Why is there no way to tell systemd to prioritise some mounts before other units, Obviously, systemd knows it has to mount / early, so why is there no way to say also mount /blah, before you get carried away doing other stuff. It sounds like i *could* possibly use "x-systemd.requires" on the root fs entry in fstab, so my partition with /var/cache on it is mounted before /. But that would be a recipe for disaster - if anything happened to my partition with /var/cache, it'd turn a minor problem into a huge ballache with no mounted /. Woops, scratch that last idea, of course there's no way i can have my partition with /var/cache mounted before /, of course / has to be mounted first. So, systemd forces me to use a bind mound, when it should just provide a way to schedule other mounts immediately after mount /. Again... Weak, systemd, weak. "Why is there no way to tell systemd to prioritise some mounts before other units" Sure there is. systemd mounts are units, just like services are. It converts fstab entries to units automatically, but you can also create mount files directly as units, and express dependencies in them. See e.g. /usr/lib/systemd/system/tmp.mount . That's a mount that uses dependencies: Conflicts=umount.target Before=local-fs.target umount.target After=swap.target |