Bug 1498318

Summary: du/df hang indefinitely (RHEL Atomic Host 7.4)
Product: Red Hat Enterprise Linux 7 Reporter: Jeremy Eder <jeder>
Component: systemdAssignee: Michal Sekletar <msekleta>
Status: CLOSED ERRATA QA Contact: Frantisek Sumsal <fsumsal>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 7.4CC: ekuric, fkrska, fsumsal, gangelop, jbyrd, jmencak, joedward, jsynacek, mifiedle, msekleta, santony, sbroz, systemd-maint-list
Target Milestone: rcKeywords: Reopened, ZStream
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard: aos-scalability-37
Fixed In Version: systemd-219-46.el7 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1562092 (view as bug list) Environment:
Last Closed: 2018-04-10 11:22:48 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1562092    
Attachments:
Description Flags
journalctl -b none

Comment 2 Lukáš Nykrýn 2017-10-04 07:56:42 UTC
To be honest I don't know if this is the cases, but at least it looks related. The backport in this case may not be that easy, so let's do some debugging first.

If you have a reproducer, can you please switch systemd log level to debug
systemd-analyze set-log-level debug
reproduce the issue and send us the output of journal
journalctl -b

Comment 3 Jiří Mencák 2017-10-04 10:02:38 UTC
Created attachment 1334156 [details]
journalctl -b

root@172: /tmp # systemctl -pLogLevel show
Failed to get properties: Connection timed out
root@172: /tmp # systemd-analyze set-log-level debug
Failed to issue method call: Activation of org.freedesktop.systemd1 timed out

"journalctl -b" attached, but likely with LogLevel info

Comment 4 Michal Sekletar 2017-10-04 14:46:38 UTC
From what I've seen on the test system you've hit this bug,

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=856035

Reproducer is very straight forward,

systemctl mask proc-sys-fs-binfmt_misc.automount
systemctl daemon-reload
ls -l /proc/sys/fs/binfmt_misc

Blocking that you see is due to unhandled autofs events. Obviously, systemd can't handle them because it crashed.

This was fixed upstream a while ago,

https://github.com/systemd/systemd/commit/e350ca3f1ecb6672b74cd25d09ef23c7b309aa5a

Here is backport for RHEL-7 (it wasn't as straightforward as I'd like so it will require proper review).

https://github.com/msekletar/systemd-rhel/commit/c2bf138a9507e9faca776500a505c08e4fa156c4

And here is scratch build with backport,

https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=14174304

Comment 5 Jiří Mencák 2017-10-04 17:58:51 UTC
Thanks for getting this out so fast.  Upgraded systemd-*, systemd-sysv-* and systemd-libs-* on the affected system.  While this helped with the debian reproducer, df still hanged.  Resorted to "systemctl mask proc-sys-fs-binfmt_misc.automount", no more df hangs at the moment.

Comment 10 Lukáš Nykrýn 2017-10-30 09:30:42 UTC
fix merged to staging branch -> https://github.com/lnykryn/systemd-rhel/pull/154 -> post

Comment 19 errata-xmlrpc 2018-04-10 11:22:48 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:0711