RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 996547 - Systemd assertion failure in cg_is_empty_recursive
Summary: Systemd assertion failure in cg_is_empty_recursive
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: systemd
Version: 7.0
Hardware: x86_64
OS: Unspecified
unspecified
unspecified
Target Milestone: rc
: ---
Assignee: systemd-maint
QA Contact: Jan Ščotka
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2013-08-13 12:12 UTC by Nikolai Kondrashov
Modified: 2014-01-31 13:48 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2014-01-31 13:48:52 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)
core.28556.gz (356.14 KB, application/x-gzip)
2013-08-13 12:12 UTC, Nikolai Kondrashov 		no flags Details
core.15923.gz (353.42 KB, application/x-gzip)
2013-08-13 18:16 UTC, Nikolai Kondrashov 		no flags Details
core.3272.gz (341.11 KB, application/x-gzip)
2013-08-28 15:45 UTC, Nikolai Kondrashov 		no flags Details
View All

Description Nikolai Kondrashov 2013-08-13 12:12:45 UTC 
Created attachment 786130 [details]
core.28556.gz

Description of problem:
Systemd hits assertion failure (compressed coredump attached) after repeatedly restarting sssd, with the following in /var/log/messages:

Aug 13 14:52:38 client-rhel7 systemd: Stopping System Security Services Daemon...
Aug 13 14:52:38 client-rhel7 sssd[pam]: Shutting down
Aug 13 14:52:38 client-rhel7 sssd[be[LDAP]]: Shutting down
Aug 13 14:52:38 client-rhel7 sssd[nss]: Shutting down
Aug 13 14:52:38 client-rhel7 systemd: Stopped System Security Services Daemon.
Aug 13 14:52:38 client-rhel7 systemd: Assertion 'path' failed at src/shared/cgroup-util.c:866, function cg_is_empty_recursive(). Aborting.
Aug 13 14:52:38 client-rhel7 systemd: Caught <ABRT>, dumped core as pid 28556.
Aug 13 14:52:38 client-rhel7 systemd: Freezing execution.

Shortly before that systemd refused to restart sssd with the following in
"journalctl -xn" output:

-- Subject: Unit sssd.service has begun with start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit sssd.service has begun starting up.
Aug 13 14:44:55 client-rhel7.sss-test.test systemd[1]: sssd.service start request repeated too quickly, refusing to start.
Aug 13 14:44:55 client-rhel7.sss-test.test systemd[1]: Failed to start System Security Services Daemon.
-- Subject: Unit sssd.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- Documentation: http://www.freedesktop.org/wiki/Software/systemd/catalog/be02cf6855d2428ba40df7e9d022f03d
--
-- Unit sssd.service has failed.
--
-- The result is failed.

Version-Release number of selected component (if applicable):
systemd-libs-206-3.el7.x86_64
systemd-206-3.el7.x86_64
systemd-journal-gateway-206-3.el7.x86_64
systemd-sysv-206-3.el7.x86_64
systemd-debuginfo-206-3.el7.x86_64

How reproducible:
Encountered once so far

Steps to Reproduce:
Restart sssd repeatedly with about one second between restarts.

Backtrace:
#0  0x00007fb093d59ffb in raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/pt-raise.c:37
#1  0x00007fb0958a195e in crash (sig=6) at src/core/main.c:144
#2  <signal handler called>
#3  0x00007fb0939bf999 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#4  0x00007fb0939c10a8 in __GI_abort () at abort.c:90
#5  0x00007fb0958fa533 in log_assert (text=<optimized out>, file=0x7fb095953463 "src/shared/cgroup-util.c", line=866, func=0x7fb095953620 <__PRETTY_FUNCTION__.7898> "cg_is_empty_recursive", format=format@entry=0x7fb095954a10 "Assertion '%s' failed at %s:%u, function %s(). Aborting.") at src/shared/log.c:695
#6  0x00007fb0958fade0 in log_assert_failed (text=<optimized out>, file=<optimized out>, line=<optimized out>, func=<optimized out>) at src/shared/log.c:700
#7  0x00007fb0958f13f3 in cg_is_empty_recursive (controller=controller@entry=0x7fb09594bd07 "name=systemd", path=0x0, ignore_self=ignore_self@entry=true) at src/shared/cgroup-util.c:866
#8  0x00007fb0958e2890 in manager_notify_cgroup_empty (m=m@entry=0x7fb09766a360, cgroup=<optimized out>) at src/core/cgroup.c:736
#9  0x00007fb0958d338d in private_bus_message_filter (connection=0x7fb097770e40, message=0x7fb097978de0, data=0x7fb09766a360) at src/core/dbus.c:491
#10 0x00007fb0943949e6 in dbus_connection_dispatch (connection=connection@entry=0x7fb097770e40) at dbus-connection.c:4631
#11 0x00007fb0958d3dda in bus_dispatch (m=m@entry=0x7fb09766a360) at src/core/dbus.c:525
#12 0x00007fb0958a769f in manager_loop (m=0x7fb09766a360) at src/core/manager.c:1816
#13 0x00007fb09589efb6 in main (argc=5, argv=0x7fff6416bc88) at src/core/main.c:1705
Comment 2 Nikolai Kondrashov 2013-08-13 18:16:56 UTC 
Created attachment 786233 [details]
core.15923.gz

Another crash, this time SIGSEGV, but, it seems, during the same assertion failure.

Backtrace:
#0  0x00007f4c3d4deffb in raise (sig=sig@entry=11) at ../nptl/sysdeps/unix/sysv/linux/pt-raise.c:37
#1  0x00007f4c3f02695e in crash (sig=11) at src/core/main.c:144
#2  <signal handler called>
#3  0x00007f4c3f07572b in isempty (p=0x3a30303a30303030 <Address 0x3a30303a30303030 out of bounds>) at src/shared/util.h:106
#4  join_path (controller=0x7f4c3f0a2e22 "systemd", path=0x3a30303a30303030 <Address 0x3a30303a30303030 out of bounds>, suffix=0x7f4c3f0d84ef "cgroup.procs", fs=0x7fffecb49eb8) at src/shared/cgroup-util.c:454
#5  0x00007f4c3f075ea8 in cg_enumerate_processes (controller=controller@entry=0x7f4c3f0d0d07 "name=systemd", path=path@entry=0x3a30303a30303030 <Address 0x3a30303a30303030 out of bounds>, _f=_f@entry=0x7fffecb49ee8) at src/shared/cgroup-util.c:50
#6  0x00007f4c3f07619a in cg_is_empty (controller=controller@entry=0x7f4c3f0d0d07 "name=systemd", path=path@entry=0x3a30303a30303030 <Address 0x3a30303a30303030 out of bounds>, ignore_self=ignore_self@entry=true) at src/shared/cgroup-util.c:827
#7  0x00007f4c3f07630f in cg_is_empty_recursive (controller=controller@entry=0x7f4c3f0d0d07 "name=systemd", path=0x3a30303a30303030 <Address 0x3a30303a30303030 out of bounds>, ignore_self=ignore_self@entry=true) at src/shared/cgroup-util.c:868
#8  0x00007f4c3f067890 in manager_notify_cgroup_empty (m=m@entry=0x7f4c3f4f7360, cgroup=<optimized out>) at src/core/cgroup.c:736
#9  0x00007f4c3f05838d in private_bus_message_filter (connection=0x7f4c3f5ba450, message=0x7f4c3f5ba560, data=0x7f4c3f4f7360) at src/core/dbus.c:491
#10 0x00007f4c3db199e6 in dbus_connection_dispatch (connection=connection@entry=0x7f4c3f5ba450) at dbus-connection.c:4631
#11 0x00007f4c3f058dda in bus_dispatch (m=m@entry=0x7f4c3f4f7360) at src/core/dbus.c:525
#12 0x00007f4c3f02c69f in manager_loop (m=0x7f4c3f4f7360) at src/core/manager.c:1816
#13 0x00007f4c3f023fb6 in main (argc=5, argv=0x7fffecb4a9d8) at src/core/main.c:1705
Comment 3 Kay Sievers 2013-08-14 10:46:44 UTC 
Hmm, just to check, was there an earlier version of systemd on the
machine before?

If yes, was the machine rebooted after the update to the systemd.rpm
version 206?
Comment 4 Nikolai Kondrashov 2013-08-14 10:53:04 UTC 
I'm not sure about the first crash (will try to check), but the second was after a reboot. Note though, that I haven't seen this again so far.
Comment 5 Nikolai Kondrashov 2013-08-28 15:45:50 UTC 
Created attachment 791441 [details]
core.3272.gz

Another crash after a fresh reboot. This time with the following package versions:

systemd-libs-206-6.el7.x86_64
systemd-journal-gateway-206-6.el7.x86_64
systemd-debuginfo-206-6.el7.x86_64
systemd-206-6.el7.x86_64  
systemd-sysv-206-6.el7.x86_64

Repeated sssd restarts were involved as well.

Backtrace:
#0  0x00007ff25d0e4ffb in raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/pt-raise.c:37
#1  0x00007ff25ec2c95e in crash (sig=6) at src/core/main.c:144
#2  <signal handler called>
#3  0x00007ff25cd4a999 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#4  0x00007ff25cd4c0a8 in __GI_abort () at abort.c:90
#5  0x00007ff25ec85563 in log_assert (text=<optimized out>, file=0x7ff25ecde503 "src/shared/cgroup-util.c", line=866, func=0x7ff25ecde6c0 <__PRETTY_FUNCTION__.7913> "cg_is_empty_recursive", format=format@entry=0x7ff25ecdfab0 "Assertion '%s' failed at %s:%u, function %s(). Aborting.") at src/shared/log.c:699
#6  0x00007ff25ec85e10 in log_assert_failed (text=<optimized out>, file=<optimized out>, line=<optimized out>, func=<optimized out>) at src/shared/log.c:704
#7  0x00007ff25ec7c3f3 in cg_is_empty_recursive (controller=controller@entry=0x7ff25ecd6da7 "name=systemd", path=0x0, ignore_self=ignore_self@entry=true) at src/shared/cgroup-util.c:866
#8  0x00007ff25ec6d890 in manager_notify_cgroup_empty (m=m@entry=0x7ff25f6e1ad0, cgroup=<optimized out>) at src/core/cgroup.c:741
#9  0x00007ff25ec5e37d in private_bus_message_filter (connection=0x7ff25f7a15d0, message=0x7ff25f7a16e0, data=0x7ff25f6e1ad0) at src/core/dbus.c:491
#10 0x00007ff25d71f9e6 in dbus_connection_dispatch (connection=connection@entry=0x7ff25f7a15d0) at dbus-connection.c:4631
#11 0x00007ff25ec5edca in bus_dispatch (m=m@entry=0x7ff25f6e1ad0) at src/core/dbus.c:525
#12 0x00007ff25ec3269f in manager_loop (m=0x7ff25f6e1ad0) at src/core/manager.c:1816
#13 0x00007ff25ec29fb6 in main (argc=5, argv=0x7fff83ebf748) at src/core/main.c:1705
Comment 6 Harald Hoyer 2013-08-28 15:58:05 UTC 
should be fixed with systemd-206-7 , which was built today
Note You need to log in before you can comment on or make changes to this bug.