Bug 1221370 - [abrt] sssd-ipa: semanage_disconnect(): selinux_child killed by SIGABRT
Summary: [abrt] sssd-ipa: semanage_disconnect(): selinux_child killed by SIGABRT
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: sssd
Version: 21
Hardware: x86_64
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Jakub Hrozek
QA Contact: Fedora Extras Quality Assurance
URL: https://retrace.fedoraproject.org/faf...
Whiteboard: abrt_hash:f1f17dfa11e8d2ebc154aafc8e8...
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-05-13 21:21 UTC by Samuel Sieb
Modified: 2015-06-24 20:35 UTC (History)
8 users (show)

Fixed In Version: sssd-1.12.5-2.fc21
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-06-23 09:13:54 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)
File: backtrace (9.22 KB, text/plain)
2015-05-13 21:21 UTC, Samuel Sieb
no flags Details
File: cgroup (188 bytes, text/plain)
2015-05-13 21:21 UTC, Samuel Sieb
no flags Details
File: core_backtrace (1.58 KB, text/plain)
2015-05-13 21:21 UTC, Samuel Sieb
no flags Details
File: dso_list (2.34 KB, text/plain)
2015-05-13 21:21 UTC, Samuel Sieb
no flags Details
File: environ (333 bytes, text/plain)
2015-05-13 21:21 UTC, Samuel Sieb
no flags Details
File: limits (1.29 KB, text/plain)
2015-05-13 21:21 UTC, Samuel Sieb
no flags Details
File: maps (11.73 KB, text/plain)
2015-05-13 21:21 UTC, Samuel Sieb
no flags Details
File: proc_pid_status (906 bytes, text/plain)
2015-05-13 21:21 UTC, Samuel Sieb
no flags Details
selinux_child.log (56.36 KB, text/plain)
2015-06-01 16:58 UTC, Samuel Sieb
no flags Details

Description Samuel Sieb 2015-05-13 21:21:32 UTC
Version-Release number of selected component:
sssd-ipa-1.12.4-3.fc21

Additional info:
reporter:       libreport-2.3.0
backtrace_rating: 4
cmdline:        /usr/libexec/sssd/selinux_child --debug-microseconds=0 --debug-timestamps=1 --debug-fd=26 --debug-level=0x0010
crash_function: semanage_disconnect
executable:     /usr/libexec/sssd/selinux_child
kernel:         3.19.7-200.fc21.x86_64
open_fds:       
runlevel:       N 5
type:           CCpp
uid:            0
var_log_messages: [System Logs]:\n-- Logs begin at Wed 2015-05-13 13:57:37 PDT, end at Wed 2015-05-13 14:05:57 PDT. --

Truncated backtrace:
Thread no. 1 (5 frames)
 #4 semanage_disconnect at handle.c:398
 #5 sss_semanage_close at src/util/sss_semanage.c:74
 #6 sss_semanage_init at src/util/sss_semanage.c:114
 #7 get_seuser at src/util/sss_semanage.c:382
 #8 seuser_needs_update at src/providers/ipa/selinux_child.c:175

Comment 1 Samuel Sieb 2015-05-13 21:21:35 UTC
Created attachment 1025189 [details]
File: backtrace

Comment 2 Samuel Sieb 2015-05-13 21:21:36 UTC
Created attachment 1025190 [details]
File: cgroup

Comment 3 Samuel Sieb 2015-05-13 21:21:37 UTC
Created attachment 1025191 [details]
File: core_backtrace

Comment 4 Samuel Sieb 2015-05-13 21:21:38 UTC
Created attachment 1025192 [details]
File: dso_list

Comment 5 Samuel Sieb 2015-05-13 21:21:39 UTC
Created attachment 1025193 [details]
File: environ

Comment 6 Samuel Sieb 2015-05-13 21:21:40 UTC
Created attachment 1025194 [details]
File: limits

Comment 7 Samuel Sieb 2015-05-13 21:21:41 UTC
Created attachment 1025195 [details]
File: maps

Comment 8 Samuel Sieb 2015-05-13 21:21:42 UTC
Created attachment 1025196 [details]
File: proc_pid_status

Comment 9 Lukas Slebodnik 2015-05-14 07:05:46 UTC
It is very likely upstream bug https://fedorahosted.org/sssd/ticket/2649

Do you have a reliable reproducer?
Did you see any AVC?

Comment 10 Samuel Sieb 2015-05-14 18:08:38 UTC
It happens continuously:
type=ANOM_ABEND msg=audit(1431626671.924:422): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=system_u:system_r:sssd_t:s0 pid=2152 comm="selinux_child" exe="/usr/libexec/sssd/selinux_child" sig=6
There are always PAM messages around it.

The only AVC messages are:
type=USER_AVC msg=audit(1431626461.362:418): pid=601 uid=81 auid=4294967295 ses=4294967295 subj=system_u:system_r:system_dbusd_t:s0-s0:c0.c1023 msg='avc:  denied  { send_msg } for msgtype=signal interface=org.freedesktop.NetworkManager member=CheckPermissions dest=org.freedesktop.DBus spid=590 tpid=683 scontext=system_u:system_r:NetworkManager_t:s0 tcontext=system_u:system_r:cupsd_t:s0-s0:c0.c1023 tclass=dbus  exe="/usr/bin/dbus-daemon" sauid=81 hostname=? addr=? terminal=?'

Comment 11 Lukas Slebodnik 2015-05-14 20:00:35 UTC
I would like to see sssd log files. It might help to find reasonable reproducer.
There can be also another problem which can be hidden by fix of #2649

Please:
* put "debug_level = 9" into domain section
* reproduce crash
* provide sssd log files from directory /var/log/sssd/

Comment 12 Jakub Hrozek 2015-05-15 07:04:20 UTC
Some AVC denials can be marked as "noaudit" so they wouldn't show up in the SSSD logs. It would also make sense to set SELinux to permissive mode for the test.

An strace of the selinux_child process would also be useful, to see what the selinux_child is doing:
https://fedorahosted.org/sssd/wiki/DevelTips#UsingstracetotracktheSSSDprocesses

Comment 13 Jakub Hrozek 2015-05-15 07:09:09 UTC
(In reply to Jakub Hrozek from comment #12)
> Some AVC denials can be marked as "noaudit" so they wouldn't show up in the
> SSSD logs.
  ~~~~~~~~~

Audit logs, sorry

Comment 14 Samuel Sieb 2015-05-18 23:27:57 UTC
I put "debug_level = 9" in, but there wasn't anything interesting in the logs.  There isn't even any indication that there was a crash.

I will try to get the strace output.  This is a teacher's laptop and I'm not on site any more, so I need to do it remotely without disturbing her work.

Comment 15 Lukas Slebodnik 2015-05-29 07:36:02 UTC
(In reply to Samuel Sieb from comment #14)
> I put "debug_level = 9" in, but there wasn't anything interesting in the
> logs.  There isn't even any indication that there was a crash.
> 
Could you attach selinux_child.log. We might find something useful there.

> I will try to get the strace output.  This is a teacher's laptop and I'm not
> on site any more, so I need to do it remotely without disturbing her work.

Have you had a time to obtain strace output?

BTW I can provide you a scratch build with fix for testing purposes.
But it would be very handy to have a reproducer.

Comment 16 Samuel Sieb 2015-06-01 16:58:02 UTC
Created attachment 1033424 [details]
selinux_child.log

Comment 17 Samuel Sieb 2015-06-01 17:08:41 UTC
This is a little tricky.  In order to run strace, selinux needs to be permissive.  However, then selinux_child doesn't crash.  Suggestions?

Comment 18 Lukas Slebodnik 2015-06-01 17:16:27 UTC
Some conditions had to change.

Do I understand it correctly that you cannot reproduce crash with strace
or you cannot reproduce crash at all?

Comment 19 Samuel Sieb 2015-06-01 19:51:07 UTC
It's still crashing.  But if I set selinux to permissive in order to be able to use strace, it doesn't crash.  I realized I could just run strace manually as root instead of trying to automate it.  Here's the end of the trace:

access("/sbin/load_policy", X_OK)       = -1 EACCES (Permission denied)
access("/sbin/setfiles", X_OK)          = -1 EACCES (Permission denied)
access("/sbin/sefcontext_compile", X_OK) = 0
open("/etc/selinux/semanage.conf", O_RDONLY) = 0
ioctl(0, TCGETS, 0x7ffbffffaef0)        = -1 ENOTTY (Inappropriate ioctl for device)
fstat(0, {st_mode=S_IFREG|0644, st_size=2321, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7efbf7ffc000
read(0, "# Authors: Jason Tang <jtang>\n#\n# Copyright (C) 2004-2005 Tresys Technology, LLC\n#\n#  This library is free software; you can redistribute it and/or\n#  modify it under the terms of the GNU L"..., 8192) = 2321
read(0, "", 4096)                       = 0
read(0, "", 8192)                       = 0
ioctl(0, TCGETS, 0x7ffbffffaef0)        = -1 ENOTTY (Inappropriate ioctl for device)
close(0)                                = 0
munmap(0x7efbf7ffc000, 4096)            = 0
access("/etc/selinux/targeted/modules/active", R_OK|X_OK) = 0
access("/etc/selinux/targeted/modules/semanage.read.LOCK", R_OK) = -1 EACCES (Permission denied)
access("/etc/selinux/targeted/modules/semanage.read.LOCK", F_OK) = 0
access("/etc/selinux/targeted/modules/active", R_OK|X_OK) = 0
access("/etc/selinux/targeted/modules/semanage.read.LOCK", R_OK) = -1 EACCES (Permission denied)
access("/etc/selinux/targeted/modules/semanage.read.LOCK", F_OK) = 0
stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=2901, ...}) = 0
stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=2901, ...}) = 0
write(26, "(Mon Jun  1 12:44:43 2015) [[sssd[selinux_child[2885]]]] [sss_semanage_init] (0x0020): Cannot read SELinux policy store\n", 120) = 120
write(2, "selinux_child: handle.c:399: semanage_disconnect: Assertion `sh != ((void *)0) && sh->funcs != ((void *)0) && sh->funcs->disconnect != ((void *)0)' failed.\n", 156) = 156
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7efbf7ffc000
rt_sigprocmask(SIG_UNBLOCK, [ABRT], NULL, 8) = 0
tgkill(2885, 2885, SIGABRT)             = 0
--- SIGABRT {si_signo=SIGABRT, si_code=SI_TKILL, si_pid=2885, si_uid=0} ---
+++ killed by SIGABRT (core dumped) +++

Comment 20 Lukas Slebodnik 2015-06-02 15:53:28 UTC
It needn't crash in permissive mode because SELinux would not prevent access to file  /etc/selinux/targeted/modules/semanage.read.LOCK
 
access("/etc/selinux/targeted/modules/active", R_OK|X_OK) = 0
access("/etc/selinux/targeted/modules/semanage.read.LOCK", R_OK) = -1 EACCES (Permission denied)
access("/etc/selinux/targeted/modules/semanage.read.LOCK", F_OK) = 0
access("/etc/selinux/targeted/modules/active", R_OK|X_OK) = 0
access("/etc/selinux/targeted/modules/semanage.read.LOCK", R_OK) = -1 EACCES (Permission denied)

Could you provide output of following commands (after reproducing issue)?

ls -lZ /etc/selinux/targeted/modules/
ausearch -m avc,user_avc -ts today

Comment 21 Jakub Hrozek 2015-06-02 18:17:12 UTC
It might also be interesting to see what the actual permissions are. IIRC libselinux requires a particular umask to be set (or rather relies on particular permissions and stricter umask can screw that up..).

Comment 22 Lukas Slebodnik 2015-06-11 16:38:33 UTC
Samuel,
The crash should be fixed in pre-release of sssd-1.12.5
You can test packages from COPR repo[1] or you can wait few days
until sssd-1.12.5 will be officially released in upstream and packaged in fedora.

[1] https://copr.fedoraproject.org/coprs/lslebodn/sssd-1-12-latest/

Comment 23 Samuel Sieb 2015-06-11 17:14:28 UTC
Thank you and sorry for the lack of response...
I will test it out as soon as it shows up in testing.

Would it still be useful for you to have that information you requested?

Comment 24 Lukas Slebodnik 2015-06-11 17:22:15 UTC
Yes, it will be useful. If not for us then for SELinux(semanage) group.

However I read BZ one more time and you mentioned that there were not AVCs
1221370#c10. It's possible that they AVC can be seen just in (enforcing or permissive) more. 

Try to provide as much info as possible.

Comment 25 Fedora Update System 2015-06-12 20:43:35 UTC
sssd-1.12.5-2.fc21 has been submitted as an update for Fedora 21.
https://admin.fedoraproject.org/updates/sssd-1.12.5-2.fc21

Comment 26 Fedora Update System 2015-06-14 17:32:04 UTC
Package sssd-1.12.5-2.fc21:
* should fix your issue,
* was pushed to the Fedora 21 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing sssd-1.12.5-2.fc21'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2015-9990/sssd-1.12.5-2.fc21
then log in and leave karma (feedback).

Comment 27 Fedora Update System 2015-06-23 09:13:54 UTC
sssd-1.12.5-2.fc21 has been pushed to the Fedora 21 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 28 Samuel Sieb 2015-06-23 20:59:41 UTC
# ls -lZ /etc/selinux/targeted/modules/
drwx------. root root system_u:object_r:selinux_config_t:s0 active
-rw-r--r--. root root system_u:object_r:semanage_read_lock_t:s0 semanage.read.LOCK
-rw-r--r--. root root system_u:object_r:semanage_trans_lock_t:s0 semanage.trans.LOCK

The AVCs are all related to NetworkManager.  Here is a sampling.
Mostly this one:
time->Tue Jun 23 10:01:03 2015
type=USER_AVC msg=audit(1435078863.034:330): pid=598 uid=81 auid=4294967295 ses=4294967295 subj=system_u:system_r:system_dbusd_t:s0-s0:c0.c1023 msg='avc:  denied  { send_msg } for msgtype=signal interface=org.freedesktop.NetworkManager member=CheckPermissions dest=org.freedesktop.DBus spid=587 tpid=679 scontext=system_u:system_r:NetworkManager_t:s0 tcontext=system_u:system_r:cupsd_t:s0-s0:c0.c1023 tclass=dbus  exe="/usr/bin/dbus-daemon" sauid=81 hostname=? addr=? terminal=?'


But occasionally:
----
time->Tue Jun 23 08:23:29 2015
type=USER_AVC msg=audit(1435073009.661:269): pid=598 uid=81 auid=4294967295 ses=4294967295 subj=system_u:system_r:system_dbusd_t:s0-s0:c0.c1
023 msg='avc:  denied  { send_msg } for msgtype=signal interface=org.freedesktop.NetworkManager member=StateChanged dest=org.freedesktop.DBu
s spid=587 tpid=679 scontext=system_u:system_r:NetworkManager_t:s0 tcontext=system_u:system_r:cupsd_t:s0-s0:c0.c1023 tclass=dbus  exe="/usr/
bin/dbus-daemon" sauid=81 hostname=? addr=? terminal=?'
----
time->Tue Jun 23 08:23:29 2015
type=USER_AVC msg=audit(1435073009.666:270): pid=598 uid=81 auid=4294967295 ses=4294967295 subj=system_u:system_r:system_dbusd_t:s0-s0:c0.c1
023 msg='avc:  denied  { send_msg } for msgtype=signal interface=org.freedesktop.NetworkManager member=PropertiesChanged dest=org.freedeskto
p.DBus spid=587 tpid=679 scontext=system_u:system_r:NetworkManager_t:s0 tcontext=system_u:system_r:cupsd_t:s0-s0:c0.c1023 tclass=dbus  exe="
/usr/bin/dbus-daemon" sauid=81 hostname=? addr=? terminal=?'


I will now upgrade sssd to verify the fix.

Comment 29 Samuel Sieb 2015-06-24 20:35:55 UTC
After the update, there have been no more ABEND messages from selinux_child in the audit log.


Note You need to log in before you can comment on or make changes to this bug.