Description of problem: /etc/init.d/dirsrv-admin segfaults if selinux is enforced Version-Release number of selected component (if applicable): # rpm -qa |grep 389 389-ds-1.2.1-1.el5 389-admin-1.1.11-1.el5 389-admin-console-doc-1.1.5-1.el5 389-ds-base-1.2.6-1.el5 389-dsgw-1.1.5-1.el5 389-ds-console-1.2.3-1.el5 389-ds-console-doc-1.2.3-1.el5 389-adminutil-1.1.8-4.el5 389-console-1.1.4-1.el5 389-admin-console-1.1.5-1.el5 How reproducible: Mostly always, on one system on each second start. Steps to Reproduce: 1. /etc/init.d/dirsrv-admin start Actual results: /var/log/dirsrv/admin-serv/error [Wed Sep 29 08:08:54 2010] [notice] SELinux policy enabled; httpd running as context user_u:system_r:httpd_t:s0 [Wed Sep 29 08:08:55 2010] [notice] Access Host filter is: *.prod.venyon.local [Wed Sep 29 08:08:55 2010] [notice] Access Address filter is: * [Wed Sep 29 08:08:56 2010] [notice] Access Host filter is: *.prod.venyon.local [Wed Sep 29 08:08:56 2010] [notice] Access Address filter is: * [Wed Sep 29 08:08:56 2010] [notice] Apache/2.2 configured -- resuming normal operations [Wed Sep 29 08:08:57 2010] [notice] child pid 8614 exit signal Segmentation fault (11) [Wed Sep 29 08:08:58 2010] [notice] Access Host filter is: *.prod.venyon.local [Wed Sep 29 08:08:58 2010] [notice] Access Address filter is: * [Wed Sep 29 08:08:59 2010] [notice] child pid 8684 exit signal Segmentation fault (11) [Wed Sep 29 08:09:00 2010] [notice] Access Host filter is: *.prod.venyon.local [Wed Sep 29 08:09:00 2010] [notice] Access Address filter is: * [Wed Sep 29 08:09:01 2010] [notice] child pid 8751 exit signal Segmentation fault (11) [Wed Sep 29 08:09:02 2010] [notice] Access Host filter is: *.prod.venyon.local [Wed Sep 29 08:09:02 2010] [notice] Access Address filter is: * [Wed Sep 29 08:09:03 2010] [notice] child pid 8818 exit signal Segmentation fault (11) [Wed Sep 29 08:09:04 2010] [notice] Access Host filter is: *.prod.venyon.local [Wed Sep 29 08:09:04 2010] [notice] Access Address filter is: * [Wed Sep 29 08:09:04 2010] [notice] caught SIGTERM, shutting down Expected results: Not such result Additional info: /var/log/audit.log tells: type=ANOM_ABEND msg=audit(1285747846.609:3349): auid=1000 uid=99 gid=99 ses=403 subj=user_u:system_r:httpd_t:s0 pid=9150 comm="httpd.worker" sig=11 I have already applied workaround from https://bugzilla.redhat.com/show_bug.cgi?id=552419#c21 (port 9830 issue) to be able to start, and also I've tried the semodule command from https://bugzilla.redhat.com/show_bug.cgi?id=624310 But this won't help. BTW: looks like something is very strange in addition: # pwd /root # /etc/init.d/dirsrv-admin start Starting dirsrv-admin: [ OK ] => segfaulting # pwd /var/lib/dirsrv/slapd-myinstance # /etc/init.d/dirsrv-admin start Starting dirsrv-admin: [FAILED] Log: [Wed Sep 29 08:13:45 2010] [notice] SELinux policy enabled; httpd running as context user_u:system_r:httpd_t:s0 [Wed Sep 29 08:13:46 2010] [error] Unable to change directory to /var/lib/dirsrv/slapd-myinstance Why depends it on the current pwd, whether the initscript will start dirsrv-admin or not. Perhaps this maillist entry relates on the same issue: http://www.spinics.net/lists/fedora-directory/msg12144.html
Do any AVC messages related to dirsrv or dirsrv-admin show up in /var/log/audit/audit when you run 'service dirsrv-admin start'?
Unfortunately, no AVC messages appear besides the one shown above. But I know that SELinux is able to have "silent denies", which are not shown by default in log. But I don't remember, how to display them...
If you can reproduce the crash, I would appreciate using gdb to get a stack trace. To make the stack trace useful, the 389-admin debuginfo would need to be installed.
Just the same problem when SELinux is enforced. Putting SELinux in permissive mode solves the problem.
(In reply to comment #7) > Just the same problem when SELinux is enforced. Putting SELinux in permissive > mode solves the problem. Are you running RHEL5 or CentOS 5?
(In reply to comment #8) > Are you running RHEL5 or CentOS 5? CentOS release 5.5
(In reply to comment #7) > Just the same problem when SELinux is enforced. Putting SELinux in permissive > mode solves the problem. Do you see any AVCs in the audit log (/var/log/audit/audit)?
I got some spare time(In reply to comment #5) > If you can reproduce the crash, I would appreciate using gdb to get a stack > trace. To make the stack trace useful, the 389-admin debuginfo would need to > be installed. Unfortunately debuginfo.centos.org is out-of-date: Could not find debuginfo for main pkg: 389-admin-1.1.11-1.el5.i386 Could not find debuginfo pkg for dependency package 389-adminutil-1.1.8-4.el5.i386 Could not find debuginfo pkg for dependency package glibc-2.5-49.el5_5.7.i386 Therefore I can't debug further on as requested
(In reply to comment #11) > I got some spare time(In reply to comment #5) > > If you can reproduce the crash, I would appreciate using gdb to get a stack > > trace. To make the stack trace useful, the 389-admin debuginfo would need to > > be installed. > > Unfortunately debuginfo.centos.org is out-of-date: > > Could not find debuginfo for main pkg: 389-admin-1.1.11-1.el5.i386 > Could not find debuginfo pkg for dependency package > 389-adminutil-1.1.8-4.el5.i386 > Could not find debuginfo pkg for dependency package glibc-2.5-49.el5_5.7.i386 > > Therefore I can't debug further on as requested try yum install --enablerepo=epel-debuginfo 389-admin-debuginfo
(In reply to comment #10) > Do you see any AVCs in the audit log (/var/log/audit/audit)? When I restart the dirsrv-admin service with SELinux enforced: $/etc/init.d/dirsrv-admin restart I see the following messages: $tailf /var/log/audit/audit.log type=ANOM_ABEND msg=audit(1290015232.285:21): auid=0 uid=501 gid=501 ses=2 subj=root:system_r:httpd_t:s0 pid=2400 comm="httpd.worker" sig=11 type=ANOM_ABEND msg=audit(1290015234.298:22): auid=0 uid=501 gid=501 ses=2 subj=root:system_r:httpd_t:s0 pid=2470 comm="httpd.worker" sig=11 type=ANOM_ABEND msg=audit(1290015236.307:23): auid=0 uid=501 gid=501 ses=2 subj=root:system_r:httpd_t:s0 pid=2537 comm="httpd.worker" sig=11 type=ANOM_ABEND msg=audit(1290015238.342:24): auid=0 uid=501 gid=501 ses=2 subj=root:system_r:httpd_t:s0 pid=2604 comm="httpd.worker" sig=11 ...
While I was not able to start the ds with gdb, at least I found that it depends on the SELinux context, whether it crashes or not. -rwxr-xr-x root root system_u:object_r:initrc_exec_t:s0 /etc/init.d/dirsrv-admin -rwxr-xr-x root root system_u:object_r:dirsrvadmin_exec_t:s0 /usr/sbin/start-ds-admin If scripts are copied and have e.g. -rwxr-xr-x root root root:object_r:user_home_t:s0 Then the problem went away.
same problem. CentOS 5.5 x64 389-ds-1.2.1-1.el5 389-admin-1.1.13-1.el5 389-ds-base-1.2.7.5-1.el5 389-ds-console-1.2.3-1.el5 389-console-1.1.4-1.el5 389-dsgw-1.1.5-1.el5 389-ds-console-doc-1.2.3-1.el5 389-adminutil-1.1.8-4.el5 389-admin-console-1.1.5-1.el5
you must upgrade 389-adminutil to the latest
I am able to reproduce this issue on a fully up to date RHEL5 system using the latest 389 packages from EPEL. Installing 389-adminutil-1.1.13 from epel-testing does not fix the issue.
I was able to get a stack trace from one of the httpd child process when it crashes: Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x40d14940 (LWP 15914)] 0x00002b17df807820 in ?? () (gdb) bt #0 0x00002b17df807820 in ?? () #1 0x00002b17d96c7ad9 in __nptl_deallocate_tsd () from /lib64/libpthread.so.0 #2 0x00002b17d96c874b in start_thread () from /lib64/libpthread.so.0 #3 0x00002b17d9bb5f6d in clone () from /lib64/libc.so.6 I'll attach a dump of all threads. I'm not sure if this is a selinux issue or an httpd issue, but it doesn't appear to be an issue with the 389-admin code itself.
Created attachment 471755 [details] threads stack traces
You can turn off dontaudit rules semodule -DB To see if an SELinux dontaudit rule is causing the problem.
Created attachment 471766 [details] AVC messages Here are the AVC messages that are reported after turning off dontaudit messages.
Creating a custom selinux module with the following rule appears to fix this issue: allow dirsrvadmin_t httpd_t:process { siginh rlimitinh noatsecure };
Created attachment 471771 [details] Patch
Pushed patch to master. Thanks to Noriko for her review! Counting objects: 7, done. Delta compression using up to 2 threads. Compressing objects: 100% (4/4), done. Writing objects: 100% (4/4), 555 bytes, done. Total 4 (delta 3), reused 0 (delta 0) To ssh://git.fedorahosted.org/git/389/admin.git f40a2f1..6d86721 master -> master
1. Tested on Red Hat Enterprise Linux Server release 6.0 (Santiago) with [root@testvm ~]# rpm -qa | grep 389 389-adminutil-1.1.13-1.el6.x86_64 389-console-1.1.4-1.el6.noarch 389-admin-console-1.1.7-1.el6.noarch 389-ds-base-libs-1.2.8.2-1.el6_1.1.x86_64 389-admin-1.1.16-2.el6.x86_64 389-ds-console-1.2.5-1.el6.noarch 389-ds-base-1.2.8.2-1.el6_1.1.x86_64 2. [root@testvm ~]# cat /etc/sysconfig/selinux # This file controls the state of SELinux on the system. # SELINUX= can take one of these three values: # enforcing - SELinux security policy is enforced. # permissive - SELinux prints warnings instead of enforcing. # disabled - No SELinux policy is loaded. SELINUX=enforcing # SELINUXTYPE= can take one of these two values: # targeted - Targeted processes are protected, # mls - Multi Level Security protection. SELINUXTYPE=targeted 3. /etc/init.d/dirsrv-admin start 4. [root@testvm ~]# tail -f /var/log/audit/audit.log type=LOGIN msg=audit(1305718261.449:1359): login pid=8480 uid=0 old auid=4294967295 new auid=0 old ses=4294967295 new ses=220 type=USER_START msg=audit(1305718261.449:1360): user pid=8480 uid=0 auid=0 ses=220 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:session_open acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success' type=CRED_DISP msg=audit(1305718261.483:1361): user pid=8480 uid=0 auid=0 ses=220 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:setcred acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success' type=USER_END msg=audit(1305718261.483:1362): user pid=8480 uid=0 auid=0 ses=220 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:session_close acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success' type=USER_ACCT msg=audit(1305718801.491:1363): user pid=8524 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:accounting acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success' type=CRED_ACQ msg=audit(1305718801.491:1364): user pid=8524 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:setcred acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success' type=LOGIN msg=audit(1305718801.491:1365): login pid=8524 uid=0 old auid=4294967295 new auid=0 old ses=4294967295 new ses=221 type=USER_START msg=audit(1305718801.491:1366): user pid=8524 uid=0 auid=0 ses=221 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:session_open acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success' type=CRED_DISP msg=audit(1305718801.507:1367): user pid=8524 uid=0 auid=0 ses=221 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:setcred acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success' type=USER_END msg=audit(1305718801.507:1368): user pid=8524 uid=0 auid=0 ses=221 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:session_close acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success' No problem encountered here, marking the bug as VERIFIED.