Bug 748924

Summary: RHEL6.1/sssd_pam segmentation fault
Product: Red Hat Enterprise Linux 6 Reporter: Masaki Furuta ( RH ) <mfuruta>
Component: sssdAssignee: Stephen Gallagher <sgallagh>
Status: CLOSED ERRATA QA Contact: IDM QE LIST <seceng-idm-qe-list>
Severity: high Docs Contact:
Priority: high    
Version: 6.1CC: ddumas, grajaiya, jgalipea, jzeleny, kbanerje, prc
Target Milestone: rc   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: sssd-1.5.1-62.el6 Doc Type: Bug Fix
Doc Text:
Cause: When an error occurred in SSSD during composition of reply message to PAM, SSSD tried to send a reply packet to pam_sss even though the packet was not prepared yet. Consequence: SSSD PAM responder crashed. Fix: SSSD now detects if the response packet is already created. Result: In case of internal error such as that described above, the client will be forcibly disconnected and the SSSD won't crash.
Story Points: ---
Clone Of: Environment:
Last Closed: 2011-12-06 16:41:17 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 750914    

Description Masaki Furuta ( RH ) 2011-10-25 15:14:33 UTC
Created attachment 530119 [details]
corefile of segrault with sssd_pam

Description of problem:

  sssd_pam segmentation fault

  messages
  ---------
  Sep 20 09:33:19 jonah kernel: sssd_pam[1971]: segfault at 10 ip 000000000042f453 sp 00007fffbab2d780 error 4 in sssd_pam[400000+3d000]
  Sep 20 09:33:19 jonah abrt[10948]: saved core dump of pid 1971 (/usr/libexec/sssd/sssd_pam) to /var/spool/abrt/ccpp-1316478799-

  1971.new/coredump (1568768 bytes)
  Sep 20 09:33:19 jonah abrtd: Directory 'ccpp-1316478799-1971' creation detected
  Sep 20 09:33:20 jonah sssd[pam]: Starting up
  Sep 20 09:33:20 jonah abrtd: New crash /var/spool/abrt/ccpp-1316478799-1971, processing

Version-Release number of selected component (if applicable):

  RHEL6.1
  kernel 2.6.32-131.0.15.el6.x86_64
  sssd-1.5.1-34.el6.x86_64 

How reproducible:

  Sometimes, the customer is trying to reproduce it, but still not sure.
  Nothig but core file.

----
$ tar tvzf sssd_pam_abrtlog.tar.gz 
-rw-r----- abrt/root         4 2011-09-20 09:33 ./analyzer
-rw-r----- abrt/root         6 2011-09-20 09:33 ./architecture
-rw-r----- abrt/root      4786 2011-09-20 11:45 ./backtrace
-rw-r----- abrt/root        48 2011-09-20 09:33 ./cmdline
-rw-r----- abrt/root         4 2011-09-20 09:33 ./component
-rw-r--r-- root/root   1568768 2011-09-20 09:33 ./coredump <======= core file
-rw-r----- abrt/root       359 2011-09-20 09:33 ./description
-rw-r----- abrt/root        26 2011-09-20 09:33 ./executable
-rw-r----- abrt/root        40 2011-09-20 11:45 ./global_uuid
-rw-r----- abrt/root         5 2011-09-20 09:33 ./hostname
-rw-r----- abrt/root        26 2011-09-20 09:33 ./kernel
-rw-r----- abrt/root        17 2011-09-20 09:33 ./package
-rw-r----- abrt/root         1 2011-09-20 11:45 ./rating
-rw-r----- abrt/root        68 2011-09-20 09:33 ./reason
-rw-r----- abrt/root        54 2011-09-20 09:33 ./release
-rw------- root/root  12357332 2011-09-20 09:33 ./sosreport.tar.xz
-rw-r----- abrt/root        10 2011-09-20 09:33 ./time
-rw-r----- abrt/root         1 2011-09-20 09:33 ./uid
---

Steps to Reproduce:
1.
2.
3.
  
Actual results:

  Crashed with Segfault with sssd_pam

Expected results:

No crash.

Additional info:

[root@dhcp-207-132 sssd_pam_abrtlog]# gdb /usr/libexec/sssd/sssd_pam coredump 
<snip>
Core was generated by `/usr/libexec/sssd/sssd_pam -d 0 --debug-to-files'.
Program terminated with signal 11, Segmentation fault.
#0  sss_packet_send (packet=0x0, fd=27)
    at src/responder/common/responder_packet.c:221
221	    len = *packet->len - packet->iop;
(gdb) info share
From                To                  Syms Read   Shared Object Library
0x0000003dfb4022d0  0x0000003dfb4068a8  Yes         /usr/lib64/libtevent.so.0.9.8
0x0000003dfd001870  0x0000003dfd007a18  Yes         /usr/lib64/libtalloc.so.2.0.1
0x0000003e0ac01b10  0x0000003e0ac06ee8  Yes         /lib64/libpopt.so.0.0.0
0x0000003dfc0074a0  0x0000003dfc026968  Yes         /usr/lib64/libldb.so.0.9.10
0x0000003dfec07090  0x0000003dfec2e518  Yes         /lib64/libdbus-1.so.3.4.0
0x0000003dfb002140  0x0000003dfb0055a8  Yes         /lib64/librt-2.12.so
0x0000003e0b001540  0x0000003e0b01ae58  Yes         /lib64/libpcre.so.0.0.1
0x0000003dfc402740  0x0000003dfc406138  Yes         /usr/lib64/libini_config.so.2.0.0
0x0000003dfd802f80  0x0000003dfd808958  Yes         /usr/lib64/libcollection.so.2.0.0
0x0000003dfd400980  0x0000003dfd401b48  Yes         /usr/lib64/libdhash.so.1.0.0
0x0000003e0b803590  0x0000003e0b80a848  Yes         /lib64/liblber-2.4.so.2.5.6
0x0000003e0c80e050  0x0000003e0c83a468  Yes         /lib64/libldap-2.4.so.2.5.6
0x0000003e0fc01e30  0x0000003e0fc09cb8  Yes         /usr/lib64/libtdb.so.1.2.1
0x0000003e09807b90  0x0000003e09829ed8  Yes         /usr/lib64/libssl3.so
0x0000003e09c09880  0x0000003e09c219c8  Yes         /usr/lib64/libsmime3.so
0x0000003e08818630  0x0000003e088fd178  Yes         /usr/lib64/libnss3.so
0x0000003e08408560  0x0000003e084137e8  Yes         /usr/lib64/libnssutil3.so
0x0000003e09400ea0  0x0000003e09401d58  Yes         /lib64/libplds4.so
0x0000003e08c013d0  0x0000003e08c02b08  Yes         /lib64/libplc4.so
0x0000003e0900cf90  0x0000003e0902c758  Yes         /lib64/libnspr4.so
0x0000003dfa405640  0x0000003dfa410f28  Yes         /lib64/libpthread-2.12.so
0x0000003df9c00de0  0x0000003df9c01998  Yes         /lib64/libdl-2.12.so
0x0000003dfa01e9e0  0x0000003dfa13d370  Yes         /lib64/libc-2.12.so
0x0000003df9800b00  0x0000003df981984b  Yes         /lib64/ld-2.12.so
0x0000003dfb800c80  0x0000003dfb801ee8  Yes         /usr/lib64/libpath_utils.so.1.0.0
0x0000003dfc8008d0  0x0000003dfc801158  Yes         /usr/lib64/libref_array.so.1.0.0
0x0000003dfbc038c0  0x0000003dfbc12558  Yes         /lib64/libresolv-2.12.so
0x0000003e0a8046a0  0x0000003e0a814408  Yes         /usr/lib64/libsasl2.so.2.0.23
0x0000003dfac01ef0  0x0000003dfac0d1a8  Yes         /lib64/libz.so.1.2.3
0x0000003e05400c00  0x0000003e054059a8  Yes         /lib64/libcrypt-2.12.so
0x0000003e06c03270  0x0000003e06c42928  Yes         /lib64/libfreebl3.so
0x00007f6ea1d13270  0x00007f6ea1d19c48  Yes         /usr/lib64/ldb/memberof.so
(gdb) bt full
#0  sss_packet_send (packet=0x0, fd=27)
    at src/responder/common/responder_packet.c:221
        rb = <value optimized out>
        len = <value optimized out>
        buf = <value optimized out>
#1  0x000000000042b74b in client_send (ev=<value optimized out>, 
    fde=<value optimized out>, flags=<value optimized out>, 
    ptr=<value optimized out>) at src/responder/common/responder_common.c:134
        ret = <value optimized out>
#2  client_fd_handler (ev=<value optimized out>, fde=<value optimized out>, 
    flags=<value optimized out>, ptr=<value optimized out>)
    at src/responder/common/responder_common.c:224
        cctx = 0x1a196a0
#3  0x0000003dfb405456 in epoll_event_loop (ev=<value optimized out>, 
    location=<value optimized out>) at tevent_standard.c:309
        fde = <value optimized out>
        flags = <value optimized out>
        ret = 1
        i = <value optimized out>
        events = {{events = 4, data = {ptr = 0x1a18350, fd = 27362128, 
              u32 = 27362128, u64 = 27362128}}}
        timeout = <value optimized out>
#4  std_event_loop_once (ev=<value optimized out>, 
    location=<value optimized out>) at tevent_standard.c:544
        std_ev = 0x1a103e0
        tval = {tv_sec = 30, tv_usec = 0}
#5  0x0000003dfb4026d0 in _tevent_loop_once (ev=0x1a10320, 
    location=0x436b75 "src/util/server.c:526") at tevent.c:490
        ret = <value optimized out>
        nesting_stack_ptr = 0x0
#6  0x0000003dfb40273b in tevent_common_loop_wait (ev=0x1a10320, 
    location=0x436b75 "src/util/server.c:526") at tevent.c:591
        ret = <value optimized out>
#7  0x0000000000425861 in server_loop (main_ctx=0x1a11420)
    at src/util/server.c:526
No locals.
#8  0x00000000004079b0 in main (argc=<value optimized out>, 
    argv=<value optimized out>) at src/responder/pam/pamsrv.c:230
        opt = <value optimized out>
        pc = <value optimized out>
        main_ctx = 0x1a11420
        ret = <value optimized out>
        long_options = {{longName = 0x0, shortName = 0 '\000', argInfo = 4, 
            arg = 0x63dd20, val = 0, descrip = 0x430e02 "Help options:", 
            argDescrip = 0x0}, {longName = 0x430e10 "debug-level", 
            shortName = 100 'd', argInfo = 2, arg = 0x63de18, val = 0, 
            descrip = 0x430de1 "Debug level", argDescrip = 0x0}, {
            longName = 0x430e1c "debug-to-files", shortName = 102 'f', 
            argInfo = 0, arg = 0x63de1c, val = 0, 
            descrip = 0x430f48 "Send the debug output to files instead of stderr", argDescrip = 0x0}, {longName = 0x430e2b "debug-timestamps", 
            shortName = 0 '\000', argInfo = 2, arg = 0x63dce0, val = 0, 
            descrip = 0x430ded "Add debug timestamps", argDescrip = 0x0}, {
            longName = 0x0, shortName = 0 '\000', argInfo = 0, arg = 0x0, 
            val = 0, descrip = 0x0, argDescrip = 0x0}}
        __FUNCTION__ = "main"

Comment 2 Stephen Gallagher 2011-10-25 18:13:41 UTC
Upstream ticket:
https://fedorahosted.org/sssd/ticket/1061

Comment 7 Jan Zeleny 2011-10-27 11:40:52 UTC
    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
Cause: When an error occurred in SSSD during composition of reply message to PAM, SSSD tried to send a reply packet to pam_sss even though the packet was not prepared yet.
Consequence: SSSD PAM responder crashed.
Fix: SSSD now detects if the response packet is already created.
Result: In case of internal error such as that described above, the client will be forcibly disconnected and the SSSD won't crash.

Comment 8 Kaushik Banerjee 2011-11-07 14:23:38 UTC
Marking this bug VERIFIED as all automation regression tests have passed.

Verified in version:
# rpm -qi sssd | head
Name        : sssd                         Relocations: (not relocatable)
Version     : 1.5.1                             Vendor: Red Hat, Inc.
Release     : 66.el6                        Build Date: Tue 01 Nov 2011 02:05:40 AM IST
Install Date: Thu 03 Nov 2011 04:06:20 PM IST      Build Host: x86-003.build.bos.redhat.com
Group       : Applications/System           Source RPM: sssd-1.5.1-66.el6.src.rpm
Size        : 3628521                          License: GPLv3+
Signature   : (none)
Packager    : Red Hat, Inc. <http://bugzilla.redhat.com/bugzilla>
URL         : http://fedorahosted.org/sssd/
Summary     : System Security Services Daemon

Comment 9 errata-xmlrpc 2011-12-06 16:41:17 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2011-1529.html