Bug 1019882

Summary: RHEL7 ipa ad trusted user lookups failed with sssd_be crash
Product: Red Hat Enterprise Linux 7 Reporter: Scott Poore <spoore>
Component: sssdAssignee: Jakub Hrozek <jhrozek>
Status: CLOSED CURRENTRELEASE QA Contact: Kaushik Banerjee <kbanerje>
Severity: unspecified Docs Contact:
Priority: medium    
Version: 7.0CC: grajaiya, jgalipea, lslebodn, mkosek, nsoman, pbrezina, sbose
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: sssd-1.11.1-2.el7 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-06-13 10:35:35 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
sssd_be crash abrt files none

Description Scott Poore 2013-10-16 14:51:11 UTC
Description of problem:

I am seeing user lookups fail right after setting up a trust with active directory.  Initial failure occurred trying to ssh.


administrator:*:1436800500:1436800500:Administrator:/:
:: [   PASS   ] :: Running 'getent passwd 'ADLABS\Administrator'' (Expected 0, got 0)
domain users:*:1436800513:
:: [   PASS   ] :: Running 'getent group 'ADLABS\Domain Users'' (Expected 0, got 0)
:: [   PASS   ] :: Running 'echo Secret123|kinit administrator' (Expected 0, got 0)
Connection closed by UNKNOWN
:: [   FAIL   ] :: Running 'timeout 20s             ssh -K -l administrator cloud-qe-6.spoore10151337.test 'echo login successful'' (Expected 0, got 255)

In /var/log/secure:

Oct 15 15:27:27 cloud-qe-6 sshd[12163]: Authorized to administrator, krb5 principal administrator (ssh_gssapi_krb5_cmdok)
Oct 15 15:27:29 cloud-qe-6 sshd[12163]: pam_sss(sshd:account): Access denied for user administrator: 4 (System error)
Oct 15 15:27:29 cloud-qe-6 sshd[12163]: fatal: Access denied for user administrator by PAM account configuration [preauth]

Near the end of the lookup at this timeframe in /var/log/sssd/sssd_spoore10151337.test.log:

(Tue Oct 15 15:27:29 2013) [sssd[be[spoore10151337.test]]] [sdap_access_send] (0x0400): Performing access check for user [Administrat
or]
(Tue Oct 15 15:27:29 2013) [sssd[be[spoore10151337.test]]] [sdap_access_send] (0x0040): find_subdomain_by_name failed.
(Tue Oct 15 15:27:29 2013) [sssd[be[spoore10151337.test]]] [be_pam_handler_callback] (0x0100): Backend returned: (3, 4, Cannot allocate memory) [Internal Error (System error)]

And with some help from Alexander from Dev:
<snip>
To me it looks like a new bug in sssd code. "Cannot allocate memory"
returned by the backend, crash of the domain child process in sssd.log:

(Tue Oct 15 15:27:28 2013) [sssd] [mt_svc_exit_handler] (0x0040): Child [spoore10151337.test] terminated with signal [11]
(Tue Oct 15 15:27:28 2013) [sssd] [mt_svc_restart] (0x0400): Scheduling service spoore10151337.test for restart 1
(Tue Oct 15 15:27:28 2013) [sssd] [get_ping_config] (0x0100): Time between service pings for [spoore10151337.test]: [10]
(Tue Oct 15 15:27:28 2013) [sssd] [get_ping_config] (0x0100): Time between SIGTERM and SIGKILL for [spoore10151337.test]: [60]
(Tue Oct 15 15:27:28 2013) [sssd] [start_service] (0x0100): Queueing service spoore10151337.test for startup
(Tue Oct 15 15:27:28 2013) [sssd] [sbus_server_init_new_connection] (0x0200): Entering.

and in messages:
Oct 15 15:27:28 cloud-qe-6 kernel: [ 1712.193799] traps: sssd_be[24270] general protection ip:7f75c9381e51 sp:7fffcdcdae00 error:0 in libsss_ipa.so[7f75c934f000+78000]
Oct 15 15:27:28 cloud-qe-6 abrt[12206]: Saved core dump of pid 24270 (/usr/libexec/sssd/sssd_be) to /var/tmp/abrt/ccpp-2013-10-15-15:27:28-24270 (36352000 bytes)
Oct 15 15:27:28 cloud-qe-6 abrtd: New client connected
Oct 15 15:27:28 cloud-qe-6 sssd[be[spoore10151337.test]]: Starting up
Oct 15 15:27:28 cloud-qe-6 abrt-server[12207]: Generating core_backtrace
Oct 15 15:27:28 cloud-qe-6 abrt-server[12207]: Generating backtrace
Oct 15 15:27:29 cloud-qe-6 abrt-server[12207]: New problem directory /var/tmp/abrt/ccpp-2013-10-15-15:27:28-24270, processing
Oct 15 15:27:29 cloud-qe-6 abrt-server[12207]: Sending an email...
Oct 15 15:27:29 cloud-qe-6 abrt-server[12207]: Email was sent to: seceng-idm-qe-list

Can we get access to this abrt report?
</snip>

And help from Sumit from Dev:

<snip>
I think the original issue is the core dump which was caused by an
uninitialized struct member.
</snip>

I'll upload the abrt also here for future reference as well if needed.

Version-Release number of selected component (if applicable):
sssd-1.11.1-1.el7.x86_64

How reproducible:
Unknown at this time.

Steps to Reproduce:
1. Setup AD Server
2. Setup IPA Server
3. Setup Trust to AD Server
4. kinit <aduser>
5. ssh -K -l <aduser> <ipahost>


Actual results:
failed to find the user and ssh.  sssd_be crashed

Expected results:
no crash and ssh in without password prompt

Additional info:

Comment 1 Scott Poore 2013-10-16 14:52:47 UTC
Created attachment 812985 [details]
sssd_be crash abrt files

Comment 2 Jakub Hrozek 2013-10-16 15:19:06 UTC
Upstream ticket:
https://fedorahosted.org/sssd/ticket/2121

Comment 3 Jakub Hrozek 2013-10-16 15:30:52 UTC
Fixed upstream.

Comment 5 Scott Poore 2013-11-14 23:44:30 UTC
Verified.

Version ::

sssd-ipa-1.11.2-1.el7.x86_64


Test Results ::

::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: [   LOG    ] :: ipa_trust_func_BZ1019882: [BZ1019882] RHEL7 ipa ad trusted user lookups failed with sssd_be crash
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

:: [   PASS   ] :: File '/var/log/sssd/sssd_spoore11141502.test.log' should not contain 'sssd\[be.*internal.*error' 
:: [   PASS   ] :: File '/var/log/messages' should not contain 'abrt.*core.*sssd_be' 
:: [   PASS   ] :: BZ1019882 not found

Comment 6 Ludek Smid 2014-06-13 10:35:35 UTC
This request was resolved in Red Hat Enterprise Linux 7.0.

Contact your manager or support representative in case you have further questions about the request.