Bug 1278566

Summary: crash in sssd when non-Englsh locale is used and pam_strerror prints non-ASCII characters
Product: Red Hat Enterprise Linux 7 Reporter: Jakub Hrozek <jhrozek>
Component: sssdAssignee: SSSD Maintainers <sssd-maint>
Status: CLOSED ERRATA QA Contact: Steeve Goveas <sgoveas>
Severity: medium Docs Contact:
Priority: medium    
Version: 7.2CC: abokovoy, apeetham, extras-qa, grajaiya, jgalipea, jhrozek, lslebodn, mkosek, mzidek, pbrezina, rharwood, sbose, sgoveas, ssorce, terje.rosten
Target Milestone: rcKeywords: Regression
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: sssd-1.14.0-0.1.alpha.el7 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 1269232 Environment:
Last Closed: 2016-11-04 07:12:23 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1269232    
Bug Blocks:    

Description Jakub Hrozek 2015-11-05 19:53:44 UTC
+++ This bug was initially created as a clone of Bug #1269232 +++

Description of problem:

Since sssd 1.13 has been crashing, downgrading to 1.12, seems to solve the problem. 

I get the following trace:
 
    Stack trace of thread 8379:
                                                              #0  0x00007fac4d7d2a98 raise (libc.so.6)
                                                              #1  0x00007fac4d7d469a abort (libc.so.6)
                                                              #2  0x00007fac5168d3b4 _dbus_abort (libdbus-1.so.3)
                                                              #3  0x00007fac51683e3e _dbus_warn_check_failed (libdbus-1.so.3)
                                                              #4  0x00007fac51675d96 dbus_message_iter_append_basic (libdbus-1.so.3)
                                                              #5  0x00007fac51676698 dbus_message_append_args_valist (libdbus-1.so.3)
                                                              #6  0x00007fac51d1c329 sbus_request_return_and_finish (libsss_util.so)
                                                              #7  0x000055865389b3f3 acctinfo_callback (sssd_be)
                                                              #8  0x00007fac427e0d19 sdap_account_info_complete (libsss_ldap_common.so)
                                                              #9  0x00007fac427ddfe0 users_get_done (libsss_ldap_common.so)
                                                              #10 0x00007fac4df7c1c2 _tevent_req_error (libtevent.so.0)
                                                              #11 0x00007fac4df7c1c2 _tevent_req_error (libtevent.so.0)
                                                              #12 0x00007fac427f6314 sdap_search_user_process (libsss_ldap_common.so)
                                                              #13 0x00007fac427f137c generic_ext_search_handler.isra.3 (libsss_ldap_common.so)
                                                              #14 0x00007fac427f2ee3 sdap_get_generic_op_finished (libsss_ldap_common.so)
                                                              #15 0x00007fac427f22fd sdap_process_result (libsss_ldap_common.so)
                                                              #16 0x00007fac4df803c3 epoll_event_loop_once (libtevent.so.0)
                                                              #17 0x00007fac4df7e907 std_event_loop_once (libtevent.so.0)
                                                              #18 0x00007fac4df7b11d _tevent_loop_once (libtevent.so.0)
                                                              #19 0x00007fac4df7b2bb tevent_common_loop_wait (libtevent.so.0)
                                                              #20 0x00007fac4df7e8a7 std_event_loop_wait (libtevent.so.0)
                                                              #21 0x00007fac51d21fd3 server_loop (libsss_util.so)
                                                              #22 0x000055865389ab76 main (sssd_be)
                                                              #23 0x00007fac4d7be580 __libc_start_main (libc.so.6)
                                                              #24 0x000055865389ac09 _start (sssd_be)


Version-Release number of selected component (if applicable):

sssd-1.13.0-7.fc23.x86_64

How reproducible:

Config sssd with something like this:

[sssd]
config_file_version = 2
reconnection_retries = 3
sbus_timeout = 30
services = nss, pam, autofs
domains = default

[nss]
filter_groups = root
filter_users = root
reconnection_retries = 3

[pam]
reconnection_retries = 3
[domain/default]
auth_provider = ldap
ldap_tls_reqcert = never
ldap_id_use_start_tls = True
cache_credentials = True
ldap_search_base = dc=foo,dc=bar
krb5_realm = EXAMPLE.COM
chpass_provider = ldap
id_provider = ldap
use_fully_qualified_names = False
store_legacy_passwords = False
debug_level = 0
min_id = 100
ldap_uri = ldap://example.com
enumerate = True
autofs_provider = ldap

ldap_tls_cacertdir = /etc/openldap/cacerts
[autofs]

and enable sssd in authconfig.

Start sssd service and wait for crash.

--- Additional comment from Jakub Hrozek on 2015-10-06 14:42:20 EDT ---

Can you please also upload the logs and the corefile?

--- Additional comment from Terje Røsten on 2015-10-06 14:44:36 EDT ---

where do I find the core file? 

Anything I need to config to get core file from sssd?

--- Additional comment from Jakub Hrozek on 2015-10-06 15:13:37 EDT ---

abrt should help you with that, abrt-cli might print where the corefiles are. Normally they're at /var/spool/abrt/ IIRC.

But since we switched to calling setuid and drop privileges, the corefiles might not be generated automatically. In that case, you can attach gdb as described here:
https://fedorahosted.org/sssd/wiki/Reporting_sssd_bugs

How to get debug files is described here:
https://fedorahosted.org/sssd/wiki/Troubleshooting

--- Additional comment from Terje Røsten on 2015-10-06 15:21:48 EDT ---

Thanks, I have removed all abrt* packages any way, let me see what I can out from gdb.

--- Additional comment from Lukas Slebodnik on 2015-10-08 04:31:08 EDT ---

(In reply to Terje Røsten from comment #4)
> Thanks, I have removed all abrt* packages any way, let me see what I can out
> from gdb.

You should be able t obtain coredump even without abrt.
You might take a look into coredumpctl[1]. If you didn't change core_pattern manually.

[root@host ~]# cat /proc/sys/kernel/core_pattern
|/usr/lib/systemd/systemd-coredump %p %u %g %s %t %e

[1] http://www.freedesktop.org/software/systemd/man/coredumpctl.html

If you are able to reproduce crash. We might also try to run sssd with valgrind.

--- Additional comment from Jakub Hrozek on 2015-10-08 16:58:34 EDT ---

In general Terje, please let us know if we can help reproduce the issue or get the core file!

--- Additional comment from Jakub Hrozek on 2015-10-14 10:01:09 EDT ---

ping?

Can we help you get the debug information?

--- Additional comment from Lukas Slebodnik on 2015-10-30 16:45:39 EDT ---

It seems to be related to your locales.
I was able to reproduce with setting locales to "norwegian" 

So the workaround is to add line "LANG=C" to the /etc/sysconfig/sssd.

--- Additional comment from Terje Røsten on 2015-10-31 04:55:27 EDT ---

So far so good, thanks guys!

--- Additional comment from Lukas Slebodnik on 2015-11-05 08:29:07 EST ---

Upstream ticket:
https://fedorahosted.org/sssd/ticket/2861

Comment 3 Martin Kosek 2015-11-06 17:14:47 UTC
Workaround:
https://bugzilla.redhat.com/show_bug.cgi?id=1269232#c8

Comment 5 Jakub Hrozek 2015-11-06 20:17:22 UTC
I did some more investigation today and I don't think it's as severe as we initially thought. This bug doesn't happen with any non-English locale, but only with those that are non-UTF-8.

The root cause is that we transfer diagnostic messages over sbus and if the message is not UTF-8, dbus aborts.

Comment 6 Jakub Hrozek 2015-11-06 20:26:21 UTC
patch is on review btw.

Comment 7 Jakub Hrozek 2015-11-14 12:41:28 UTC
Fixed upstream:

master:
    6b01dae732eedee808f32a9cdd4b5656a9f839c4
    e8ae3af6724164048a85c374ea8045a368a2d34e
    8bc6bc6d87127d615f7a81d7151cb46007feff63 
sssd-1-13:
    620b2adf7c19c1b9abc456c37926d77649d262b7
    518c003f3d555a525fe3aebf60366e0c0716eff4
    b51ed15a72a3c8542f3dab72158aeee1d7254341

Comment 8 Mike McCune 2016-03-28 23:36:13 UTC
This bug was accidentally moved from POST to MODIFIED via an error in automation, please see mmccune with any questions

Comment 12 Amith 2016-09-20 04:50:30 UTC
Verified Sanity Only, on sssd version: sssd-1.14.0-42.el7.x86_64

Comment 14 errata-xmlrpc 2016-11-04 07:12:23 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHEA-2016-2476.html