Hide Forgot
This bug is created as a clone of upstream ticket: https://fedorahosted.org/sssd/ticket/3118 testing sssd with overrides (users and groups) on centos 7.2 (sssd 1.13.0, libldb 1.1.25), i soon ran in to problems with sssd_nss crashing. dmesg shows: sssd_nss[28935]: segfault at 51 ip 00007fa5e39d46af sp 00007ffcd6f18290 error 4 in libldb.so.1.1.25[7fa5e39c4000+2d000] backtrace: {{{ Program received signal SIGSEGV, Segmentation fault. 0x00007f4b514276af in ldb_dn_from_ldb_val (mem_ctx=mem_ctx@entry=0x7f4b52297300, ldb=0x7f4b5229dad0, strdn=0x25) at ../common/ldb_dn.c:97 97 if (strdn && strdn->data (gdb) bt #0 0x00007f4b514276af in ldb_dn_from_ldb_val (mem_ctx=mem_ctx@entry=0x7f4b52297300, ldb=0x7f4b5229dad0, strdn=0x25) at ../common/ldb_dn.c:97 #1 0x00007f4b5188109a in sysdb_add_group_member_overrides (domain=domain@entry=0x7f4b52296120, obj=0x7f4b522a3e40) at src/db/sysdb_views.c:1308 #2 0x00007f4b5187373c in sysdb_getgrgid_with_views (mem_ctx=mem_ctx@entry=0x7f4b52295ea0, domain=domain@entry=0x7f4b52296120, gid=65751, res=res@entry=0x7f4b522a3260) at src/db/sysdb_search.c:659 #3 0x00007f4b51ef292c in nss_cmd_getgrgid_search (dctx=dctx@entry=0x7f4b522a3240) at src/responder/nss/nsssrv_cmd.c:3349 #4 0x00007f4b51ef672d in nss_cmd_getbyid (cmd=<optimized out>, cctx=0x7f4b522a0900) at src/responder/nss/nsssrv_cmd.c:1975 #5 0x00007f4b51f01b2e in client_cmd_execute (sss_cmds=0x7f4b521182e0 <nss_cmds>, cctx=0x7f4b522a0900) at src/responder/common/responder_common.c:249 #6 client_recv (cctx=0x7f4b522a0900) at src/responder/common/responder_common.c:283 #7 client_fd_handler (ev=<optimized out>, fde=<optimized out>, flags=<optimized out>, ptr=<optimized out>) at src/responder/common/responder_common.c:335 #8 0x00007f4b4e15bd0b in epoll_event_loop_once () from /lib64/libtevent.so.0 #9 0x00007f4b4e15a1d7 in std_event_loop_once () from /lib64/libtevent.so.0 #10 0x00007f4b4e15636d in _tevent_loop_once () from /lib64/libtevent.so.0 #11 0x00007f4b4e15650b in tevent_common_loop_wait () from /lib64/libtevent.so.0 #12 0x00007f4b4e15a177 in std_event_loop_wait () from /lib64/libtevent.so.0 #13 0x00007f4b51891553 in server_loop (main_ctx=0x7f4b5228d2a0) at src/util/server.c:668 #14 0x00007f4b51eeaf77 in main (argc=8, argv=<optimized out>) at src/responder/nss/nsssrv.c:626 }}} running sssd with -d 9, and having added ldb tracing to src/db/sysdb.c (line 59 in the unpatched source): ret = ldb_connect(ldb, filename, LDB_FLG_ENABLE_TRACING, NULL); things die just after retrieving the override information for a member of a group, e.g. (username/domain name removed): {{{ [sssd[nss]] [ldb] (0x4000): ldb_trace_response: ENTRY dn: overrideAnchorUUID=:LOCAL:name\3Dusername\,cn\3Dusers\,cn\3Ddomainname\,cn\3Dsysdb,cn=LOCAL,cn=views,cn=sysdb loginShell: /bin/tcsh name: username objectClass: userOverride overrideObjectDN: name=username,cn=users,cn=domainname,cn=sysdb uidNumber: 6272 [sssd[nss]] [ldb] (0x4000): Destroying timer event 0x7f4b522a3cc0 "ltdb_timeout" [sssd[nss]] [ldb] (0x4000): Ending timer event 0x7f4b522ae710 "ltdb_callback" [sssd[nss]] [sysdb_add_group_member_overrides] (0x4000): Added [username] to [overridememberUid]. }}} it always seems to fail on the first or second member of the group, and it is always when the group being looked at has overrides (gid). when dropping back to the previous ldb packages for centos 7.2 (ldb 1.1.20) everything seems to work just fine, so i looked at the differences, and it seems that this change, which was added in ldb 1.1.24 might be significant: {{{ --- ldb-1.1.20/ldb_tdb/ldb_search.c 2014-09-16 19:04:31.000000000 +0100 +++ ldb-1.1.25/ldb_tdb/ldb_search.c 2015-12-10 11:01:40.000000000 +0000 @@ -407,10 +407,18 @@ } talloc_free(msg->elements); - msg->elements = talloc_realloc(msg, el2, struct ldb_message_element, msg->num_elements); + + if (num_elements > 0) { + msg->elements = talloc_realloc(msg, el2, struct ldb_message_element, + num_elements); + } else { + msg->elements = talloc_array(msg, struct ldb_message_element, 0); + talloc_free(el2); + } if (msg->elements == NULL) { return -1; } }}} reverting this changes stops things from crashing, as does just adding 1 to num_elements in the talloc_realloc call, e.g.: {{{ --- ldb-1.1.25/ldb_tdb/ldb_search.c 2015-12-10 11:01:40.000000000 +0000 +++ ldb-1.1.25.test/ldb_tdb/ldb_search.c 2016-08-02 16:37:01.823488833 +0100 @@ -410,7 +410,7 @@ if (num_elements > 0) { msg->elements = talloc_realloc(msg, el2, struct ldb_message_element, - num_elements); + num_elements+1); }}} i have had a bit of a poke around, but can't say i have been able to work out exactly why this is the case ... i would like to have been able to give a better report of the exact cause of the problem, but have unfortunately run out of time to look at this for now. at the moment, i can stick with ldb-1.1.20, but that's not really a long term solution. i did also do some quick testing with sssd-1.14.0, and the problem remains. let me know if i can provide any more information. thanks, richard
Created attachment 1218032 [details] Simple LD_PRELOAD for reproducing a crash How to reproduce a crash: * compile a module gcc -Wall -fPIC -shared -o slow_realloc.so slow_realloc.c -ldl * cp slow_realloc.so /usr/lib64/slow_realloc.so * Use this module in sssd echo 'export LD_PRELOAD=/usr/lib64/slow_realloc.so' >> /etc/sysconfig/sssd * run test for local_overrides
sssd-1-13: * 55fc0bb19e6205af13828a98592b283d3b6d24e0 * 19ba10fcc7dbdfdd7a238fa94f57605cf16fc28e * 8e19dce22b286f1f815cba7150149ab249a62854 * 5d64343d5ffed9cb42184eb30e5bf1871d8196d5 * ce714745ad28dfb6dcfd9f8f8983e492661a6e2f * 3bea6818a3432a349a9901a84fd517c052b19f69
Versions: Reproducer: ============ sssd-common-pac-1.13.3-22.el6.x86_64 sssd-ad-1.13.3-22.el6.x86_64 sssd-tools-1.13.3-22.el6.x86_64 sssd-client-1.13.3-22.el6.x86_64 sssd-common-1.13.3-22.el6.x86_64 sssd-proxy-1.13.3-22.el6.x86_64 sssd-krb5-common-1.13.3-22.el6.x86_64 sssd-ipa-1.13.3-22.el6.x86_64 sssd-krb5-1.13.3-22.el6.x86_64 python-sssdconfig-1.13.3-22.el6.noarch sssd-qe-tests-sssd-rhel68-libs-mniranja-20161213172256-0.noarch sssd-ldap-1.13.3-22.el6.x86_64 sssd-1.13.3-22.el6.x86_64 sssd.conf ========= [sssd] config_file_version = 2 services = nss, pam domains = LDAP [nss] filter_groups = root filter_users = root [pam] [domain/LDAP] debug_level=0xFFF0 id_provider = ldap ldap_uri = ldap://vm-idm-011.lab.eng.pnq.redhat.com ldap_search_base = dc=example,dc=com ldap_tls_cacert = /etc/openldap/certs/cacert.asc Ldap server: OS: RHEL7.3 Red Hat Directory Server [root@auto-hv-02-guest03 mniranja]# py.test --with-beakerlib local_overrides.py -s -v ============================================================================================================ test session starts ============================================================================================================= platform linux2 -- Python 2.7.5, pytest-3.0.5, py-1.4.31, pluggy-0.4.0 -- /opt/rh/python27/root/usr/bin/python2 cachedir: .cache rootdir: /mnt/tests/sssd/rhel68/client/ldap_provider/local_overrides/mniranja, inifile: plugins: beakerlib-0.6 collected 12 items local_overrides.py::test_simple_user_override Starting sssd: [ OK ] SSSD needs to be restarted for the changes to take effect. Stopping sssd: [ OK ] Starting sssd: [ OK ] PASSEDcreate_sssd_fixture.teardown Stopping sssd: [ OK ] local_overrides.py::test_root_user_override Starting sssd: [ OK ] SSSD needs to be restarted for the changes to take effect. Stopping sssd: [ OK ] Starting sssd: [ OK ] PASSEDcreate_sssd_fixture.teardown Stopping sssd: [ OK ] local_overrides.py::test_replace_user_override Starting sssd: [ OK ] SSSD needs to be restarted for the changes to take effect. Stopping sssd: [ OK ] Starting sssd: [ OK ] Stopping sssd: [ OK ] Starting sssd: [ OK ] PASSEDcreate_sssd_fixture.teardown Stopping sssd: [ OK ] local_overrides.py::test_remove_user_override Starting sssd: [ OK ] SSSD needs to be restarted for the changes to take effect. Stopping sssd: [ OK ] Starting sssd: [ OK ] PASSEDcreate_sssd_fixture.teardown Stopping sssd: [ OK ] local_overrides.py::test_imp_exp_user_override Starting sssd: [ OK ] SSSD needs to be restarted for the changes to take effect. Stopping sssd: [ OK ] Starting sssd: [ OK ] user1@LDAP:ov_user1:10010:20010:Overridden User 1:/home/ov/user1:/bin/ov_user1_shell user2@LDAP:ov_user2:10020:20020:Overridden User 2:/home/ov/user2:/bin/ov_user2_shell Stopping sssd: [ OK ] Starting sssd: [ OK ] PASSEDcreate_sssd_fixture.teardown Stopping sssd: [ OK ] local_overrides.py::test_simple_group_override Starting sssd: [ OK ] SSSD needs to be restarted for the changes to take effect. Stopping sssd: [ OK ] Starting sssd: [ OK ] PASSEDcreate_sssd_fixture.teardown Stopping sssd: [ OK ] local_overrides.py::test_root_group_override Starting sssd: [ OK ] SSSD needs to be restarted for the changes to take effect. Stopping sssd: [ OK ] Starting sssd: [ OK ] PASSEDcreate_sssd_fixture.teardown Stopping sssd: [ OK ] local_overrides.py::test_replace_group_override Starting sssd: [ OK ] SSSD needs to be restarted for the changes to take effect. Stopping sssd: [ OK ] Starting sssd: [ OK ] Stopping sssd: [ OK ] Starting sssd: [ OK ] PASSEDcreate_sssd_fixture.teardown Stopping sssd: [ OK ] local_overrides.py::test_remove_group_override Starting sssd: [ OK ] SSSD needs to be restarted for the changes to take effect. Stopping sssd: [ OK ] Starting sssd: [ OK ] PASSEDcreate_sssd_fixture.teardown Stopping sssd: [ OK ] local_overrides.py::test_imp_exp_group_override Starting sssd: [ OK ] SSSD needs to be restarted for the changes to take effect. Stopping sssd: [ OK ] Starting sssd: [ OK ] Stopping sssd: [ OK ] Starting sssd: [ OK ] PASSEDcreate_sssd_fixture.teardown Stopping sssd: [ OK ] local_overrides.py::test_regr_2757_override Starting sssd: [ OK ] SSSD needs to be restarted for the changes to take effect. Stopping sssd: [ OK ] Starting sssd: [ OK ] PASSEDcreate_sssd_fixture.teardown Stopping sssd: [ OK ] local_overrides.py::test_regr_2790_override Starting sssd: [ OK ] SSSD needs to be restarted for the changes to take effect. Stopping sssd: [ OK ] Starting sssd: [ OK ] FAILEDcreate_sssd_fixture.teardown Stopping sssd: [ OK ] sssd crashed Process /usr/libexec/sssd/sssd_nss was killed by signal 11 (SIGSEGV) dmesg: sssd_nss[3595] general protection ip:39d021219f sp:7ffd83642260 error:0 in libldb.so.1.1.25[39d0200000+2d000] Update sssd to latest sssd-ldap-1.13.3-52.el6.x86_64 sssd-krb5-common-1.13.3-52.el6.x86_64 sssd-ad-1.13.3-52.el6.x86_64 sssd-krb5-1.13.3-52.el6.x86_64 sssd-proxy-1.13.3-52.el6.x86_64 sssd-ipa-1.13.3-52.el6.x86_64 sssd-tools-1.13.3-52.el6.x86_64 python-sssdconfig-1.13.3-52.el6.noarch sssd-common-1.13.3-52.el6.x86_64 sssd-common-pac-1.13.3-52.el6.x86_64 sssd-1.13.3-52.el6.x86_64 sssd-client-1.13.3-52.el6.x86_64 [root@auto-hv-02-guest03 mniranja]# py.test --with-beakerlib local_overrides.py -s -v =================================================================== test session starts =========================================== platform linux2 -- Python 2.7.5, pytest-3.0.5, py-1.4.31, pluggy-0.4.0 -- /opt/rh/python27/root/usr/bin/python2 cachedir: .cache rootdir: /mnt/tests/sssd/rhel68/client/ldap_provider/local_overrides/mniranja, inifile: plugins: beakerlib-0.6 collected 12 items local_overrides.py::test_simple_user_override Starting sssd: [ OK ] SSSD needs to be restarted for the changes to take effect. Stopping sssd: [ OK ] Starting sssd: [ OK ] PASSEDcreate_sssd_fixture.teardown Stopping sssd: [ OK ] local_overrides.py::test_root_user_override Starting sssd: [ OK ] SSSD needs to be restarted for the changes to take effect. Stopping sssd: [ OK ] Starting sssd: [ OK ] PASSEDcreate_sssd_fixture.teardown Stopping sssd: [ OK ] local_overrides.py::test_replace_user_override Starting sssd: [ OK ] SSSD needs to be restarted for the changes to take effect. Stopping sssd: [ OK ] Starting sssd: [ OK ] Stopping sssd: [ OK ] Starting sssd: [ OK ] PASSEDcreate_sssd_fixture.teardown Stopping sssd: [ OK ] local_overrides.py::test_remove_user_override Starting sssd: [ OK ] SSSD needs to be restarted for the changes to take effect. Stopping sssd: [ OK ] Starting sssd: [ OK ] PASSEDcreate_sssd_fixture.teardown Stopping sssd: [ OK ] local_overrides.py::test_imp_exp_user_override Starting sssd: [ OK ] SSSD needs to be restarted for the changes to take effect. Stopping sssd: [ OK ] Starting sssd: [ OK ] user1@LDAP:ov_user1:10010:20010:Overridden User 1:/home/ov/user1:/bin/ov_user1_shell user2@LDAP:ov_user2:10020:20020:Overridden User 2:/home/ov/user2:/bin/ov_user2_shell Stopping sssd: [ OK ] Starting sssd: [ OK ] PASSEDcreate_sssd_fixture.teardown Stopping sssd: [ OK ] local_overrides.py::test_simple_group_override Starting sssd: [ OK ] SSSD needs to be restarted for the changes to take effect. Stopping sssd: [ OK ] Starting sssd: [ OK ] PASSEDcreate_sssd_fixture.teardown Stopping sssd: [ OK ] local_overrides.py::test_root_group_override Starting sssd: [ OK ] SSSD needs to be restarted for the changes to take effect. Stopping sssd: [ OK ] Starting sssd: [ OK ] PASSEDcreate_sssd_fixture.teardown Stopping sssd: [ OK ] local_overrides.py::test_replace_group_override Starting sssd: [ OK ] SSSD needs to be restarted for the changes to take effect. Stopping sssd: [ OK ] Starting sssd: [ OK ] Stopping sssd: [ OK ] Starting sssd: [ OK ] PASSEDcreate_sssd_fixture.teardown Stopping sssd: [ OK ] local_overrides.py::test_remove_group_override Starting sssd: [ OK ] SSSD needs to be restarted for the changes to take effect. Stopping sssd: [ OK ] Starting sssd: [ OK ] PASSEDcreate_sssd_fixture.teardown Stopping sssd: [ OK ] local_overrides.py::test_imp_exp_group_override Starting sssd: [ OK ] SSSD needs to be restarted for the changes to take effect. Stopping sssd: [ OK ] Starting sssd: [ OK ] Stopping sssd: [ OK ] Starting sssd: [ OK ] PASSEDcreate_sssd_fixture.teardown Stopping sssd: [ OK ] local_overrides.py::test_regr_2757_override Starting sssd: [ OK ] SSSD needs to be restarted for the changes to take effect. Stopping sssd: [ OK ] Starting sssd: [ OK ] PASSEDcreate_sssd_fixture.teardown Stopping sssd: [ OK ] local_overrides.py::test_regr_2790_override Starting sssd: [ OK ] SSSD needs to be restarted for the changes to take effect. Stopping sssd: [ OK ] Starting sssd: [ OK ] PASSEDcreate_sssd_fixture.teardown Stopping sssd: [ OK ] ================================ 12 passed in 115.10 seconds ====================== [root@auto-hv-02-guest03 mniranja]#
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHBA-2017-0632.html