Bug 740501
| Summary: | SSSD not functional after "self" reboot | |||
|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 6 | Reporter: | Kemot1000 <kemot1000> | |
| Component: | sssd | Assignee: | Stephen Gallagher <sgallagh> | |
| Status: | CLOSED ERRATA | QA Contact: | IDM QE LIST <seceng-idm-qe-list> | |
| Severity: | high | Docs Contact: | ||
| Priority: | unspecified | |||
| Version: | 6.1 | CC: | grajaiya, jgalipea, jhrozek, jzeleny, kbanerje, prc | |
| Target Milestone: | rc | |||
| Target Release: | --- | |||
| Hardware: | Unspecified | |||
| OS: | Unspecified | |||
| Whiteboard: | ||||
| Fixed In Version: | sssd-1.5.1-53.el6 | Doc Type: | Bug Fix | |
| Doc Text: |
Cause: SSSD's NSS responder process uses an internal hash table. If SSSD backend is restarted and the NSS responder reconnects, the hash table is iterated over, but elements in it are not checked for initialization.
Consequence: NSS responder can crash after it is restarted due to access to already freed memory.
Fix: All elements from the hash table are copied first and iterated over afterwards.
Result: The crash in NSS responder caused by possible access to freed memory doesn't occur any more.
|
Story Points: | --- | |
| Clone Of: | ||||
| : | 748818 (view as bug list) | Environment: | ||
| Last Closed: | 2011-12-06 16:40:13 UTC | Type: | --- | |
| Regression: | --- | Mount Type: | --- | |
| Documentation: | --- | CRM: | ||
| Verified Versions: | Category: | --- | ||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
| Cloudforms Team: | --- | Target Upstream Version: | ||
| Embargoed: | ||||
| Bug Depends On: | ||||
| Bug Blocks: | 748818 | |||
|
Description
Kemot1000
2011-09-22 09:21:24 UTC
I suspect this is the same issue as https://fedorahosted.org/sssd/ticket/932 which we fixed in the master branch. Backporting this fix should be fairly simple. FWIW, I just tested that the upstream commit 0e27817133b931dcbe9d196e9ed0d737164ba613 applies cleanly on both upstream 1.5 branch and the RHEL 6.2 code. No backport needed, just a cherry-pick. I had this problem again. Do you have any update on when this patch will be available through RHN ? logs from /var/log/messages: Oct 11 10:05:01 MYSERVER ntpdate[6114]: adjust time server MYTIMESERVER offset -0.005464 sec Oct 11 10:07:28 MYSERVER sssd[be[MYDOMAIN]]: Shutting down Oct 11 10:07:28 MYSERVER sssd[be[MYDOMAIN]]: Starting up Oct 11 10:07:40 MYSERVER sssd[nss]: Starting up Oct 11 10:07:41 MYSERVER sssd[nss]: Starting up Oct 11 10:07:42 MYSERVER sssd[nss]: Starting up Oct 11 10:07:43 MYSERVER sssd[nss]: Starting up Oct 11 10:10:01 MYSERVER ntpdate[11952]: adjust time server MYTIMESERVER offset -0.041573 sec and after manual sssd restart: Oct 11 10:40:12 MYSERVER init: tty (/dev/tty1) main process ended, respawning Oct 11 10:40:32 MYSERVER sssd[be[MYDOMAIN]]: Shutting down Oct 11 10:40:32 MYSERVER sssd[pam]: Shutting down Oct 11 10:40:32 MYSERVER kernel: sssd_pam[1374]: segfault at ca92ac ip 0000000000ca92ac sp 00007fff58ded938 error 15 Oct 11 10:40:33 MYSERVER sssd: Starting up Oct 11 10:40:33 MYSERVER sssd[be[MYDOMAIN]]: Starting up Oct 11 10:40:33 MYSERVER sssd[nss]: Starting up Oct 11 10:40:33 MYSERVER sssd[pam]: Starting up Oct 11 10:41:44 MYSERVER init: tty (/dev/tty1) main process ended, respawning The fix Stephen referenced will be available when RHEL6.2 comes out. But the logs you posted indicate it might as well be a different issue (upstream #1038). Did you manage to get a corefile? Unfortunately no core file. I found this in: sssd.log (Tue Oct 11 10:07:21 2011) [sssd] [ping_check] (0): A service PING returned an error [org.freedesktop.DBus.Error.NoReply], closing connection. (Tue Oct 11 10:07:31 2011) [sssd] [global_checks_handler] (0): Unknown child (1369) did exit (Tue Oct 11 10:07:44 2011) [sssd] [svc_try_restart] (0): Process [nss], definitely stopped! (Tue Oct 11 10:40:32 2011) [sssd] [monitor_quit] (0): Monitor received Terminated: terminating children sssd_nss.log (Tue Oct 11 10:07:40 2011) [sssd[nss]] [sss_dp_init] (0): Failed to connect to monitor services. (Tue Oct 11 10:07:40 2011) [sssd[nss]] [sss_process_init] (0): fatal error setting up backend connector (Tue Oct 11 10:07:41 2011) [sssd[nss]] [sss_dp_init] (0): Failed to connect to monitor services. (Tue Oct 11 10:07:41 2011) [sssd[nss]] [sss_process_init] (0): fatal error setting up backend connector (Tue Oct 11 10:07:42 2011) [sssd[nss]] [sss_dp_init] (0): Failed to connect to monitor services. (Tue Oct 11 10:07:42 2011) [sssd[nss]] [sss_process_init] (0): fatal error setting up backend connector (Tue Oct 11 10:07:43 2011) [sssd[nss]] [sss_dp_init] (0): Failed to connect to monitor services. (Tue Oct 11 10:07:43 2011) [sssd[nss]] [sss_process_init] (0): fatal error setting up backend connector sssd_pam.log (Tue Oct 11 10:07:44 2011) [sssd[pam]] [pam_dp_reconnect_init] (0): Could not reconnect to MYDOMAIN provider. (Tue Oct 11 10:08:14 2011) [sssd[pam]] [pam_dp_reconnect_init] (0): Could not reconnect to MYDOMAIN provider. .... (Tue Oct 11 10:39:45 2011) [sssd[pam]] [pam_dp_reconnect_init] (0): Could not reconnect to MYDOMAIN provider. (Tue Oct 11 10:40:15 2011) [sssd[pam]] [pam_dp_reconnect_init] (0): Could not reconnect to MYDOMAIN provider. You are right this looks like this bug: https://fedorahosted.org/sssd/ticket/1038 Can you please add clear steps to verify? This is how I'd just reproduced it with -53: 1) configure an SSSD domain with a very low timeout (timeout=1 worked for me). Start SSSD 2) trigger many logins with SSSD coming in rapidly. I used the "check_user" testcase that Kaushik is using. Wait until sssd stops responding. 3) shut down or restart SSSD Thanks!
Technical note added. If any revisions are required, please edit the "Technical Notes" field
accordingly. All revisions will be proofread by the Engineering Content Services team.
New Contents:
Cause: SSSD's NSS responder process uses an internal hash table. If SSSD backend is restarted and the NSS responder reconnects, the hash table is iterated over, but elements in it are not checked for initialization.
Consequence: NSS responder can crash after it is restarted due to access to already freed memory.
Fix: All elements from the hash table are copied first and iterated over afterwards.
Result: The crash in NSS responder caused by possible access to freed memory doesn't occur any more.
Verified with timeout = 1 that sssd doesn't crash after the backend is restarted. Enumeration and auth works even after the backend is restarted. Verified in version: # rpm -qi sssd | head Name : sssd Relocations: (not relocatable) Version : 1.5.1 Vendor: Red Hat, Inc. Release : 66.el6 Build Date: Tue 01 Nov 2011 02:05:40 AM IST Install Date: Tue 01 Nov 2011 02:37:31 PM IST Build Host: x86-003.build.bos.redhat.com Group : Applications/System Source RPM: sssd-1.5.1-66.el6.src.rpm Size : 3628521 License: GPLv3+ Signature : (none) Packager : Red Hat, Inc. <http://bugzilla.redhat.com/bugzilla> URL : http://fedorahosted.org/sssd/ Summary : System Security Services Daemon Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. http://rhn.redhat.com/errata/RHBA-2011-1529.html |