Bug 1254518
Summary: | Fix crash in nss responder | ||
---|---|---|---|
Product: | Red Hat Enterprise Linux 7 | Reporter: | Jakub Hrozek <jhrozek> |
Component: | sssd | Assignee: | Lukas Slebodnik <lslebodn> |
Status: | CLOSED ERRATA | QA Contact: | Kaushik Banerjee <kbanerje> |
Severity: | unspecified | Docs Contact: | |
Priority: | medium | ||
Version: | 7.0 | CC: | apeetham, grajaiya, jgalipea, jhrozek, lslebodn, mkosek, mzidek, nsoman, pbrezina, preichl |
Target Milestone: | rc | ||
Target Release: | --- | ||
Hardware: | Unspecified | ||
OS: | Unspecified | ||
Whiteboard: | |||
Fixed In Version: | sssd-1.13.0-24.el7 | Doc Type: | Bug Fix |
Doc Text: |
Cause: When two domains were configured and initgroups were run on a user from the second domain memory was double freed
Consequence: SSSD crashed
Fix: Memory deallocation is handled properly
Result: SSSD doesn't crash
|
Story Points: | --- |
Clone Of: | Environment: | ||
Last Closed: | 2015-11-19 11:40:15 UTC | Type: | --- |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: |
Description
Jakub Hrozek
2015-08-18 10:11:48 UTC
Patch acked upstream, acking for RHEL Pls add steps to verify (In reply to Namita Soman from comment #2) > Pls add steps to verify Configure two domains and run initgroups on a user from the second domain. Please note this is a use-after-free error, so it's possible sssd wouldn't crash each time, especially on a fast or idle machine. There is a mistake in Doc text. The statement "memory was double freed" is wrong. Double free and use after free are two different kind of bugs. The memory was accessed after releasing it. (In reply to Jakub Hrozek from comment #3) > (In reply to Namita Soman from comment #2) > > Pls add steps to verify > > Configure two domains and run initgroups on a user from the second domain. > > Please note this is a use-after-free error, so it's possible sssd wouldn't > crash each time, especially on a fast or idle machine. It's very likely it might crash in such setup. So it is not a deterministic reproducer. But you might use valgrind. Put following line to "[nss]" section of sssd command = valgrind -v --log-file=/var/log/sssd/valgrind_nss_%p.log /usr/libexec/sssd/sssd_nss --uid 0 --gid 0 --debug-to-files # there is a assumption that sssd_nss was spawn by sssd with the same arguments # It should be the same as output of `pgrep -af sssd_nss` In case of use after free you should be able to see an error "Invalid read of size" in log file /var/log/sssd/valgrind_nss_%p.log The similar to in commit message https://git.fedorahosted.org/cgit/sssd.git/commit/?id=b9901fe3d6cfe05cd75a2440c0f9c7985aea36c6 I followed the instructions on comment #6 and tried to verify the bug. The valgrind log file shows "Invalid read of size". Does that mean "use after free" error still exists and bug is not fixed? OR is it the other way around ? Also, encountered other issues like SSSD service fails to start upon adding the following to "[nss]" section: command = valgrind -v --log-file=/var/log/sssd/valgrind_nss_%p.log /usr/libexec/sssd/sssd_nss --uid 0 --gid 0 --debug-to-files SSSD service error: -------------------------------------------------- # service sssd status Redirecting to /bin/systemctl status sssd.service ● sssd.service - System Security Services Daemon Loaded: loaded (/usr/lib/systemd/system/sssd.service; disabled; vendor preset: disabled) Drop-In: /etc/systemd/system/sssd.service.d └─journal.conf Active: failed (Result: exit-code) since Wed 2015-09-16 06:13:58 EDT; 10s ago Process: 17987 ExecStart=/usr/sbin/sssd -D -f (code=exited, status=1/FAILURE) Main PID: 16897 (code=exited, status=0/SUCCESS) sssd[be[sssdad2012r2.com]][17989]: Starting up sssd[pam][17992]: Starting up sssd[17988]: Exiting the SSSD. Could not restart critical service [nss]. sssd[pam][17992]: Shutting down sssd[be[LDAP]][17990]: Shutting down sssd[be[sssdad2012r2.com]][17989]: Shutting down systemd[1]: sssd.service: control process exited, code=exited status=1 systemd[1]: Failed to start System Security Services Daemon. systemd[1]: Unit sssd.service entered failed state. systemd[1]: sssd.service failed. When SSSD was initiated from the command line, it works fine. # sssd -D # ps -ef | grep sss root 18028 1 0 06:23 ? 00:00:00 sssd -D root 18029 18028 0 06:23 ? 00:00:00 /usr/libexec/sssd/sssd_be --domain sssdad2012r2.com --uid 0 --gid 0 root 18030 18028 0 06:23 ? 00:00:00 /usr/libexec/sssd/sssd_be --domain LDAP --uid 0 --gid 0 root 18031 18028 0 06:23 ? 00:00:03 valgrind -v --log-file=/var/log/sssd/valgrind_nss_%p.log /usr/libexec/sssd/sssd_nss --uid 0 --gid 0 --debug-to-files root 18032 18028 0 06:23 ? 00:00:00 /usr/libexec/sssd/sssd_pam --uid 0 --gid 0 ------------------------------------------------------- The valgrind log file shows the error "Invalid read of size 4". # grep Invalid /var/log/sssd/valgrind_nss_18031.log ==18031== Invalid read of size 4 Contents of SSSD.CONF with multiple domains: ---------------------------------------------------- [sssd] domains = sssdad2012r2.com, LDAP config_file_version = 2 services = nss, pam [nss] command = valgrind -v --log-file=/var/log/sssd/valgrind_nss_%p.log /usr/libexec/sssd/sssd_nss --uid 0 --gid 0 --debug-to-files [domain/sssdad2012r2.com] ad_domain = sssdad2012r2.com krb5_realm = SSSDAD2012R2.COM realmd_tags = manages-system joined-with-adcli cache_credentials = True id_provider = ad krb5_store_password_if_offline = True default_shell = /bin/bash ldap_id_mapping = True use_fully_qualified_names = True fallback_homedir = /home/%u@%d access_provider = ad [domain/LDAP] id_provider = ldap auth_provider = ldap enumerate = true debug_level = 0xFFF0 cache_credentials = FALSE ldap_uri = ldaps://hubcap.lab.eng.pnq.redhat.com ldap_tls_cacert = /etc/openldap/certs/cacert.asc ldap_search_base = dc=example,dc=com ---------------------------------------------------- (In reply to Amith from comment #7) > I followed the instructions on comment #6 and tried to verify the bug. The > valgrind log file shows "Invalid read of size". Does that mean "use after > free" yes > error still exists and bug is not fixed? OR is it the other way around > ? > Which version of sssd did you test? The error should be gone with the latest sssd. > Also, encountered other issues like SSSD service fails to start upon adding > the following to "[nss]" section: > command = valgrind -v --log-file=/var/log/sssd/valgrind_nss_%p.log > /usr/libexec/sssd/sssd_nss --uid 0 --gid 0 --debug-to-files > I assume that valgrind was installed on that machine. So it might have been caused by SELinux policy. By default sssd is not allowed to execute code from memory; which is required by valgrind. #============= sssd_t ============== allow sssd_t self:process execmem; > SSSD service error: > -------------------------------------------------- > # service sssd status > Redirecting to /bin/systemctl status sssd.service > ● sssd.service - System Security Services Daemon > Loaded: loaded (/usr/lib/systemd/system/sssd.service; disabled; vendor > preset: disabled) > Drop-In: /etc/systemd/system/sssd.service.d > └─journal.conf > Active: failed (Result: exit-code) since Wed 2015-09-16 06:13:58 EDT; 10s > ago > Process: 17987 ExecStart=/usr/sbin/sssd -D -f (code=exited, > status=1/FAILURE) > Main PID: 16897 (code=exited, status=0/SUCCESS) > > sssd[be[sssdad2012r2.com]][17989]: Starting up > sssd[pam][17992]: Starting up > sssd[17988]: Exiting the SSSD. Could not restart critical service [nss]. > sssd[pam][17992]: Shutting down > sssd[be[LDAP]][17990]: Shutting down > sssd[be[sssdad2012r2.com]][17989]: Shutting down > systemd[1]: sssd.service: control process exited, code=exited status=1 > systemd[1]: Failed to start System Security Services Daemon. > systemd[1]: Unit sssd.service entered failed state. > systemd[1]: sssd.service failed. > > When SSSD was initiated from the command line, it works fine. > This might be explained by mislabeled files (in /run/... ). and errors caused by SELinux. You should either prepare own temporary SELinux policy or test in permissive mode. > # sssd -D > # ps -ef | grep sss > root 18028 1 0 06:23 ? 00:00:00 sssd -D > root 18029 18028 0 06:23 ? 00:00:00 /usr/libexec/sssd/sssd_be > --domain sssdad2012r2.com --uid 0 --gid 0 > root 18030 18028 0 06:23 ? 00:00:00 /usr/libexec/sssd/sssd_be > --domain LDAP --uid 0 --gid 0 > root 18031 18028 0 06:23 ? 00:00:03 valgrind -v > --log-file=/var/log/sssd/valgrind_nss_%p.log /usr/libexec/sssd/sssd_nss > --uid 0 --gid 0 --debug-to-files > root 18032 18028 0 06:23 ? 00:00:00 /usr/libexec/sssd/sssd_pam > --uid 0 --gid 0 > ------------------------------------------------------- Verified the bug with SSSD Version: sssd-1.13.0-26.el7.x86_64 Steps followed during verification: 1. Setup SSSD with multiple domains and add the following in nss section: command = valgrind -v --log-file=/var/log/sssd/valgrind_nss_%p.log /usr/libexec/sssd/sssd_nss --uid 0 --gid 0 --debug-to-files 2. Restart SSSD and run initgroups against user from first domain. # id testuser OR id --groups testuser # id --groups GROUP 3. Verify valgrind log file for the following error: ==29279== Invalid read of size 1 ==29279== at 0x4C2CBA2: strlen (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==29279== by 0x89A7AC4: talloc_strdup (in /usr/lib64/libtalloc.so.2.1.2) ==29279== by 0x11668A: nss_cmd_initgroups_search (nsssrv_cmd.c:4191) 4. Above mentioned errors were not discovered in valgrind log file. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHSA-2015-2355.html |