Bug 1079237 - sssd is crashing after several quick invokes of automount -m
Summary: sssd is crashing after several quick invokes of automount -m
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: sssd
Version: 20
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Jakub Hrozek
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-03-21 09:23 UTC by habicht
Modified: 2020-05-02 17:40 UTC (History)
7 users (show)

Fixed In Version: sssd-1.11.5.1-1.fc20
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2014-04-24 07:36:19 UTC
Type: Bug


Attachments (Terms of Use)
core generated with crashing automount/sssd (1.39 MB, application/x-core)
2014-03-21 10:02 UTC, habicht
no flags Details
sssd log files (5.42 MB, application/x-tar)
2014-03-21 13:34 UTC, habicht
no flags Details
core files from abrtd (575.06 KB, application/x-gzip)
2014-03-21 14:43 UTC, habicht
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Github SSSD sssd issues 3330 0 None closed sssd is crashing after several quick invokes of automount -m 2020-05-11 08:10:13 UTC

Description habicht 2014-03-21 09:23:36 UTC
Description of problem: 

I have a strange problem with sssd, ldap and autofs. I am using sssd and ldap for authentication and automount. Our fileserver has 4 shares, while the 4. is a new one. The mounting of the 4. share is not stabil and the systems have problems to see the possible mounts. While still investigating this problem i found a strange effect now describing here:

When i involve automount -m the output is sometimes hanging at this strange share. But not everytime. When i re-involve automount -m very quick and very often and it hangs again and than make several CTRL-C, than sssd is crashed.


Version-Release number of selected component (if applicable):
sssd-common-1.11.4-1.fc20.x86_64
sssd-client-1.11.4-1.fc20.i686
sssd-1.11.4-1.fc20.x86_64
sssd-ldap-1.11.4-1.fc20.x86_64
sssd-client-1.11.4-1.fc20.x86_64
autofs-5.0.7-40.fc20.x86_64

How reproducible:

See above


Steps to Reproduce:
1. See above
2.
3.

Actual results:

Making several CTRL-C at hanging automount -m output:

....
Mount point: /data

source(s):
Segmentation fault (core dumped)

And than:

[root@pxe-122 log]# systemctl status sssd
sssd.service - System Security Services Daemon
   Loaded: loaded (/usr/lib/systemd/system/sssd.service; enabled)
   Active: failed (Result: exit-code) since Fri 2014-03-21 09:28:09 CET; 5s ago
  Process: 488 ExecStart=/usr/sbin/sssd -D -f (code=exited, status=0/SUCCESS)
 Main PID: 563 (code=exited, status=1/FAILURE)




Expected results:

Normal output of automount -m . No crash of sssd.


Additional info:

Mar 21 09:27:18 pxe-122 kernel: [ 1571.044469] traps: sssd_be[1164] general protection ip:7fe295f1561a sp:7fffcbb365f8 error:0 in libc-2.18.so[7fe295e8f000+1b4000]
Mar 21 09:27:18 pxe-122 kernel: traps: sssd_be[1164] general protection ip:7fe295f1561a sp:7fffcbb365f8 error:0 in libc-2.18.so[7fe295e8f000+1b4000]
Mar 21 09:27:18 pxe-122 sssd[be[default]]: Starting up
Mar 21 09:27:19 pxe-122 automount[4745]: setautomntent: lookup(sss): setautomntent: No such file or directory
Mar 21 09:27:21 pxe-122 kernel: [ 1574.493951] traps: sssd_be[4751] general protection ip:7f8b58e2f61a sp:7fffd0bf44a8 error:0 in libc-2.18.so[7f8b58da9000+1b4000]
Mar 21 09:27:21 pxe-122 kernel: traps: sssd_be[4751] general protection ip:7f8b58e2f61a sp:7fffd0bf44a8 error:0 in libc-2.18.so[7f8b58da9000+1b4000]
Mar 21 09:27:23 pxe-122 sssd[be[default]]: Starting up
Mar 21 09:27:47 pxe-122 kernel: [ 1600.703958] traps: sssd_be[4764] general protection ip:7f567182761a sp:7fff371c6ed8 error:0 in libc-2.18.so[7f56717a1000+1b4000]
Mar 21 09:27:47 pxe-122 kernel: traps: sssd_be[4764] general protection ip:7f567182761a sp:7fff371c6ed8 error:0 in libc-2.18.so[7f56717a1000+1b4000]
Mar 21 09:27:51 pxe-122 sssd[be[default]]: Starting up
Mar 21 09:28:09 pxe-122 kernel: [ 1622.336917] traps: sssd_be[4834] general protection ip:7fcba70b861a sp:7fff04291f88 error:0 in libc-2.18.so[7fcba7032000+1b4000]
Mar 21 09:28:09 pxe-122 kernel: traps: sssd_be[4834] general protection ip:7fcba70b861a sp:7fff04291f88 error:0 in libc-2.18.so[7fcba7032000+1b4000]
Mar 21 09:28:09 pxe-122 kernel: [ 1622.338391] automount[4840]: segfault at 0 ip 00007fd58451791f sp 00007fffa34b86b0 error 4 in libsss_autofs.so[7fd584515000+4000]
Mar 21 09:28:09 pxe-122 sssd[autofs]: Shutting down
Mar 21 09:28:09 pxe-122 kernel: automount[4840]: segfault at 0 ip 00007fd58451791f sp 00007fffa34b86b0 error 4 in libsss_autofs.so[7fd584515000+4000]
Mar 21 09:28:09 pxe-122 sssd[pam]: Shutting down
Mar 21 09:28:09 pxe-122 sssd[nss]: Shutting down
Mar 21 09:28:09 pxe-122 systemd: sssd.service: main process exited, code=exited, status=1/FAILURE
Mar 21 09:28:09 pxe-122 systemd: Unit sssd.service entered failed state.

Comment 1 Lukas Slebodnik 2014-03-21 09:35:20 UTC
Could you provide coredump?

If you have installed abrt it should be in directory /var/tmp/abrt/ccpp-<date>

Comment 2 habicht 2014-03-21 10:02:15 UTC
Created attachment 877190 [details]
core generated with crashing automount/sssd

I have only this in the moment. But this is not from /var/tmp/abrt/. When you need a different one i can try to crash a system again.

Comment 3 Jakub Hrozek 2014-03-21 11:05:53 UTC
Upstream ticket:
https://fedorahosted.org/sssd/ticket/2288

Comment 4 Lukas Slebodnik 2014-03-21 11:59:23 UTC
I would say there are two different issues(crashes):

1. automount used sss_autofs plugin for retrieving maps
2. sss_autofs plugin sent request to sssd_be proccess.
3. sssd_be should download automounter maps from from ldap server.

Mar 21 09:28:09 pxe-122 kernel: traps: sssd_be[4834] general protection ip:7fcba70b861a sp:7fff04291f88 error:0 in libc-2.18.so[7fcba7032000+1b4000]
^^^^^^^
There should be coredump from sssd_be

4. sssd_be didn't replied to sss_autofs plugin.
5. sss_autofs plugin didn't handle failed requesst properly.

Mar 21 09:28:09 pxe-122 kernel: [ 1622.338391] automount[4840]: segfault at 0 ip 00007fd58451791f sp 00007fffa34b86b0 error 4 in libsss_autofs.so[7fd584515000+4000]
^^^^^^^^
This is a coredump you provided in attachment 877190 [details]

Could you also provide coredump from sssd_be?
The lod file from sssd_be process will be helpful as well.
You will need to add debug_level=10 into "domain" and "autofs" section and send log files from directory /var/log/sssd. You can send them directly to me.

Comment 5 habicht 2014-03-21 13:34:41 UTC
Created attachment 877271 [details]
sssd log files

This are sssd log files debug level 10.

I am not able to find a core for sssd_be. 
abrtd is running. Any idea how i can create a core?

Comment 6 habicht 2014-03-21 14:43:26 UTC
Created attachment 877321 [details]
core files from abrtd

core files from abrtd

Comment 7 Lukas Slebodnik 2014-03-21 15:53:23 UTC
Thank you very much for coredump and log files.

sssd_be crashed due to use after free error.

I can see in log files some LDAP entries have more then one attribute "automountKey"

[sdap_process_message] (0x4000): Message type: [LDAP_RES_SEARCH_ENTRY]
[sdap_parse_entry] (0x4000): OriginalDN: [cn=teamviewer9,nisMapName=auto.opt,ou=ea,dc=ims,dc=uni-hannover,dc=de].
[sdap_parse_range] (0x2000): No sub-attributes for [cn]
[sdap_parse_range] (0x2000): No sub-attributes for [nisMapEntry]
[sdap_parse_range] (0x2000): No sub-attributes for [objectClass]
[sdap_process_result] (0x2000): Trace: sh[0x7fdaf2bf4e10], connected[1], ops[0x7fdaf2c07360], ldap[0x7fdaf2bfcc20]
[sdap_process_result] (0x2000): Trace: ldap_result found nothing!
[sdap_process_result] (0x2000): Trace: sh[0x7fdaf2bf4e10], connected[1], ops[0x7fdaf2c07360], ldap[0x7fdaf2bfcc20]
[sdap_process_message] (0x4000): Message type: [LDAP_RES_SEARCH_RESULT]
[sdap_get_generic_ext_done] (0x0400): Search result: Success(0), no errmsg set
[automntmaps_process_members_done] (0x2000): No more search bases to try
[automntmaps_process_members_done] (0x0400): Search for autofs entries, returned 46 results.
[sdap_get_automntmap_done] (0x0400): automount map members received
[sdap_autofs_setautomntent_save] (0x1000): Got 46 map entries from LDAP
[get_autofs_entry_attr] (0x0020): Expected one entry got 2
[sdap_autofs_setautomntent_save] (0x0080): Malformed entry, skipping

If you fix following ldap entries in LDAP server sssd_autofs should work
cn=teamviewer9,nisMapName=auto.opt,ou=ea,dc=ims,dc=uni-hannover,dc=de
cn=envmodules,nisMapName=auto.appl,ou=ea,dc=ims,dc=uni-hannover,dc=de
cn=eod,nisMapName=auto.appl,ou=ea,dc=ims,dc=uni-hannover,dc=de

Thank you very much for report.

Comment 8 Lukas Slebodnik 2014-03-21 16:53:29 UTC
I prepared scratch build for case you don;t want (cannot) modify entries in LDAP server. SSSD will not crash if some entries contain more then one attribute "automountKey" (these entries will be skipped)

http://koji.fedoraproject.org/koji/taskinfo?taskID=6659466
and x86_64 packges
http://koji.fedoraproject.org/koji/taskinfo?taskID=6659470

Comment 9 Jakub Hrozek 2014-04-04 12:17:54 UTC
A fix for the sssd_be crash was pushed upstream in e18d5c703c529f99bd375da887d9aeaee6b82fcd

Comment 10 Fedora Update System 2014-04-08 12:23:39 UTC
sssd-1.11.5-1.fc19 has been submitted as an update for Fedora 19.
https://admin.fedoraproject.org/updates/sssd-1.11.5-1.fc19

Comment 11 Fedora Update System 2014-04-08 12:24:02 UTC
sssd-1.11.5-1.fc20 has been submitted as an update for Fedora 20.
https://admin.fedoraproject.org/updates/sssd-1.11.5-1.fc20

Comment 12 Fedora Update System 2014-04-09 13:16:01 UTC
Package sssd-1.11.5-1.fc19:
* should fix your issue,
* was pushed to the Fedora 19 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing sssd-1.11.5-1.fc19'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2014-4932/sssd-1.11.5-1.fc19
then log in and leave karma (feedback).

Comment 13 Fedora Update System 2014-04-11 18:21:32 UTC
sssd-1.11.5.1-1.fc20 has been submitted as an update for Fedora 20.
https://admin.fedoraproject.org/updates/sssd-1.11.5.1-1.fc20

Comment 14 Fedora Update System 2014-04-11 18:21:55 UTC
sssd-1.11.5.1-1.fc19 has been submitted as an update for Fedora 19.
https://admin.fedoraproject.org/updates/sssd-1.11.5.1-1.fc19

Comment 15 Fedora Update System 2014-04-24 07:36:19 UTC
sssd-1.11.5.1-1.fc19 has been pushed to the Fedora 19 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 16 Fedora Update System 2014-04-24 07:42:15 UTC
sssd-1.11.5.1-1.fc20 has been pushed to the Fedora 20 stable repository.  If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.