Bug 1460689

Summary:	KCM/secrets: Storing many secrets in a rapid succession segfaults the secrets responder
Product:	Red Hat Enterprise Linux 7	Reporter:	Jakub Hrozek <jhrozek>
Component:	sssd	Assignee:	SSSD Maintainers <sssd-maint>
Status:	CLOSED ERRATA	QA Contact:	Amith <apeetham>
Severity:	unspecified	Docs Contact:	Aneta Šteflová Petrová <apetrova>
Priority:	unspecified
Version:	7.3	CC:	apeetham, apetrova, fidencio, grajaiya, jhrozek, lslebodn, mkosek, mzidek, pasik, pbrezina, sgoveas, tscherf
Target Milestone:	rc
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	sssd-1.16.0-1.el7	Doc Type:	Known Issue
Doc Text:	The sssd-secrets component crashes when it is under load When the sssd-secrets component receives many requests, the situation triggers a bug in the Network Security Services (NSS) library that causes sssd-secrets to terminate unexpectedly. However, the systemd service restarts sssd-secrets for the next request, which means that the denial of service is only temporary.	Story Points:	---
Clone Of:		Environment:
Last Closed:	2018-04-10 17:11:33 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Jakub Hrozek 2017-06-12 12:15:58 UTC

This bug is created as a clone of upstream ticket:
https://pagure.io/SSSD/sssd/issue/3424

Currently the NSS library is fully initialized and torn down on each secrets storage. Eventually, this triggers a segfault with a backtrace pointing to NSS.

This is probably a NSS bug, but we're not making it any easier on us by calling setup and teardown of NSS on each encrypt. We should just remove the cleanup totally or clean up on responder shutdown.

To reproduce:
1. configure sssd+kcm
2. kinit repeatedly, after several thousand tries, sssd-secrets segfaults

Comment 5 Lukas Slebodnik 2017-07-25 09:54:09 UTC

master:
* a6f606117e5cfe64c4b49f94e514bf82054716d3

Comment 6 Jakub Hrozek 2017-08-10 18:26:51 UTC

To reproduce and verify: see comment #0.

Comment 8 Amith 2018-01-29 06:33:59 UTC

Verified the bug on SSSD Version : sssd-1.16.0-14.el7.x86_64

Steps followed during verification:
1. Reproduce the issue with older sssd build : sssd-1.15.2-50.el7.x86_64.

2. Configure KCM + Secrets in SSSD.CONF

[sssd]
config_file_version = 2
sbus_timeout = 30
services = nss, pam
domains = LDAP-KRB5

[nss]
debug_level = 0xFFF0

[pam]

[secrets]
debug_level = 0xFFF0
provider = local

[domain/LDAP-KRB5]
debug_level=0xFFF0
id_provider = ldap
ldap_uri = ldap://SERVER
ldap_search_base = dc=example,dc=com
auth_provider = krb5
krb5_server = KRB-SERVER
krb5_realm = EXAMPLE.COM

3. Create a for loop which executes kinit for over 200 times. Login as test user and execute the following script.

for i in {1..100}; do
echo "Loop $i"
echo Secret123 | kinit
done

4. In the case of old sssd build sssd-1.15.2-50.el7.x86_64, you will see following error around loop 69:

Loop 69
Password for user_ssftest1: 
kinit: Failed to store credentials: Internal credentials cache error while getting initial credentials

5. Monitor the "sssd-secrets.service" status. You will see that the process for sssd_secrets restarts and Main PID changes.

6. In the case of latest sssd build on RHEL-7.5, the test remains consistent without any error. The sssd_secrets process looks stable.

Comment 11 errata-xmlrpc 2018-04-10 17:11:33 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2018:0929