Bug 979047

Summary:	sssd_be goes to 99% CPU and causes significant login delays when client is under load
Product:	Red Hat Enterprise Linux 5	Reporter:	Dmitri Pal <dpal>
Component:	sssd	Assignee:	Jakub Hrozek <jhrozek>
Status:	CLOSED ERRATA	QA Contact:	Kaushik Banerjee <kbanerje>
Severity:	unspecified	Docs Contact:
Priority:	medium
Version:	5.10	CC:	grajaiya, jgalipea, lslebodn, nsoman, okos, pbrezina
Target Milestone:	rc
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	sssd-1.5.1-70.el5	Doc Type:	Bug Fix
Doc Text:	Cause: While performing access control in the IPA backend, the SSSD errorneously downloaded the "member" attribute from the server and then attempted to use it in the cache verbatim Consequence: The cache attempted to use the "member" attribute values as if they were pointing to the local cache which was quite CPU intensive. The users saw the CPU spiking up. Fix: We no longer download and process the member attribute when processing host groups Result: The login process is reasonably fast even with large host groups.	Story Points:	---
Clone Of:	979046	Environment:
Last Closed:	2013-09-30 22:46:09 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	979045, 979046
Bug Blocks:

Description Dmitri Pal 2013-06-27 13:36:39 UTC

+++ This bug was initially created as a clone of Bug #979046 +++

+++ This bug was initially created as a clone of Bug #979045 +++

This bug is created as a clone of upstream ticket:
https://fedorahosted.org/sssd/ticket/1806

I have a system with a reproducible problem with sssd when under load.

The sssd.log shows a reoccurring number of messages stating:  A service PING timed out on [domain.com]. Attempt [0]

Followed by: Killing service [expertcity.com], not responding to pings!

Following a restart of sssd, the sssd_be process spikes at 99% cpu, and a delay of 30-60secs can be experienced sshing to the device.  Subsequent logins seem fine until whichever cache is effected needs to be renewed again, which in turn reproduces the long delay.

The system is a VM with 2 cores assigned.  Load can be anywhere from 4-12 to reproduce the issue.

Comment 1 RHEL Program Management 2013-06-27 13:38:24 UTC

This request was evaluated by Red Hat Product Management for inclusion
in a Red Hat Enterprise Linux release.  Product Management has
requested further review of this request by Red Hat Engineering, for
potential inclusion in a Red Hat Enterprise Linux release for currently
deployed products.  This request is not yet committed for inclusion in
a release.

Comment 2 Jakub Hrozek 2013-06-27 14:51:54 UTC

Steps to reproduce:
https://bugzilla.redhat.com/show_bug.cgi?id=979045#c2

Comment 3 Jakub Hrozek 2013-06-27 14:52:34 UTC

Fixed upstream.

Comment 5 Namita Soman 2013-08-01 17:41:14 UTC

Tested using ipa-server-3.0.0-26.el6_4.4.x86_64, sssd-1.5.1-70.el5, ipa-client-2.1.3-7.el5

Added a host group - hostgroup1
Added 2000 hosts
Added these hosts to the hostgroup
Installed ipaclient, and added that host to same hostgroup
Added hbac rule, allowing user (user one) to access hosts in the hostgroup (hostgroup1), and allowing access to a service (sshd).
Disabled hbac rule allow_all 
Ran kdestroy
ssh'd as user (one) from master server to the host where the rhel 5.10 client is installed.

There was no cpu spikes or messages in sssd.log

Comment 7 errata-xmlrpc 2013-09-30 22:46:09 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHSA-2013-1319.html