Bug 1285066 - pam_sss.so event causing delayed response after received result from idm server.
pam_sss.so event causing delayed response after received result from idm server.
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: sssd (Show other bugs)
x86_64 Linux
urgent Severity urgent
: rc
: ---
Assigned To: SSSD Maintainers
Namita Soman
Depends On:
Blocks: 1269194
  Show dependency treegraph
Reported: 2015-11-24 14:33 EST by jdang
Modified: 2016-08-10 08:21 EDT (History)
12 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2016-08-10 08:21:00 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
krb5kdc log (17.83 MB, application/x-gzip)
2015-12-07 17:42 EST, jstephen
no flags Details

  None (edit)
Description jdang 2015-11-24 14:33:08 EST
Description of problem:
Delay issues on pam_sss.so.
 "Postgresql is calling pam_sss.so via pam stack. Sometimes, response time is longer than 3s. It happened randomly. But happened on both RHEL5 and RHEL6. On both OS, we are using service record for load balancing.

Verified in logs on idm server side, server returns result in subsecond.
Something happened inside pam_sss.so that delayed response after received result from idm server"

Version-Release number of selected component (if applicable):

How reproducible:
- It appears it is reproducible on the customer side.  On the Red Hat side, per comment #31 in the case (Justin Stephenson  (11/17/2015 2:29 PM))

Steps to Reproduce:

Actual results:

Expected results:

Additional info:
We're attempting to debug pam_sss callouts at a granular level. Something like this, it could benefit to have something like a stap script in place. But that's likely the next step is to look at the callouts coming from the pam libraries and seeing where the delays are counted at.
Comment 2 Lukas Slebodnik 2015-11-25 03:45:43 EST
Could you provide log files from sssd? 
We would need to increase debug_level in domain and pam section.

Could you also provide log file /var/log/secure?

You might also use tips for trubleshooting authentication.
Comment 3 Jakub Hrozek 2015-11-25 03:52:53 EST
When you attach those logs, please also make sure they are from a RHEL-6 machine because a) this is a performance issue and in RHEL-5 we no longer fix those and b) RHEL-6 would have Kerberos tracing info in krb5_child.logs.
Comment 8 jstephen 2015-12-07 17:42 EST
Created attachment 1103387 [details]
krb5kdc log

Note You need to log in before you can comment on or make changes to this bug.