RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Description of problem:
 
Delay issues on pam_sss.so.
 "Postgresql is calling pam_sss.so via pam stack. Sometimes, response time is longer than 3s. It happened randomly. But happened on both RHEL5 and RHEL6. On both OS, we are using service record for load balancing.

Verified in logs on idm server side, server returns result in subsecond.
Something happened inside pam_sss.so that delayed response after received result from idm server"

Version-Release number of selected component (if applicable):
RHEL5 and RHEL6

How reproducible:
Yes
- It appears it is reproducible on the customer side.  On the Red Hat side, per comment #31 in the case (Justin Stephenson  (11/17/2015 2:29 PM))

Steps to Reproduce:
Unsure

Actual results:
Unsure

Expected results:
Unsure

Additional info:
We're attempting to debug pam_sss callouts at a granular level. Something like this, it could benefit to have something like a stap script in place. But that's likely the next step is to look at the callouts coming from the pam libraries and seeing where the delays are counted at.

Could you provide log files from sssd? 
We would need to increase debug_level in domain and pam section.
https://fedorahosted.org/sssd/wiki/Troubleshooting#SSSDdebuglogs

Could you also provide log file /var/log/secure?

You might also use tips for trubleshooting authentication.
https://fedorahosted.org/sssd/wiki/Troubleshooting#TroubleshootingAuthenticationPasswordChangeandAccessControl

When you attach those logs, please also make sure they are from a RHEL-6 machine because a) this is a performance issue and in RHEL-5 we no longer fix those and b) RHEL-6 would have Kerberos tracing info in krb5_child.logs.

Created attachment 1103387 [details]
krb5kdc log