1807933 – sssd failover leads to delayed and failed logins [rhel-7.6.z]

RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1807933 - sssd failover leads to delayed and failed logins [rhel-7.6.z]

Summary: sssd failover leads to delayed and failed logins [rhel-7.6.z]

Keywords:
Status:	CLOSED ERRATA
Alias:	None
Product:	Red Hat Enterprise Linux 7
Classification:	Red Hat
Component:	sssd
Sub Component:
Version:	7.8
Hardware:	Unspecified
OS:	Unspecified
Priority:	urgent
Severity:	urgent
Target Milestone:	rc
Target Release:	---
Assignee:	SSSD Maintainers
QA Contact:	ipa-qe
Docs Contact:
URL:
Whiteboard:
Depends On:	1769755
Blocks:
TreeView+	depends on / blocked

Reported:	2020-02-27 14:12 UTC by RAD team bot copy to z-stream
Modified:	2023-09-07 22:06 UTC (History)
CC List:	16 users (show)
Fixed In Version:	sssd-1.16.2-13.el7_6.9
Doc Type:	If docs needed, set a value
Doc Text:
Clone Of:	1769755
Environment:
Last Closed:	2020-04-14 17:40:07 UTC
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)

Links
System	ID	Private	Priority	Status	Summary	Last Updated
Github	SSSD sssd issues 5075	0	None	closed	sssd failover leads to delayed and failed logins	2020-07-09 16:53:11 UTC
Red Hat Product Errata	RHBA-2020:1463	0	None	None	None	2020-04-14 17:40:11 UTC

Description RAD team bot copy to z-stream 2020-02-27 14:12:15 UTC

This bug has been copied from bug #1769755 and has been proposed to be backported to 7.6 z-stream (EUS).

Comment 5 Nikhil Dehadrai 2020-03-04 09:53:44 UTC

# rpm -q ipa-server ipa-client sssd
ipa-server-4.6.4-10.el7_6.6.x86_64
ipa-client-4.6.4-10.el7_6.6.x86_64
sssd-1.16.2-13.el7_6.9.x86_64


Verified the bug on the basis of following steps/observations:
1. Setup IPA master at RHEL76z
2. Setup IPA Replica at RHEL76z
3. Setup IPA client at RHEL76z (Ensuring that resolv.conf has entries for both MASTER and REPLICA)
4. Alternately Start / Stop Master and Replica and check if kinit works on client machine


Script used:
while true; do
date
echo --------------------
echo MASTER OFF
ssh -t root.test "ipactl status"
ssh -t root.test "ipactl stop"
ssh -t root.test "ipactl status"
echo REPLICA ON
ssh -t root.test "ipactl restart"
ssh -t root.test "ipactl status"
systemctl stop sssd; rm -rf /var/lib/sss/db/*; systemctl start sssd
kdestroy
klist
echo Secret123 | kinit admin
klist
getent passwd admin
echo ===============================================
date
echo --------------------
echo MASTER ON
ssh -t root.test  "ipactl restart"
ssh -t root.test "ipactl status"
echo REPLICA OFF
ssh -t root.test "ipactl status"
ssh -t root.test "ipactl stop"
ssh -t root.test "ipactl status"
systemctl stop sssd; rm -rf /var/lib/sss/db/*; systemctl start sssd
kdestroy
klist
echo Secret123 | kinit admin
klist
getent passwd admin
echo ===============================================
done




Ran the above script continuously for 10mins and the kinit was successful with FAILOVER from Master to REPLICA and Vice-Versa.
Observations:

--------------------
+ echo MASTER OFF and REPLICA ON
MASTER OFF and REPLICA ON
+ ssh -t root.test 'ipactl status'
Directory Service: RUNNING
krb5kdc Service: RUNNING
kadmin Service: RUNNING
named Service: RUNNING
httpd Service: RUNNING
ipa-custodia Service: RUNNING
pki-tomcatd Service: RUNNING
ipa-otpd Service: RUNNING
ipa-dnskeysyncd Service: RUNNING
ipa: INFO: The ipactl command was successful
Connection to master.ipapnq.test closed.
+ ssh -t root.test 'ipactl stop'
Stopping ipa-dnskeysyncd Service
Stopping ipa-otpd Service
Stopping pki-tomcatd Service
Stopping ipa-custodia Service
Stopping httpd Service
Stopping named Service
Stopping kadmin Service
Stopping krb5kdc Service
Stopping Directory Service
ipa: INFO: The ipactl command was successful
Connection to master.ipapnq.test closed.
+ ssh -t root.test 'ipactl status'
Directory Service: STOPPED
Directory Service must be running in order to obtain status of other services
ipa: INFO: The ipactl command was successful
Connection to master.ipapnq.test closed.
+ ssh -t root.test 'ipactl restart'
Restarting Directory Service
Restarting krb5kdc Service
Restarting kadmin Service
Restarting named Service
Restarting httpd Service
Restarting ipa-custodia Service
Restarting ntpd Service
Restarting ipa-otpd Service
Restarting ipa-dnskeysyncd Service
Starting pki-tomcatd Service
ipa: INFO: The ipactl command was successful
Connection to replica1.ipapnq.test closed.
+ ssh -t root.test 'ipactl status'
Directory Service: RUNNING
krb5kdc Service: RUNNING
kadmin Service: RUNNING
named Service: RUNNING
httpd Service: RUNNING
ipa-custodia Service: RUNNING
ntpd Service: RUNNING
pki-tomcatd Service: RUNNING
ipa-otpd Service: RUNNING
ipa-dnskeysyncd Service: RUNNING
ipa: INFO: The ipactl command was successful
Connection to replica1.ipapnq.test closed.
+ systemctl stop sssd
+ rm -rf /var/lib/sss/db/cache_ipapnq.test.ldb /var/lib/sss/db/ccache_IPAPNQ.TEST /var/lib/sss/db/config.ldb /var/lib/sss/db/sssd.ldb /var/lib/sss/db/timestamps_ipapnq.test.ldb
+ systemctl start sssd
+ kdestroy
+ klist
klist: Credentials cache keyring 'persistent:0:0' not found
+ echo Secret123
+ kinit admin
Password for admin:
+ klist
Ticket cache: KEYRING:persistent:0:0
Default principal: admin

Valid starting     Expires            Service principal
03/04/20 04:48:02  03/05/20 04:48:02  krbtgt/IPAPNQ.TEST
+ getent passwd admin
admin:*:64800000:64800000:Administrator:/home/admin:/bin/bash
+ echo ===============================================
===============================================
+ date
Wed Mar  4 04:48:02 EST 2020
+ echo --------------------
--------------------
+ echo MASTER ON and REPLICA OFF
MASTER ON and REPLICA OFF
+ ssh -t root.test 'ipactl restart'
Starting Directory Service
Starting krb5kdc Service
Starting kadmin Service
Starting named Service
Starting httpd Service
Starting ipa-custodia Service
Starting pki-tomcatd Service
Starting ipa-otpd Service
Starting ipa-dnskeysyncd Service
ipa: INFO: The ipactl command was successful
Connection to master.ipapnq.test closed.
+ ssh -t root.test 'ipactl status'
Directory Service: RUNNING
krb5kdc Service: RUNNING
kadmin Service: RUNNING
named Service: RUNNING
httpd Service: RUNNING
ipa-custodia Service: RUNNING
pki-tomcatd Service: RUNNING
ipa-otpd Service: RUNNING
ipa-dnskeysyncd Service: RUNNING
ipa: INFO: The ipactl command was successful
Connection to master.ipapnq.test closed.
+ ssh -t root.test 'ipactl status'
Directory Service: RUNNING
krb5kdc Service: RUNNING
kadmin Service: RUNNING
named Service: RUNNING
httpd Service: RUNNING
ipa-custodia Service: RUNNING
ntpd Service: RUNNING
pki-tomcatd Service: RUNNING
ipa-otpd Service: RUNNING
ipa-dnskeysyncd Service: RUNNING
ipa: INFO: The ipactl command was successful
Connection to replica1.ipapnq.test closed.
+ ssh -t root.test 'ipactl stop'
Stopping ipa-dnskeysyncd Service
Stopping ipa-otpd Service
Stopping pki-tomcatd Service
Stopping ntpd Service
Stopping ipa-custodia Service
Stopping httpd Service
Stopping named Service
Stopping kadmin Service
Stopping krb5kdc Service
Stopping Directory Service
ipa: INFO: The ipactl command was successful
Connection to replica1.ipapnq.test closed.
+ ssh -t root.test 'ipactl status'
Directory Service: STOPPED
Directory Service must be running in order to obtain status of other services
ipa: INFO: The ipactl command was successful
Connection to replica1.ipapnq.test closed.
+ systemctl stop sssd
+ rm -rf /var/lib/sss/db/cache_ipapnq.test.ldb /var/lib/sss/db/ccache_IPAPNQ.TEST /var/lib/sss/db/config.ldb /var/lib/sss/db/sssd.ldb /var/lib/sss/db/timestamps_ipapnq.test.ldb
+ systemctl start sssd
+ kdestroy
+ klist
klist: Credentials cache keyring 'persistent:0:0' not found
+ echo Secret123
+ kinit admin
Password for admin:
+ klist
Ticket cache: KEYRING:persistent:0:0
Default principal: admin

Valid starting     Expires            Service principal
03/04/20 04:48:44  03/05/20 04:48:44  krbtgt/IPAPNQ.TEST
+ getent passwd admin
admin:*:64800000:64800000:Administrator:/home/admin:/bin/bash

Thus on the basis of above observations, marking the status of bug to "VERIFIED"

Comment 8 errata-xmlrpc 2020-04-14 17:40:07 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:1463

Note You need to log in before you can comment on or make changes to this bug.