Bug 1908711

Summary:	kinit command fails after upgrade with error "kinit: Credentials cache I/O operation failed while getting default ccache"
Product:	Red Hat Enterprise Linux 8	Reporter:	Nikhil Dehadrai <ndehadra>
Component:	sssd	Assignee:	Alexey Tikhonov <atikhono>
Status:	CLOSED ERRATA	QA Contact:	Madhuri <mupadhye>
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	8.4	CC:	dlavu, grajaiya, jhrozek, lslebodn, mzidek, pbrezina, sbose, sgoveas, ssidhaye, sssd-maint, tscherf
Target Milestone:	rc	Keywords:	Regression, TestBlocker, Triaged
Target Release:	8.0	Flags:	pm-rhel: mirror+
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:	sync-to-jira
Fixed In Version:	sssd-2.4.0-4.el8	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2021-05-18 15:04:21 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1903942

Description Nikhil Dehadrai 2020-12-17 12:11:14 UTC

Description of problem:
kinit command fails  after upgrade with error "kinit: Credentials cache I/O operation failed while getting default ccache", when upgraded from RHEL83z > 84(nightly)

Version-Release number of selected component (if applicable):
ipa-server-4.9.0-0.5.rc3.module+el8.4.0+9124+ced20601.x86_64

How reproducible:
Always

Steps to Reproduce:
1. Setup IPA server at RHEL83z
2. Check command # kinit admin
3. Use the nightly compose to initiate upgrade process (# dnf -y update)
4. Check upgrade #tail -1 /var/log/ipaupgrade.log
5. Check command # kinit admin


Actual results:
1. After step2, Kinit command works (before upgrade)
2. After step 4, upgrade is successful
3. After step5, Kinit command returns error

[root@master cloud-user]# rpm -q ipa-server
ipa-server-4.9.0-0.5.rc3.module+el8.4.0+9124+ced20601.x86_64
[root@master cloud-user]# kinit admin
kinit: Internal credentials cache error while searching for ccache for admin


Expected results:
kinit should work after IPA server is upgraded

Additional info:
1. Upon plain installation at RHEL84, the issue is not observed
[root@master ~]# echo Secret123 | kinit admin
Password for admin:
[root@master ~]#
 
2. The same issue isn't observed when upgraded to Test Compose
3. Using KRB5CCNAME=/tmp/foobar, passes the kinit commad

[root@master yum.repos.d]# kinit admin
kinit: Credentials cache I/O operation failed while getting default ccache
[root@master yum.repos.d]# echo $?
1
[root@master yum.repos.d]# KRB5CCNAME=/tmp/foobar kinit admin
Password for admin:
[root@master yum.repos.d]# echo $?
0

Comment 5 Sumit Bose 2020-12-17 12:38:28 UTC

Upstream:
 - 18b98836ef8e337992f0ecb239a32b9c3cedb750

Comment 8 Sumedh Sidhaye 2020-12-22 07:57:46 UTC

Before upgrade:

	2. You can now obtain a kerberos ticket using the command: 'kinit admin'
	   This ticket will allow you to use the IPA tools (e.g., ipa user-add)
	   and the web user interface.

Be sure to back up the CA certificates stored in /root/cacert.p12
These files are required to create replicas. The password for these
files is the Directory Manager password
The ipa-server-install command was successful
[root@ci-vm-10-0-139-248 ~]# kinit admin
Password for admin.UPSHIFT.RDU2.REDHAT.COM: 
[root@ci-vm-10-0-139-248 ~]# klist
Ticket cache: KCM:0
Default principal: admin.UPSHIFT.RDU2.REDHAT.COM

Valid starting       Expires              Service principal
2020-12-22T02:36:10  2020-12-23T02:36:07  krbtgt/CI-VM-10-0-139-248.HOSTED.UPSHIFT.RDU2.REDHAT.COM.UPSHIFT.RDU2.REDHAT.COM


After upgrade:

[root@ci-vm-10-0-139-248 ~]# rpm -q ipa-server sssd-common
ipa-server-4.9.0-0.5.rc3.module+el8.4.0+9124+ced20601.x86_64
sssd-common-2.4.0-4.el8.x86_64

[root@ci-vm-10-0-139-248 ~]# kinit admin
Password for admin.UPSHIFT.RDU2.REDHAT.COM: 
[root@ci-vm-10-0-139-248 ~]# klist
Ticket cache: KCM:0
Default principal: admin.UPSHIFT.RDU2.REDHAT.COM

Valid starting       Expires              Service principal
2020-12-22T02:49:27  2020-12-23T02:49:24  krbtgt/CI-VM-10-0-139-248.HOSTED.UPSHIFT.RDU2.REDHAT.COM.UPSHIFT.RDU2.REDHAT.COM

Comment 13 Sumedh Sidhaye 2021-01-14 09:46:34 UTC

Builds used for verification

Compose ID : RHEL-8.4.0-20210114.n.0
[root@ci-vm-10-0-139-155 ~]# rpm -q ipa-server sssd-common
ipa-server-4.9.0-1.module+el8.4.0+9274+259c83ee.x86_64
sssd-common-2.4.0-5.el8.x86_64
[root@ci-vm-10-0-139-155 ~]# 

Steps:

Before upgrade:

[root@ci-vm-10-0-139-155 ~]# rpm -q ipa-server sssd-common
ipa-server-4.8.7-12.module+el8.3.0+8222+c1bff54a.x86_64
sssd-common-2.3.0-9.el8.x86_64
[root@ci-vm-10-0-139-155 ~]#

The ipa-server-install command was successful
[root@ci-vm-10-0-139-155 ~]# 
[root@ci-vm-10-0-139-155 ~]# kinit admin
Password for admin.UPSHIFT.RDU2.REDHAT.COM: 
[root@ci-vm-10-0-139-155 ~]# 
[root@ci-vm-10-0-139-155 ~]# ipactl status
Directory Service: RUNNING
krb5kdc Service: RUNNING
kadmin Service: RUNNING
named Service: RUNNING
httpd Service: RUNNING
ipa-custodia Service: RUNNING
pki-tomcatd Service: RUNNING
ipa-otpd Service: RUNNING
ipa-dnskeysyncd Service: RUNNING
ipa: INFO: The ipactl command was successful
[root@ci-vm-10-0-139-155 ~]# 



After upgrade:

[root@ci-vm-10-0-139-155 ~]# tail -1 /var/log/ipaupgrade.log
2021-01-14T09:35:51Z INFO The ipa-server-upgrade command was successful
[root@ci-vm-10-0-139-155 ~]#

[root@ci-vm-10-0-139-155 ~]# rpm -q ipa-server sssd-common
ipa-server-4.9.0-1.module+el8.4.0+9274+259c83ee.x86_64
sssd-common-2.4.0-5.el8.x86_64
[root@ci-vm-10-0-139-155 ~]# 

[root@ci-vm-10-0-139-155 ~]# kinit admin
Password for admin.UPSHIFT.RDU2.REDHAT.COM: 
[root@ci-vm-10-0-139-155 ~]# klist
Ticket cache: KCM:0
Default principal: admin.UPSHIFT.RDU2.REDHAT.COM

Valid starting       Expires              Service principal
2021-01-14T04:42:44  2021-01-15T04:05:31  krbtgt/CI-VM-10-0-139-155.HOSTED.UPSHIFT.RDU2.REDHAT.COM.UPSHIFT.RDU2.REDHAT.COM
[root@ci-vm-10-0-139-155 ~]#

Comment 15 errata-xmlrpc 2021-05-18 15:04:21 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (sssd bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:1666