Bug 1489817

Summary: ipa-server-upgrade failes with "This entry already exists" [rhel-7.4.z]
Product: Red Hat Enterprise Linux 7 Reporter: Oneata Mircea Teodor <toneata>
Component: ipaAssignee: IPA Maintainers <ipa-maint>
Status: CLOSED ERRATA QA Contact: ipa-qe <ipa-qe>
Severity: high Docs Contact:
Priority: high    
Version: 7.4CC: abokovoy, fbarreto, frenaud, gparente, ipa-maint, ksiddiqu, lkimlick, ndehadra, pvoborni, pvomacka, rcritten, slaznick, tscherf
Target Milestone: rcKeywords: ZStream
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: ipa-4.5.0-21.el7.2 Doc Type: If docs needed, set a value
Doc Text:
Previously, when upgrading to Red Hat enterprise Linux 7.4, the upgrade of IdM server failed, if there was a duplicate entry containing the IdM CA certificate. Consequently, the upgrade script did not complete, and the IdM services did not restart. The upgrade script has been fixed to remove duplicate entries, and to use consistent names for IdM CA certificate. As a result, the upgrade is now successful.
Story Points: ---
Clone Of: 1480102 Environment:
Last Closed: 2017-10-19 15:12:49 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1480102    
Bug Blocks:    

Description Oneata Mircea Teodor 2017-09-08 12:16:56 UTC
This bug has been copied from bug #1480102 and has been proposed to be backported to 7.4 z-stream (EUS).

Comment 2 Petr Vobornik 2017-09-08 13:28:58 UTC
Fixed upstream
ipa-4-5:
https://pagure.io/freeipa/c/d9035a045bece8c9a205c078a8cdd2e1f101590b

Comment 5 Kaleem 2017-09-26 11:26:57 UTC
Changing the needinfo

Comment 9 Nikhil Dehadrai 2017-09-29 12:44:09 UTC
ipa server: ipa-server-4.5.0-21.el7_4.2.2.x86_64

Verified the bug in two PARTS:-
1) Upgrade
2) Plain-Installation

PART-ONE:
================
Following steps for UPGRADE SCENARIO's:

1) Setup IPA server (in my case RHEL 7.3.z, using self-sign certificate)
2) Run ldapsearch to check for certificate serial number / nickname details.
3) Modify the certificate nickname to "CA 1"
4) Re-run step2 to confirm changes implemented.
5) Add latest repo for RHEL 7.4 update1(Scenario-1) or 7.4 update2(Scenario-2), and run "yum -y update ' ipa*' sssd"
6) Run ldapsearch to check for certificate serial number / nickname details.
7) Run ipactl commands
8) Check if ipaupgrade was successful.
9) Check if UI login successful.



Scenario-1: (RHEL 7.3.z > RHEL 7.4 update1) - Upgrade-FAILS / UI accessible but login fails (Reproducer)
-----------------------------------------------------------------------------------------------------------
[root@auto-hv-02-guest07 ~]# rpm -q ipa-server
ipa-server-4.4.0-14.el7_3.7.x86_64
[root@auto-hv-02-guest07 ~]# ldapsearch  -xLLL -D "cn=directory manager" -W -b "cn=certificates,cn=ipa,cn=etc,dc=testrelm,dc=test" dn ipaCertIssuerSerial
Enter LDAP Password: 
dn: cn=certificates,cn=ipa,cn=etc,dc=testrelm,dc=test

dn: cn=TESTRELM.TEST IPA CA,cn=certificates,cn=ipa,cn=etc,dc=testrelm,dc=test
ipaCertIssuerSerial: CN=Certificate Authority,O=TESTRELM.TEST;1

[root@auto-hv-02-guest07 ~]# ldapmodify -D "cn=directory manager" -W << EOF
> dn: cn=TESTRELM.TEST IPA CA,cn=certificates,cn=ipa,cn=etc,dc=testrelm,dc=test
> changetype: modrdn
> newrdn: cn=CA 1
> deleteoldrdn: 1
> EOF
Enter LDAP Password: 
modifying rdn of entry "cn=TESTRELM.TEST IPA CA,cn=certificates,cn=ipa,cn=etc,dc=testrelm,dc=test"

[root@auto-hv-02-guest07 ~]# ldapsearch  -xLLL -D "cn=directory manager" -W -b "cn=certificates,cn=ipa,cn=etc,dc=testrelm,dc=test" dn ipaCertIssuerSerial
Enter LDAP Password: 
dn: cn=certificates,cn=ipa,cn=etc,dc=testrelm,dc=test

dn: cn=CA 1,cn=certificates,cn=ipa,cn=etc,dc=testrelm,dc=test
ipaCertIssuerSerial: CN=Certificate Authority,O=TESTRELM.TEST;1

[root@auto-hv-02-guest07 ~]# ADD REPO FOR RHEL 7.4.update1
[root@auto-hv-02-guest07 ~]# yum -y update 'ipa*' sssd
[root@auto-hv-02-guest07 ~]# rpm -q ipa-server
ipa-server-4.5.0-21.el7_4.1.2.x86_64
[root@auto-hv-02-guest07 ~]# tail -1 /var/log/ipaupgrade.log 
2017-09-26T10:14:42Z ERROR The ipa-server-upgrade command failed. See /var/log/ipaupgrade.log for more information
[root@auto-hv-02-guest07 ~]# ldapsearch  -xLLL -D "cn=directory manager" -W -b "cn=certificates,cn=ipa,cn=etc,dc=testrelm,dc=test" dn ipaCertIssuerSerial
Enter LDAP Password: 
ldap_sasl_bind(SIMPLE): Can't contact LDAP server (-1)
[root@auto-hv-02-guest07 ~]# ipactl status
Directory Service: STOPPED
Directory Service must be running in order to obtain status of other services
ipa: INFO: The ipactl command was successful
[root@auto-hv-02-guest07 ~]# tail -1 /var/log/ipaupgrade.log 
2017-09-26T10:14:42Z ERROR The ipa-server-upgrade command failed. See /var/log/ipaupgrade.log for more information
[root@auto-hv-02-guest07 ~]# tail -f /var/log/ipaupgrade.log 
2017-09-26T10:14:42Z DEBUG   duration: 0 seconds
2017-09-26T10:14:42Z ERROR IPA server upgrade failed: Inspect /var/log/ipaupgrade.log and run command ipa-server-upgrade manually.
2017-09-26T10:14:42Z DEBUG   File "/usr/lib/python2.7/site-packages/ipapython/admintool.py", line 172, in execute
    return_value = self.run()
  File "/usr/lib/python2.7/site-packages/ipaserver/install/ipa_server_upgrade.py", line 48, in run
    raise admintool.ScriptError(str(e))

2017-09-26T10:14:42Z DEBUG The ipa-server-upgrade command failed, exception: ScriptError: ('IPA upgrade failed.', 1)
2017-09-26T10:14:42Z ERROR ('IPA upgrade failed.', 1)
2017-09-26T10:14:42Z ERROR The ipa-server-upgrade command failed. See /var/log/ipaupgrade.log for more information
^C

[root@auto-hv-02-guest07 ~]# grep -rn "This entry already exists" /var/log/ipaupgrade.log 
33216:2017-09-26T10:14:41Z ERROR Upgrade failed with This entry already exists
33236:DuplicateEntry: This entry already exists
33245:RuntimeError: This entry already exists
33247:2017-09-26T10:14:41Z DEBUG   [error] RuntimeError: This entry already exists

[root@auto-hv-02-guest07 ~]# 
[root@auto-hv-02-guest07 ~]# ipactl restart -f
Skipping version check
Starting Directory Service
Starting krb5kdc Service
Starting kadmin Service
Starting named Service
Starting httpd Service
Starting ipa-custodia Service
Failed to start ipa-custodia Service
Forced start, ignoring ipa-custodia Service, continuing normal operation
Starting ntpd Service
Starting pki-tomcatd Service
Starting ipa-otpd Service
Starting ipa-dnskeysyncd Service
ipa: INFO: The ipactl command was successful
[root@auto-hv-02-guest07 ~]# #UI accessible but login fails with error "login fails due to unknown reason"
[root@auto-hv-02-guest07 ~]# 

Scenario-2: (RHEL 7.3.z > RHEL 7.4 update2)- Upgrade SUCCESS / UI accessible and login successful (verification)
--------------------------------------------------------------------------------------------------------------------
[root@auto-hv-01-guest07 ~]# rpm -q ipa-server
ipa-server-4.4.0-14.el7_3.7.x86_64
[root@auto-hv-01-guest07 ~]# ldapsearch  -xLLL -D "cn=directory manager" -W -b "cn=certificates,cn=ipa,cn=etc,dc=testrelm,dc=test" dn ipaCertIssuerSerial
Enter LDAP Password: 
dn: cn=certificates,cn=ipa,cn=etc,dc=testrelm,dc=test

dn: cn=TESTRELM.TEST IPA CA,cn=certificates,cn=ipa,cn=etc,dc=testrelm,dc=test
ipaCertIssuerSerial: CN=Certificate Authority,O=TESTRELM.TEST;1
[root@auto-hv-01-guest07 ~]# ldapmodify -D "cn=directory manager" -W << EOF
> dn: cn=TESTRELM.TEST IPA CA,cn=certificates,cn=ipa,cn=etc,dc=testrelm,dc=test
> changetype: modrdn
> newrdn: cn=CA 1
> deleteoldrdn: 1
> EOF
Enter LDAP Password: 
modifying rdn of entry "cn=TESTRELM.TEST IPA CA,cn=certificates,cn=ipa,cn=etc,dc=testrelm,dc=test"

[root@auto-hv-01-guest07 ~]# ldapsearch  -xLLL -D "cn=directory manager" -W -b "cn=certificates,cn=ipa,cn=etc,dc=testrelm,dc=test" dn ipaCertIssuerSerial
Enter LDAP Password: 
dn: cn=certificates,cn=ipa,cn=etc,dc=testrelm,dc=test

dn: cn=CA 1,cn=certificates,cn=ipa,cn=etc,dc=testrelm,dc=test
ipaCertIssuerSerial: CN=Certificate Authority,O=TESTRELM.TEST;1

[root@auto-hv-01-guest07 ~]# ADD REPO FOR RHEL 7.4.update2
[root@auto-hv-01-guest07 ~]# yum -y update 'ipa*' sssd
[root@auto-hv-01-guest07 ~]# tail -1 /var/log/ipaupgrade.log 
2017-09-26T09:20:07Z INFO The ipa-server-upgrade command was successful
[root@auto-hv-01-guest07 ~]# rpm -q ipa-server
ipa-server-4.5.0-21.el7_4.2.2.x86_64
[root@auto-hv-01-guest07 ~]# ldapsearch  -xLLL -D "cn=directory manager" -W -b "cn=certificates,cn=ipa,cn=etc,dc=testrelm,dc=test" dn ipaCertIssuerSerial
Enter LDAP Password: 
dn: cn=certificates,cn=ipa,cn=etc,dc=testrelm,dc=test

dn: cn=TESTRELM.TEST IPA CA,cn=certificates,cn=ipa,cn=etc,dc=testrelm,dc=test
ipaCertIssuerSerial: CN=Certificate Authority,O=TESTRELM.TEST;1

[root@auto-hv-01-guest07 ~]# ipactl status
Directory Service: RUNNING
krb5kdc Service: RUNNING
kadmin Service: RUNNING
named Service: RUNNING
httpd Service: RUNNING
ipa-custodia Service: RUNNING
ntpd Service: RUNNING
pki-tomcatd Service: RUNNING
ipa-otpd Service: RUNNING
ipa-dnskeysyncd Service: RUNNING
ipa: INFO: The ipactl command was successful
[root@auto-hv-01-guest07 ~]# ipactl restart
Stopping pki-tomcatd Service
Restarting Directory Service
Restarting krb5kdc Service
Restarting kadmin Service
Restarting named Service
Restarting httpd Service
Restarting ipa-custodia Service
Restarting ntpd Service
Restarting pki-tomcatd Service
Restarting ipa-otpd Service
Restarting ipa-dnskeysyncd Service
ipa: INFO: The ipactl command was successful
[root@auto-hv-01-guest07 ~]# kinit admin
Password for admin: 
[root@auto-hv-01-guest07 ~]# 
[root@auto-hv-01-guest07 ~]# grep -rn "This entry already exists" /var/log/ipaupgrade.log
[root@auto-hv-01-guest07 ~]# 
[root@auto-hv-01-guest07 ~]# # UI Login Successful
[root@auto-hv-01-guest07 ~]# 


PART-TWO
================
Tested the bug with following steps for PLAIN INSTALLATION SETUP:


RHEL 7.3.z: ipa-server-4.4.0-14.el7_3.7.x86_64 (Reproducer)
====================
IPA-Master
--------------
Install: 
#ipa-server-install --subject=O=subject_testrelm.test

Verify:
#[root@auto-hv-01-guest06 ~]# ldapsearch  -xLLL -D "cn=directory manager" -W -b "cn=certificates,cn=ipa,cn=etc,dc=testrelm,dc=test" | grep subject
Enter LDAP Password: 
ipaCertSubject: CN=Certificate Authority,O=subject_testrelm.test
ipaCertIssuerSerial: CN=Certificate Authority,O=subject_testrelm.test;1

IPA-REPLICA:
---------------
Verify:
#ldapsearch  -xLLL -D "cn=directory manager" -W -b "cn=certificates,cn=ipa,cn=etc,dc=testrelm,dc=test" | grep subject
Enter LDAP Password: 
ipaCertIssuerSerial: CN=Certificate Authority,O=subject_testrelm.test;1
ipaCertSubject: CN=Certificate Authority,O=subject_testrelm.test
dn: cn=CN\3DCertificate Authority\2CO\3Dsubject_testrelm.test,cn=certificates,
cn: CN=Certificate Authority,O=subject_testrelm.test
ipaCertSubject: CN=Certificate Authority,O=subject_testrelm.test
ipaCertIssuerSerial: CN=Certificate Authority,O=subject_testrelm.test;1


RHEL 7.4 update2: ipa-server-4.5.0-21.el7_4.2.2.x86_64 (verification)
========================
IPA-Master
--------------
Install: 
#ipa-server-install --subject=O=subject_testrelm.test

Verify:
#ldapsearch  -xLLL -D "cn=directory manager" -W -b "cn=certificates,cn=ipa,cn=etc,dc=testrelm,dc=test" | grep subject
Enter LDAP Password: 
ipaCertSubject: CN=Certificate Authority,O=subject_testrelm.test
ipaCertIssuerSerial: CN=Certificate Authority,O=subject_testrelm.test;1

IPA-Replica:
---------------
#ldapsearch  -xLLL -D "cn=directory manager" -W -b "cn=certificates,cn=ipa,cn=etc,dc=testrelm,dc=test" | grep subject
Enter LDAP Password: 
ipaCertSubject: CN=Certificate Authority,O=subject_testrelm.test
ipaCertIssuerSerial: CN=Certificate Authority,O=subject_testrelm.test;1

Thus on the basis of above observations and comment#8, marking status of bug to "VERIFIED"

Comment 12 errata-xmlrpc 2017-10-19 15:12:49 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:2935