Bug 1623549

Summary: unable to renew Server-Cert certificates - error CA_UNREACHABLE
Product: Red Hat Enterprise Linux 7 Reporter: max_ter <massimo.terranova>
Component: ipaAssignee: IPA Maintainers <ipa-maint>
Status: CLOSED DUPLICATE QA Contact: ipa-qe <ipa-qe>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 7.7-AltCC: abokovoy, frenaud, massimo.terranova, pvoborni, rcritten, tscherf
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-12-05 12:22:23 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description max_ter 2018-08-29 15:17:44 UTC
Description of problem:
Certmonger is not able to renew the certificates with nickname "Server-Cert";

running ipa-getcert list, both the certificates with nickname "Server-Cert" return the following error:

Request ID '20170320134855':
        status: CA_UNREACHABLE
        ca-error: Server at https://MY_SITE.COM/ipa/xml failed request, will retry: 4301 (RPC failed at server.  Certificate operation cannot be completed: FAILURE (CA not found: 516f847a-ddc0-4da6-b15d-6703742b74be)).
        stuck: no
        ...

Performing an ldapsearch, the CA seems to be present:

# ldapsearch -Y GSSAPI -s sub -b cn=ipa,cn=cas,cn=ca,dc=MY_SITE,dc=com
SASL/GSSAPI authentication started
SASL username: admin
SASL SSF: 56
SASL data security layer installed.
# extended LDIF
#
# LDAPv3
# base <cn=ipa,cn=cas,cn=ca,dc=MY_SITE,dc=com> with scope subtree
# filter: (objectclass=*)
# requesting: ALL
#

# ipa, cas, ca, MY_SITE.COM
dn: cn=ipa,cn=cas,cn=ca,dc=MY_SITE,dc=com
description: IPA CA
ipaCaIssuerDN: CN=Certificate Authority,O=MY_SITE.COM
objectClass: top
objectClass: ipaca
ipaCaSubjectDN: CN=Certificate Authority,O=MY_SITE.COM
ipaCaId: 516f847a-ddc0-4da6-b15d-6703742b74be
cn: ipa

# search result
search: 4
result: 0 Success

# numResponses: 2
# numEntries: 1



Version-Release number of selected component (if applicable):
# ipa --version
VERSION: 4.4.0, API_VERSION: 2.213

Comment 2 Alexander Bokovoy 2018-08-29 21:21:17 UTC
Could you please add

/etc/sysconfig/certmonger
OPTS=-d3

restart certmonger, wait a bit until certmonger settles down and provide output of 

 journalctl -u certmonger --full

for the time period since restart.

Comment 3 max_ter 2018-09-12 12:59:50 UTC
Hello Alexander,

We found and solved the issue: looking in ldap, in the entry cn=ipa,cn=cas,cn=ca,dc=my_site,dc=com we saw this attribute-value
ipaCaId: 516f847a-ddc0-4da6-b15d-6703742b74be

so, there should have been an entry like 
dn: cn=$ipacaid,ou=authorities,ou=ca,o=ipaca 

but looking under ou=authorities,ou=ca,o=ipaca, the entry had a different cn value.

We aligned the first entry with the value of the entry in the ipaca part, and the problem was solved.

Maybe this misalignment was due to a replication failure between the masters?

Thank you.
Max

Comment 4 max_ter 2018-09-18 12:11:39 UTC
(In reply to Alexander Bokovoy from comment #2)
> Could you please add
> 
> /etc/sysconfig/certmonger
> OPTS=-d3
> 
> restart certmonger, wait a bit until certmonger settles down and provide
> output of 
> 
>  journalctl -u certmonger --full
> 
> for the time period since restart.

Hello Alexander,

We found and solved the issue: looking in ldap, in the entry cn=ipa,cn=cas,cn=ca,dc=my_site,dc=com we saw this attribute-value
ipaCaId: 516f847a-ddc0-4da6-b15d-6703742b74be

so, there should have been an entry like 
dn: cn=$ipacaid,ou=authorities,ou=ca,o=ipaca 

but looking under ou=authorities,ou=ca,o=ipaca, the entry had a different cn value.

We aligned the first entry with the value of the entry in the ipaca part, and the problem was solved.

Maybe this misalignment was due to a replication failure between the masters?

Thank you.
Max

Comment 5 Alexander Bokovoy 2018-09-18 12:57:09 UTC
It is unlikely due to a replication failure (you'd then would have two different ipaCaId entries). Could you tell a bit more how these systems were installed?

Comment 6 Alexander Bokovoy 2018-10-03 05:56:11 UTC
Rob, do you think the check for ipaCaId being correct in both cn=ipa,cn=cas,cn=ca,$SUFFIX and cn=$ipacaid,ou=authorities,ou=ca,o=ipaca would be useful for the new troubleshooting tool?

If so, should we copy this bug to Pagure and close it here, since the original issue is solved by Max already.

Comment 7 Rob Crittenden 2018-10-03 14:06:37 UTC
Yes, I do think it should be checked. It is part of the tool I started at https://github.com/rcritten/checkcerts

So if existence in my tool, which I fully expect to be merged into something else eventually, is enough then I think we can close this BZ and track it in that upstream.

Comment 8 Florence Blanc-Renaud 2018-12-05 12:22:23 UTC
The upstream ticket for health check tool is https://pagure.io/freeipa/issue/4008. The corresponding BZ is BZ 1152084.

Closing this BZ as proposed in comments #c6 and #c7, as the customer issue has been solved and an upstream ticket+BZ is already tracking the check.

*** This bug has been marked as a duplicate of bug 1152084 ***