RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1138777 - Renewal with no master CA
Summary: Renewal with no master CA
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: ipa
Version: 7.0
Hardware: Unspecified
OS: Unspecified
medium
unspecified
Target Milestone: rc
: ---
Assignee: Martin Kosek
QA Contact: Namita Soman
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-09-05 15:32 UTC by Martin Kosek
Modified: 2015-03-05 10:13 UTC (History)
4 users (show)

Fixed In Version: ipa-4.0.3-1.el7
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-03-05 10:13:35 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2015:0442 0 normal SHIPPED_LIVE Moderate: ipa security, bug fix, and enhancement update 2015-03-05 14:50:39 UTC

Description Martin Kosek 2014-09-05 15:32:54 UTC 
This bug is created as a clone of upstream ticket:
https://fedorahosted.org/freeipa/ticket/4039

A user has 3 CAs. None of them is configured to do CA renewal. They are all configured to pull updated certificates from LDAP using the certmonger renew_ra_cert CA. It is unclear how it got into this situation. It could be the original master was deleted, or this was the result of some upgrade, but none of the masters is configured to do the actual subsystem renewal. The result was the renewal status was CA_WORKING on all systems.

The user is using IPA 3.0 on RHEL 6.4, so this is a dual 389-ds install.

Fixing this was a relatively straightforward process to pick one to do the renewal, tell certmonger to stop tracking the CA subsystem certificates and then configure certmonger to renew the certificates (it is a difference in the CA script that certmonger users).

So this is problem #1: provide documentation on how to recover from this situation.

This worked ok and all 4 of the required certificates were renewed, but something strange happened.

Only one certificate was added to LDAP, the RA agent cert ipaCert. We never did figure out why. 

This is problem #2: see how it can happen that a certificate is renewed and not added to LDAP.

Replication was firing as we saw the ipaCert entry in cn=ca_renewal on all masters and the ou=People entry in the CA DS instance was replicated ok as well. So in short, replication was working in both instances.

To fix this we manually pulled the updated certificates out of the various databases using certutil -L -d /path/to/db -n <nickname> -a. Then we moved the certificates to the non-updated masters and manually added them to their respective database. Note that certutil will pull ALL the certificates out of an NSS database if there is more than one certificate, such as the case for a renewal. This is fine, but it also will only ADD one certificate via certutil -A which means you'll need to edit each file such that only the latest certificate is stored there.

We had to do this for the 3 CA certificates (ocsp, audit, subsystem) in /var/lib/<location>/alias (location is distro-specific) and the ipaCert RA agent certificate in /etc/httpd/alias.
Comment 1 Martin Kosek 2014-09-05 16:14:35 UTC 
Fixed upstream:

master:
774140196360c727f11c75622ace488d591ddfba Allow changing CA renewal master in ipa-csreplica-manage.

ipa-4-1:
aae78480220203b1c64c8b3c6b8297868c849110 Allow changing CA renewal master in ipa-csreplica-manage.

ipa-4-0:
8999300894326d104ddf22a97d74d78fdab0984c Allow changing CA renewal master in ipa-csreplica-manage.
Comment 3 Scott Poore 2015-01-13 02:41:46 UTC 
Martin,

Any idea today on how we can reproduce this?  Or just test that CA renewal master can be changed with ipa-csreplica-manage?

Thanks,
Scott
Comment 4 Martin Kosek 2015-01-13 11:20:10 UTC 
Jan owned the upstream ticket, he should be able to help.
Comment 5 Scott Poore 2015-01-14 19:11:44 UTC 
Also, how can I see which host is the master?  There's a set-renewal-master command but, how do I see which one is master?

Thanks,
Scott
Comment 6 Scott Poore 2015-01-15 01:19:42 UTC 
nevermind on the last one.  I found it:

ldapsearch -xLLL -D "cn=Directory Manager" -w Secret123 \
    -b cn=CA,cn=$MASTER,cn=masters,cn=ipa,cn=etc,$BASEDN | \
    grep caRenewalMaster|wc -l

if it's the renewal master that will = 1

So, I still need to know if there's any way to reproduce this or just test setup-renewal-master option of ipa-csreplica-manage?

Thanks
Comment 7 Jan Cholasta 2015-01-15 08:18:18 UTC 
Now that renewal race conditions are removed, I think it should be enough to test "ipa-csreplica-manage set-renewal-master". Just FYI, "ipa-cacert-manage renew" changes renewal master to the host it is run on (it is necessary to make it work properly).
Comment 8 Scott Poore 2015-01-15 15:55:32 UTC 
Verified.

Version ::

ipa-server-4.1.0-13.el7.x86_64

Results ::


[root@rhel7-1 ~]# ldapsearch -xLLL -D "cn=Directory Manager" -w Secret123     -b cn=masters,cn=ipa,cn=etc,$BASEDN |egrep "cn=CA|caRenewalMaster"
dn: cn=CA,cn=rhel7-1.example.com,cn=masters,cn=ipa,cn=etc,dc=example,dc=com
ipaConfigString: caRenewalMaster
dn: cn=CA,cn=rhel7-3.example.com,cn=masters,cn=ipa,cn=etc,dc=example,dc=com
dn: cn=CA,cn=rhel7-4.example.com,cn=masters,cn=ipa,cn=etc,dc=example,dc=com
dn: cn=CA,cn=rhel7-5.example.com,cn=masters,cn=ipa,cn=etc,dc=example,dc=com

[root@rhel7-1 ~]# ipa-csreplica-manage -p Secret123 set-renewal-master 
rhel7-1.example.com is already the renewal master

[root@rhel7-1 ~]# ipa-csreplica-manage -p Secret123 set-renewal-master rhel7-2.example.com
Failed to set renewal master to rhel7-2.example.com: no such entry

[root@rhel7-1 ~]# ipa-csreplica-manage -p Secret123 set-renewal-master rhel7-3.example.com
rhel7-3.example.com is now the renewal master

[root@rhel7-1 ~]# ldapsearch -xLLL -D "cn=Directory Manager" -w Secret123     -b cn=masters,cn=ipa,cn=etc,$BASEDN |egrep "cn=CA|caRenewalMaster"
dn: cn=CA,cn=rhel7-1.example.com,cn=masters,cn=ipa,cn=etc,dc=example,dc=com
dn: cn=CA,cn=rhel7-3.example.com,cn=masters,cn=ipa,cn=etc,dc=example,dc=com
ipaConfigString: caRenewalMaster
dn: cn=CA,cn=rhel7-4.example.com,cn=masters,cn=ipa,cn=etc,dc=example,dc=com
dn: cn=CA,cn=rhel7-5.example.com,cn=masters,cn=ipa,cn=etc,dc=example,dc=com

[root@rhel7-1 ~]# ipa-csreplica-manage -p Secret123 set-renewal-master rhel7-5.example.comrhel7-5.example.com is now the renewal master

[root@rhel7-1 ~]# ldapsearch -xLLL -D "cn=Directory Manager" -w Secret123     -b cn=masters,cn=ipa,cn=etc,$BASEDN |egrep "cn=CA|caRenewalMaster"
dn: cn=CA,cn=rhel7-1.example.com,cn=masters,cn=ipa,cn=etc,dc=example,dc=com
dn: cn=CA,cn=rhel7-3.example.com,cn=masters,cn=ipa,cn=etc,dc=example,dc=com
dn: cn=CA,cn=rhel7-4.example.com,cn=masters,cn=ipa,cn=etc,dc=example,dc=com
dn: cn=CA,cn=rhel7-5.example.com,cn=masters,cn=ipa,cn=etc,dc=example,dc=com
ipaConfigString: caRenewalMaster

[root@rhel7-1 ~]# ipa-csreplica-manage -p Secret123 set-renewal-master rhel7-1.example.comrhel7-1.example.com is now the renewal master

[root@rhel7-1 ~]# ldapsearch -xLLL -D "cn=Directory Manager" -w Secret123     -b cn=masters,cn=ipa,cn=etc,$BASEDN |egrep "cn=CA|caRenewalMaster"
dn: cn=CA,cn=rhel7-1.example.com,cn=masters,cn=ipa,cn=etc,dc=example,dc=com
ipaConfigString: caRenewalMaster
dn: cn=CA,cn=rhel7-3.example.com,cn=masters,cn=ipa,cn=etc,dc=example,dc=com
dn: cn=CA,cn=rhel7-4.example.com,cn=masters,cn=ipa,cn=etc,dc=example,dc=com
dn: cn=CA,cn=rhel7-5.example.com,cn=masters,cn=ipa,cn=etc,dc=example,dc=com

[root@rhel7-3 ~]# ipa-csreplica-manage -p Secret123 set-renewal-master
rhel7-3.example.com is now the renewal master
Comment 10 errata-xmlrpc 2015-03-05 10:13:35 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2015-0442.html
Note You need to log in before you can comment on or make changes to this bug.