What do you mean "promotion"?

This seems like the re-install issue.
See: https://bugzilla.redhat.com/show_bug.cgi?id=817080

Cleaning the certmonger directories would solve the issue.
I will link this to existing ticket and kick it back into needs triage.

Upstream ticket:
https://fedorahosted.org/freeipa/ticket/2702

I'm going to guess that they previously tried to install IPA on this machine and it failed, or it was a client and something went wrong with the uninstall (in either case).

What has happened is certmonger is already tracking a certificate that we're trying to track.

To fix this, run:

# ipa-server-install --uninstall -U
# service certmonger start
# ipa-getcert list

There may be more tracked requests than just 20120616171834. If they have not specifically tracked these certificates themselves it should be safe to untrack them.

To do so, for each one run:

# ipa-getcert stop-tracking -i <request_id>

Will that uninstall the IPA server?  Do they need to reinstall the ipa server afterwards?

To be safe, can you describe what you mean by promotion and what the exactly is being done here?

I based my comment on the fact that this looked like a standard replica installation. If this is true then you run the uninstall on the replica machine, then do the clean up as I suggested.

So they need to install a replica yes.  They got to installing the CA cert but "the CA fails to start up properly on the replica server."  IPA is one of my huge knowledge gaps over here in IDM. The promotion of the replica to master status is the end goal here but they aren't to the point of having the replica installed and running yet.

If your recomendation is to perform these steps I will pass them along.

There is no promotion of replica to master. Installing another IPA server using ipa-replica-prepare/ipa-replica-install installs another IPA master server.

Yes, have them run these commands on the replica. If the installation of the CA fails they will need to look at /var/log/ipareplica-install.log for additional information. We don't get much out of the CA installer except for a pass/fail so are unable to provide a lot of guidance about what happened.

Just spoke with the customer.  They are going to run these commands Friday night. 

  # ipa-server-install --uninstall -U
  # service certmonger start
  # ipa-getcert list

Thanks for your help.  I'll let you know the outcome next week.

Just to clarify, the customer was talking about the CA authority promotion.  That is where the process is failing.  They tried the testpatch that was given and the commands: 

  # ipa-server-install --uninstall -U
  # service certmonger start
  # ipa-getcert list

Which have not resolved the issue.  What is the next course of action here?

Thank You

(In reply to comment #10)
> Just to clarify, the customer was talking about the CA authority promotion. 
> That is where the process is failing.  They tried the testpatch that was
> given and the commands: 
> 
>   # ipa-server-install --uninstall -U
>   # service certmonger start
>   # ipa-getcert list
> 
> Which have not resolved the issue.  What is the next course of action here?
> 
> Thank You

Did 
# ipa-getcert list
return the list of the certs currently being tracked and was the command 
# ipa-getcert stop-tracking -i <request_id> 
run for every cert listed?

I think you need to clarify what the customer is trying to do. They are trying to install a new replica to replace an existing master? If they are using a dogtag CA then there is no promotion, all CAs are masters. The documentation refers only to configuring which one generates a CRL and in 2.2, all masters generate a CRL.

So the customer has a single existing IPA server that they would like to decommision.  In its place they would like to have 2 redundant ipa servers.  The issue comes when, during the installation of one of the new servers, they try to promote it to be the Master CA. Does that answer your question?

Yes, but it conflates two issues. This problem has nothing to do with promoting a master, it has to do with replica installation. From what I can tell a previous installation failed for some reason and uninstall did not completely remove everything. This is preventing re-install. If they stop tracking the certificates that should allow reinstallation.

Agreed. However, the steps to stop tracking the certs (eg: # ipa-server-install --uninstall -U ; # service certmonger start ; # ipa-getcert list) did not resolve the issue. Would it behoove the customer at this point to completely reinstall or is there maybe another way you know of to try to stop the system from tracking the certs?

-----Begin Copied Message-----
Did 
# ipa-getcert list
return the list of the certs currently being tracked and was the command 
# ipa-getcert stop-tracking -i <request_id> 
run for every cert listed?
-----End Copied Message-----

I will ask the customer these questions and get back to you on that.

From the customer:
-----Begin Copied Message-----
The ipa-getcert list returned 0 entries.  Here's the output we saw before trying the reinstall.

# service certmonger start
Starting certmonger:                                       [  OK  ]
# ipa-getcert list
Number of certificates and requests being tracked: 0.

#
-----End Copied Message-----

Is there anything else we can have them try?

They should try the replica installation again then. If it fails we'll need to see the full /var/log/ipareplica-install.log.

The customer has been asked to perform the replica installation again and upload the /var/log/ipareplica-install.log when complete.  Thanks for your help so far

The customer is taring them up.  I don't expect to have them until Monday but they may be in this evening.

Created attachment 631761 [details]
The DEbug Logs from the Replica Installation

Created attachment 631762 [details]
The INstallation Logs from the Replica Installation

Hello, I've uploaded the ipa replica installation and debug logs to the BZ.  I'm going to look over them myself.

Thanks

In talking to the customer, I found out that this server used to serve another purpose and it is possible that the ca got corrupted somehow. Here is a couple of questions that could possibly lead to a fix for them:

How is the cert generated from dogtag?  Does dogtags just export the private key?
Is there a way to have the master ca generate a new ca that could be re-prepared for a replica install?

Thanks

Jesse

The prepared file contains a number of different things, including SSL certificates for the web and directory servers, the RA agent certificate and private key and the CA keys (the file is encrypted).

You can always prepare a new file for this replica using ipa-replica-prepare.

There seems to be some evidence of a lunasa HSM on the machine, can you confirm?

From the customer:

#75 Created By: Fiacchino, jim  (10/23/2012 4:14 PM) Last Modified By: Fiacchino, jim  (10/23/2012 4:14 PM) 
No, lunasa isn't installed on the server.

The customer's question regarding regenerating the ca cert from the ca authority on the IPA master was from them feeling that maybe there was corruption in the master's certificate system.  How likely do you feel it is that corruption of this nature could be the issue?

I suppose that it is possible that the wrong CA was provided to ipa-replica-prepare. It uses the file /root/ca.p12. It is expected that this is the CA as generated during the initial CA installation.

Is there a way to have the master CA generate a new CA?  If so we could re-prepare the file and retry in the installation.

There should be a way to regenerate that PKCS#12 file. It is provided to us at the end of the CA installation. I'm hoping that one of the CS engineers cc'd here will chime in with instructions on how to regenerate this file. We recommend at the end of the IPA server installation that this file be safely stored some place.

To regenerate that PKCS#12 file try the following:
https://access.redhat.com/knowledge/docs/en-US/Red_Hat_Certificate_System/8.1/html/Deploy_and_Install_Guide/exporting-keys.html

In an IPA context this is what you need to do:

On a working master with a CA installed:

- Get the internal password from /var/lib/pki-ca/conf/password.conf and put it into the file /tmp/internal.txt
- Create the file /tmp/password.txt and put your Directory Manager password into it
- run: PKCS12Export -debug -d /var/lib/pki-ca/alias -w /tmp/password.txt -p /tmp/internal.txt -o cacert.p12
- rm -f /tmp/password.txt /tmp/internal.txt

Copy/move cacert.p12 to /root (you may want to save off a copy of whatever is there, I'm conservative).

On that master re-run: ipa-replica-prepare

Use the new prepared file to try to install a replica on the non-working server.

We'd like to see the list of certificates in the CA NSS certificate database. Can you provide the output of this on the working master:

# certutil -L -d /var/lib/pki-ca/alias

Here are the requested certificates:


Certificate Nickname                                         Trust Attributes
                                                             SSL,S/MIME,JAR/XPI

caSigningCert cert-pki-ca                                    CTu,Cu,Cu
Server-Cert cert-pki-ca                                      u,u,u
auditSigningCert cert-pki-ca                                 u,u,Pu
ocspSigningCert cert-pki-ca                                  u,u,u
subsystemCert cert-pki-ca                                    u,u,u

Closing as a duplicate of ​817080.

The uninstaller was enhanced to look for remaining certificates tracked by certmonger when the uninstall finishes. The user will be notified if any remain.

*** This bug has been marked as a duplicate of bug 817080 ***