RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1371519 - pki instance creation fails during ipa replica install
Summary: pki instance creation fails during ipa replica install
Keywords:
Status: CLOSED WORKSFORME
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: ipa
Version: 7.3
Hardware: Unspecified
OS: Unspecified
urgent
urgent
Target Milestone: rc
: ---
Assignee: IPA Maintainers
QA Contact: Kaleem
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-08-30 11:58 UTC by Kaleem
Modified: 2016-12-09 09:32 UTC (History)
14 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-09-20 13:28:50 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)
console output with steps (5.10 KB, text/plain)
2016-08-30 12:01 UTC, Kaleem 		no flags Details
snip from ipa replica install log file (11.31 KB, text/plain)
2016-08-30 12:31 UTC, Kaleem 		no flags Details
View All

Description Kaleem 2016-08-30 11:58:51 UTC 
Description of problem:

During ipa replica install on 7.3 from replica file created on 6.8 IPA Master, 
pki instance creation fails with following error message.

2016-08-30 17:18:30 pkispawn    : INFO     ....... modifying '/etc/pki/pki-tomcat/alias/secmod.db'
2016-08-30 17:18:30 pkispawn    : DEBUG    ........... chmod 600 /etc/pki/pki-tomcat/alias/secmod.db
2016-08-30 17:18:30 pkispawn    : DEBUG    ........... chown 17:17 /etc/pki/pki-tomcat/alias/secmod.db
2016-08-30 17:18:36 pkispawn    : DEBUG    ....... Error Type: CalledProcessError
2016-08-30 17:18:36 pkispawn    : DEBUG    ....... Error Message: Command '['certutil', '-M', '-d', '/etc/pki/pki-tomcat/alias', '-f', '/etc/pki/pki-tomcat/pfile', '-n', 'caSigningCert cert-pki-ca', '-t', 'CTu,Cu,Cu']' returned non-zero exit status 255
2016-08-30 17:18:36 pkispawn    : DEBUG    .......   File "/usr/sbin/pkispawn", line 528, in main
    scriptlet.spawn(deployer)
  File "/usr/lib/python2.7/site-packages/pki/server/deployment/scriptlets/security_databases.py", line 159, in spawn
    trust_attributes='CTu,Cu,Cu')
  File "/usr/lib/python2.7/site-packages/pki/nssdb.py", line 165, in modify_cert
    subprocess.check_call(cmd)
  File "/usr/lib64/python2.7/subprocess.py", line 542, in check_call
    raise CalledProcessError(retcode, cmd)


Version-Release number of selected component (if applicable):


How reproducible:
[root@dhcp207-47 ~]# rpm -q ipa-server pki-ca
ipa-server-4.4.0-8.el7.x86_64
pki-ca-10.3.3-7.el7.noarch
[root@dhcp207-47 ~]#

Steps to Reproduce:
1. Prepare a RHEL 6.8 IPA Master and generate replica install file on it.
2. Copy replica install file on RHEL 7.3 replica and install replica
3.

Actual results:
Replica install fails on RHEL 7.3

Expected results:
Replica install should be successful on RHEL 7.3

Additional info:
(1) Please find the attached console output file for more info
Comment 1 Kaleem 2016-08-30 12:01:16 UTC 
Created attachment 1195864 [details]
console output with steps
Comment 2 Kaleem 2016-08-30 12:31:27 UTC 
Created attachment 1195882 [details]
snip from ipa replica install log file
Comment 4 Endi Sukma Dewata 2016-08-31 01:43:34 UTC 
It looks like the nickname of the CA certificate in the PKCS #12 file exported from the RHEL 6.8 instance does not match the nickname expected by the RHEL 7.3 instance. 

The PKCS #12 file contains the CA certificate with the following nickname:

  Serial Number: 0x1
  Nickname: Certificate Authority - TESTRELM.TEST
  Subject DN: CN=Certificate Authority,O=TESTRELM.TEST
  Issuer DN: CN=Certificate Authority,O=TESTRELM.TEST

The pkispawn configuration uses a different nickname:

  pki_ca_signing_nickname = caSigningCert cert-pki-ca

So when pkispawn tries to set the trust flags of the CA certificate using certutil it fails:

Installation failed: Command '['certutil', '-M', '-d', '/etc/pki/pki-tomcat/alias', '-f', '/etc/pki/pki-tomcat/pfile', '-n', 'caSigningCert cert-pki-ca', '-t', 'CTu,Cu,Cu']' returned non-zero exit status 255

I think the problem needs to be fixed in the RHEL 6.8 instance to make sure it generates a PKCS #12 file with the correct nickname. The RHEL 7.3 instance should continue to assume that the PKCS #12 file already contains the correct nickname.
Comment 5 Endi Sukma Dewata 2016-08-31 01:44:15 UTC 
Kaleem,

Could you list the certificates in the RHEL 6.8 instance with this command?
$ certutil -L -d /var/lib/pki-ca/alias

Was the the RHEL 6.8 instance used in this migration previously cloned from another instance?

Thanks.
Comment 6 Kaleem 2016-08-31 06:12:03 UTC 
(In reply to Endi Sukma Dewata from comment #5)
> Kaleem,
> 
> Could you list the certificates in the RHEL 6.8 instance with this command?
> $ certutil -L -d /var/lib/pki-ca/alias
> 
> Was the the RHEL 6.8 instance used in this migration previously cloned from
> another instance?
No, its was fresh 6.8 IPA master install.
> 
> Thanks.

[root@dhcp207-46 ~]# certutil -L -d /var/lib/pki-ca/alias

Certificate Nickname                                         Trust Attributes
                                                             SSL,S/MIME,JAR/XPI

ocspSigningCert cert-pki-ca                                  u,u,u
subsystemCert cert-pki-ca                                    u,u,u
caSigningCert cert-pki-ca                                    CTu,Cu,Cu
Server-Cert cert-pki-ca                                      u,u,u
auditSigningCert cert-pki-ca                                 u,u,Pu
[root@dhcp207-46 ~]#
Comment 7 Endi Sukma Dewata 2016-08-31 13:48:47 UTC 
Petr/Jan,

How does IPA export the certificates in RHEL 6.8? Any idea why the nickname changes from "caSigningCert cert-pki-ca" in the NSS database to "Certificate Authority - TESTRELM.TEST" in the PKCS #12 file?
Comment 8 Jan Cholasta 2016-09-02 08:50:58 UTC 
No idea why the nickname changed, but I don't agree that this should be fixed on the 6.8 side - the PKCS#12 file contains everything needed for install, we should detect this situation and allow install on RHEL 7.3.
Comment 9 Kaleem 2016-09-02 09:10:01 UTC 
Scenario worked till ipa build ipa-4.4.0-4.el7
Comment 10 Petr Vobornik 2016-09-06 07:24:22 UTC 
Changing component based on comments 7-9
Comment 14 Florence Blanc-Renaud 2016-09-07 13:50:37 UTC 
What I can see so far:
pkispawn is called with a configuration file containing:
pki_clone_pkcs12_path = /tmp/ca.p12
(file containing the certificates).

The file /tmp/ca.p12 is a copy of realm_info/cacert.p12 contained in the replica gpg file (the gpg file is generated by IPA 6.8 master, and the copy is done by ipa-replica-install).
It contains twice the certificate for CN=Certificate Authority,O=TESTRELM.TEST (once with a Friendly Name: caSigningCert cert-pki-ca, and once without any friendly name).

It looks like pkispawn used to work properly with this duplicate cert but does not anymore. The issue can either be fixed on Dogtag side, or we can work on the pkcs12 file on our side.
Comment 15 Petr Vobornik 2016-09-07 14:25:05 UTC 
Endi do you know if something happen in this area on dogtag side? Or, do you think the fix should be on dogtag side?
Comment 16 Endi Sukma Dewata 2016-09-07 15:44:50 UTC 
We've been cleaning up the installation code including cloning and PKCS #12 handling (some of the changes were driven by IPA-originated bugs), but the assumption has always been that the nicknames in the PKCS #12 file would match the pkispawn parameters. There might have been a bug which allowed PKI to work with incorrectly created PKCS #12 in the past, but that's apparently no longer the case.

The fix depends on where the wrong nickname was initially introduced. If IPA uses a PKI tool to export the certificate with the wrong nickname, then it needs to be fixed in PKI. If IPA uses an NSS tool instead, then it probably needs to be fixed in NSS, or IPA can switch to use PKI tool or create a workaround.
Comment 18 Petr Vobornik 2016-09-08 11:39:54 UTC 
Upstream ticket:
https://fedorahosted.org/freeipa/ticket/6310
Comment 19 Jan Cholasta 2016-09-09 06:56:48 UTC 
(In reply to Endi Sukma Dewata from comment #16)
> The fix depends on where the wrong nickname was initially introduced. If IPA
> uses a PKI tool to export the certificate with the wrong nickname, then it
> needs to be fixed in PKI. If IPA uses an NSS tool instead, then it probably
> needs to be fixed in NSS, or IPA can switch to use PKI tool or create a
> workaround.

On the RHEL 6.8 side, AFAICT the file is a copy of /root/tmp-ca.p12 created by pkisilent, so I guess this needs to be fixed in PKI.
Comment 21 Endi Sukma Dewata 2016-09-09 17:31:40 UTC 
I haven't looked at the code, but it seems to be true that on RHEL 6.8 the pkisilent generates a PKCS #12 file with the wrong nickname, and that may need to be fixed separately.

However, that issue may be irrelevant for cloning. There is no guarantee that the PKCS #12 contains the exact same certificates actually used by PKI in the NSS database unless it's exported immediately before cloning.

I think the proper solution would be for ipa-replica-prepare to export the certificates directly from the NSS database into a new PKCS #12 file using PKCS12Export. This tool does export the nicknames correctly so it should fix the problem.
Comment 22 Ade Lee 2016-09-09 20:15:06 UTC 
I was very surprised to see that the IPA code in 6.8 still simply copies over the initial PKCS12 file generated by pkisilent when doing an ipa-replica-prepare.  This will definitely not work for cases where the certs have been renewed for instance, which is a case that increasingly likely for IPA servers on this RHEL version.

There is code in at least IPA 3.3 to regenerate the PKCS#12 using PKCS12Export.
The PKCS#12 file generated by PKCS12Export is known to have the correct nicknames and to be parsed correctly by Dogtag 10.

Therefore, I suggest you backport the fix to ipa-replica-prepare to regenerate the PKCS12 file afresh - and kills two birds with one stone.
Comment 23 Ade Lee 2016-09-09 20:18:09 UTC 
Note that until the IPA ipa-replica-prepare code in 6.8 is fixed, there is a manual workaround of recreating the pkcs12 file (/root/cacert.p12) using PKCS12Export before doing the ipa-replica-prepare.
Comment 24 Petr Vobornik 2016-09-12 10:26:03 UTC 
IPA is doing 6.8 z-stream with bug 1369470 which is AFAIK the thing[1] which Endi and Ade proposed.

[1] https://git.fedorahosted.org/cgit/freeipa.git/commit/?id=7b402b3bc30af1e57b0451bd2ecfb121ee1739e5 (scroll to the end)
Comment 32 Kaleem 2016-09-20 13:28:50 UTC 
Closing this as works for now because reported https://bugzilla.redhat.com/show_bug.cgi?id=1377730 for replication issue seen.
Note You need to log in before you can comment on or make changes to this bug.