Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1159011

Summary: Trust setting not restored for CA cert with ipa-restore command
Product: Red Hat Enterprise Linux 7 Reporter: Kaleem <ksiddiqu>
Component: ipaAssignee: Jan Cholasta <jcholast>
Status: CLOSED ERRATA QA Contact: Namita Soman <nsoman>
Severity: high Docs Contact:
Priority: medium    
Version: 7.1CC: dpal, jcholast, kengert, ksiddiqu, mkosek, nsoman, pviktori, rcritten
Target Milestone: rcKeywords: TestBlocker
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: ipa-4.1.0-5.el7 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-03-05 10:14:15 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
contents of /etc/httpd/alias
none
ipa.p11-kit file attached along with console log none

Description Kaleem 2014-10-30 16:55:37 UTC 
Description of problem:
Trust settings of CA cert not restored correctly in nss db. This observed while execution of ipa-restore which restores /etc/httpd/alias nss db which was backuped earlier and it causes httpd service restart failure. Consequently ipa service restart fails.

Version-Release number of selected component (if applicable):
[root@dhcp207-187 ~]# rpm -q ipa-server nss-tools nss
ipa-server-4.1.0-3.el7.x86_64
nss-tools-3.16.2-9.el7.x86_64
nss-3.16.2-9.el7.x86_64
[root@dhcp207-187 ~]# 

How reproducible:
Always.

Steps to Reproduce:
1. Please find the attached file (steps-with-console-output.txt) for detail.

Actual results:
Trust settings of CA cert not restored correctly.

Expected results:
Trust settings of CA cert should have been restored correctly

Additional info:
Comment 3 Kaleem 2014-10-30 16:59:42 UTC 
======= Test machine =======

[root@dhcp207-187 alias]# certutil -L -d .

Certificate Nickname                                         Trust Attributes
                                                             SSL,S/MIME,JAR/XPI

Signing-Cert                                                 u,u,u
ipaCert                                                      u,u,u
Server-Cert                                                  u,u,u
TESTRELM.TEST IPA CA                                         ,,   
[root@dhcp207-187 alias]# rpm -q nss-tools
nss-tools-3.16.2-9.el7.x86_64

[root@dhcp207-187 alias]# sha1sum * | sha1sum
a6e1002f1f2da3e8ee822f32e796ef5cd5a326a7  -


======= My laptop =======

pviktori@dhcp-31-13:/tmp/tx/alias$ certutil -L -d .

Certificate Nickname                                         Trust Attributes
                                                             SSL,S/MIME,JAR/XPI

Signing-Cert                                                 u,u,u
ipaCert                                                      u,u,u
Server-Cert                                                  u,u,u
TESTRELM.TEST IPA CA                                         CT,C,C
pviktori@dhcp-31-13:/tmp/tx/alias$ rpm -q nss-tools
nss-tools-3.17.1-1.fc21.x86_64
pviktori@dhcp-31-13:/tmp/tx/alias$ sha1sum * | sha1sum
a6e1002f1f2da3e8ee822f32e796ef5cd5a326a7  -
Comment 5 Petr Viktorin (pviktori) 2014-11-04 15:16:38 UTC 
Kaleem, could you also attach the contents of /etc/httpd/alias from the affected machine?

The database is the same (see the hashes above), but sometimes the trust flags are reported missing. It would be nice if NSS the folks could investigate.
Comment 6 Kaleem 2014-11-04 17:03:57 UTC 
[root@dhcp207-187 alias]# certutil -L -d .

Certificate Nickname                                         Trust Attributes
                                                             SSL,S/MIME,JAR/XPI

Signing-Cert                                                 u,u,u
ipaCert                                                      u,u,u
Server-Cert                                                  u,u,u
TESTRELM.TEST IPA CA                                         ,,   
[root@dhcp207-187 alias]# rpm -q nss-tools
nss-tools-3.16.2-9.el7.x86_64
[root@dhcp207-187 alias]# sha1sum * | sha1sum
a6e1002f1f2da3e8ee822f32e796ef5cd5a326a7  -
[root@dhcp207-187 alias]#
Comment 7 Petr Viktorin (pviktori) 2014-11-04 17:24:02 UTC 
I meant, add the contents of the /etc/httpd/alias directory as an attachment to the bug, so they can be examined.
Comment 8 Kaleem 2014-11-05 03:21:01 UTC 
Created attachment 953904 [details]
contents of /etc/httpd/alias
Comment 10 Martin Kosek 2014-11-10 07:21:42 UTC 
I tested the attachment and on both my F20 and my RHEL-7.x (nss-3.16.2-9.el7.x86_64) I get:

# certutil -L -d .

Certificate Nickname                                         Trust Attributes
                                                             SSL,S/MIME,JAR/XPI

Signing-Cert                                                 u,u,u
ipaCert                                                      u,u,u
Server-Cert                                                  u,u,u
TESTRELM.TEST IPA CA                                         CT,C,C

Is this the right attachment or am I missing anything? Also CCing Kai in case he some idea what may be wrong.

BTW, I think the hashes demonstrated in Comment 3 will not work well as the dir points to libnssckbi.so which can change with NSS version.
Comment 11 Jan Cholasta 2014-11-10 07:52:12 UTC 
Kaleem, could you please post the output of "certutil -L -d /etc/httpd/alias/ -h 'System Trust'" before and after the restore?
Comment 12 Kaleem 2014-11-10 09:14:24 UTC 
[root@ibm-x3650m4-02-vm-03 ~]# certutil -L -d /etc/httpd/alias/ -h 'System Trust'

Certificate Nickname                                         Trust Attributes
                                                             SSL,S/MIME,JAR/XPI

System Trust:TESTRELM.TEST IPA CA                            CT,C,C
[root@ibm-x3650m4-02-vm-03 ~]# ipa-restore /var/lib/ipa/backup/ipa-full-2014-11-10-03-53-26/
Directory Manager (existing master) password: 

Preparing restore from /var/lib/ipa/backup/ipa-full-2014-11-10-03-53-26/ on ibm-x3650m4-02-vm-03.testrelm.test
Restoring data will overwrite existing live data. Continue to restore? [no]: yes
Each master will individually need to be re-initialized or
re-created from this one. The replication agreements on
masters running IPA 3.1 or earlier will need to be manually
re-enabled. See the man page for details.
Disabling all replication.
Stopping IPA services
Restoring files
Restoring from userRoot in TESTRELM-TEST
Restoring from ipaca in TESTRELM-TEST
Starting IPA services
Command ''ipactl' 'start'' returned non-zero exit status 1
[root@ibm-x3650m4-02-vm-03 ~]# certutil -L -d /etc/httpd/alias/ -h 'System Trust'

Certificate Nickname                                         Trust Attributes
                                                             SSL,S/MIME,JAR/XPI

System Trust:TESTRELM.TEST IPA CA                            ,,   
[root@ibm-x3650m4-02-vm-03 ~]# rpm -q ipa-server 389-ds-base nss
ipa-server-4.1.0-3.el7.x86_64
389-ds-base-1.3.3.1-6.el7.x86_64
nss-3.16.2-9.el7.x86_64
[root@ibm-x3650m4-02-vm-03 ~]#
Comment 13 Jan Cholasta 2014-11-10 10:29:38 UTC 
OK, we found where the bug is, now we have to find out why it happens:

1) Do any of the files /etc/pki/ca-trust/source/ipa.p11-kit or /etc/pki/ca-trust/source/anchors/ipa-ca.crt exist before the restore? If they do, please attach them.

2) Do any of the files /etc/pki/ca-trust/source/ipa.p11-kit or /etc/pki/ca-trust/source/anchors/ipa-ca.crt exist after the restore? If they do, please attach them.

3) Does running update-ca-trust after the restore fix the problem?
Comment 14 Kaleem 2014-11-10 11:35:01 UTC 
Created attachment 955782 [details]
ipa.p11-kit file attached along with console log
Comment 15 Jan Cholasta 2014-11-10 11:43:29 UTC 
Could you please also try if running "update-ca-trust" fixes the problem, as requested in 3)?
Comment 16 Kaleem 2014-11-10 11:51:14 UTC 
Sorry. forgot to include this as i ran this with earlier setup. Again ran with new setup also. update-ca-trust not fixing the problem.

[root@ibm-x3650m4-02-vm-03 ~]# update-ca-trust 
[root@ibm-x3650m4-02-vm-03 ~]# systemctl status httpd.service
httpd.service - The Apache HTTP Server
   Loaded: loaded (/usr/lib/systemd/system/httpd.service; disabled)
   Active: failed (Result: exit-code) since Mon 2014-11-10 06:22:52 EST; 23min ago
...
...
[root@ibm-x3650m4-02-vm-03 ~]#

[root@ibm-x3650m4-02-vm-03 ~]# systemctl restart httpd.service
Job for httpd.service failed. See 'systemctl status httpd.service' and 'journalctl -xn' for details.
[root@ibm-x3650m4-02-vm-03 ~]#

[root@ibm-x3650m4-02-vm-03 ~]# systemctl status httpd.service
httpd.service - The Apache HTTP Server
   Loaded: loaded (/usr/lib/systemd/system/httpd.service; disabled)
   Active: failed (Result: exit-code) since Mon 2014-11-10 06:46:53 EST; 1s ago
..
...
[root@ibm-x3650m4-02-vm-03 ~]#

[root@ibm-x3650m4-02-vm-03 ~]# tail -n 8 /var/log/httpd/error_log 
[Mon Nov 10 06:22:52.503472 2014] [core:notice] [pid 8342] SELinux policy enabled; httpd running as context system_u:system_r:httpd_t:s0
[Mon Nov 10 06:22:52.504730 2014] [suexec:notice] [pid 8342] AH01232: suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Mon Nov 10 06:22:52.745698 2014] [:error] [pid 8342] SSL Library Error: -8172 Certificate is signed by an untrusted issuer
[Mon Nov 10 06:22:52.745725 2014] [:error] [pid 8342] Unable to verify certificate 'Server-Cert'. Add "NSSEnforceValidCerts off" to nss.conf so the server can start until the problem can be resolved.
[Mon Nov 10 06:46:53.195510 2014] [core:notice] [pid 8412] SELinux policy enabled; httpd running as context system_u:system_r:httpd_t:s0
[Mon Nov 10 06:46:53.196445 2014] [suexec:notice] [pid 8412] AH01232: suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Mon Nov 10 06:46:53.371430 2014] [:error] [pid 8412] SSL Library Error: -8172 Certificate is signed by an untrusted issuer
[Mon Nov 10 06:46:53.371464 2014] [:error] [pid 8412] Unable to verify certificate 'Server-Cert'. Add "NSSEnforceValidCerts off" to nss.conf so the server can start until the problem can be resolved.
[root@ibm-x3650m4-02-vm-03 ~]#
Comment 17 Jan Cholasta 2014-11-10 14:30:15 UTC 
The "TESTRELM.TEST IPA CA" cert in /etc/httpd/alias and /etc/pki/ca-trust/source/ipa.p11-kit do not match, which is why you are getting empty trust flags for it. This is because ipa.p11-kit is not backed up nor restored and thus wrong version of it is used after restore.
Comment 18 Jan Cholasta 2014-11-10 14:56:45 UTC 
Upstream ticket:
https://fedorahosted.org/freeipa/ticket/4711
Comment 20 Petr Viktorin (pviktori) 2014-11-11 15:36:53 UTC 
Fixed upstream
master:
https://fedorahosted.org/freeipa/changeset/2639997dfee43d66e94ef9b5441289816c465e7d
ipa-4-1:
https://fedorahosted.org/freeipa/changeset/7c2aad17da8bd5f50b9c1409f91c413bc454ce28
Comment 22 Kaleem 2014-11-24 11:24:43 UTC 
Verified.

IPA version:
------------
[root@dhcp207-1 ~]# rpm -q ipa-server
ipa-server-4.1.0-7.el7.x86_64
[root@dhcp207-1 ~]#

[root@dhcp207-1 ~]# ipa-restore -p xxxxxxxx -U /var/lib/ipa/backup/ipa-full-2014-11-24-16-26-33/
Preparing restore from /var/lib/ipa/backup/ipa-full-2014-11-24-16-26-33/ on dhcp207-1.testrelm.test
Each master will individually need to be re-initialized or
re-created from this one. The replication agreements on
masters running IPA 3.1 or earlier will need to be manually
re-enabled. See the man page for details.
Disabling all replication.
Stopping IPA services
Systemwide CA database updated.
Restoring files
Systemwide CA database updated.
Restoring from userRoot in TESTRELM-TEST
Restoring from ipaca in TESTRELM-TEST
Starting IPA services
Restarting SSSD
The ipa-restore command was successful
[root@dhcp207-1 ~]#
Comment 24 errata-xmlrpc 2015-03-05 10:14:15 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2015-0442.html