Red Hat Bugzilla – Bug 1159011
Trust setting not restored for CA cert with ipa-restore command
Last modified: 2015-03-05 05:14:15 EST
Description of problem: Trust settings of CA cert not restored correctly in nss db. This observed while execution of ipa-restore which restores /etc/httpd/alias nss db which was backuped earlier and it causes httpd service restart failure. Consequently ipa service restart fails. Version-Release number of selected component (if applicable): [root@dhcp207-187 ~]# rpm -q ipa-server nss-tools nss ipa-server-4.1.0-3.el7.x86_64 nss-tools-3.16.2-9.el7.x86_64 nss-3.16.2-9.el7.x86_64 [root@dhcp207-187 ~]# How reproducible: Always. Steps to Reproduce: 1. Please find the attached file (steps-with-console-output.txt) for detail. Actual results: Trust settings of CA cert not restored correctly. Expected results: Trust settings of CA cert should have been restored correctly Additional info:
======= Test machine ======= [root@dhcp207-187 alias]# certutil -L -d . Certificate Nickname Trust Attributes SSL,S/MIME,JAR/XPI Signing-Cert u,u,u ipaCert u,u,u Server-Cert u,u,u TESTRELM.TEST IPA CA ,, [root@dhcp207-187 alias]# rpm -q nss-tools nss-tools-3.16.2-9.el7.x86_64 [root@dhcp207-187 alias]# sha1sum * | sha1sum a6e1002f1f2da3e8ee822f32e796ef5cd5a326a7 - ======= My laptop ======= pviktori@dhcp-31-13:/tmp/tx/alias$ certutil -L -d . Certificate Nickname Trust Attributes SSL,S/MIME,JAR/XPI Signing-Cert u,u,u ipaCert u,u,u Server-Cert u,u,u TESTRELM.TEST IPA CA CT,C,C pviktori@dhcp-31-13:/tmp/tx/alias$ rpm -q nss-tools nss-tools-3.17.1-1.fc21.x86_64 pviktori@dhcp-31-13:/tmp/tx/alias$ sha1sum * | sha1sum a6e1002f1f2da3e8ee822f32e796ef5cd5a326a7 -
Kaleem, could you also attach the contents of /etc/httpd/alias from the affected machine? The database is the same (see the hashes above), but sometimes the trust flags are reported missing. It would be nice if NSS the folks could investigate.
[root@dhcp207-187 alias]# certutil -L -d . Certificate Nickname Trust Attributes SSL,S/MIME,JAR/XPI Signing-Cert u,u,u ipaCert u,u,u Server-Cert u,u,u TESTRELM.TEST IPA CA ,, [root@dhcp207-187 alias]# rpm -q nss-tools nss-tools-3.16.2-9.el7.x86_64 [root@dhcp207-187 alias]# sha1sum * | sha1sum a6e1002f1f2da3e8ee822f32e796ef5cd5a326a7 - [root@dhcp207-187 alias]#
I meant, add the contents of the /etc/httpd/alias directory as an attachment to the bug, so they can be examined.
Created attachment 953904 [details] contents of /etc/httpd/alias
I tested the attachment and on both my F20 and my RHEL-7.x (nss-3.16.2-9.el7.x86_64) I get: # certutil -L -d . Certificate Nickname Trust Attributes SSL,S/MIME,JAR/XPI Signing-Cert u,u,u ipaCert u,u,u Server-Cert u,u,u TESTRELM.TEST IPA CA CT,C,C Is this the right attachment or am I missing anything? Also CCing Kai in case he some idea what may be wrong. BTW, I think the hashes demonstrated in Comment 3 will not work well as the dir points to libnssckbi.so which can change with NSS version.
Kaleem, could you please post the output of "certutil -L -d /etc/httpd/alias/ -h 'System Trust'" before and after the restore?
[root@ibm-x3650m4-02-vm-03 ~]# certutil -L -d /etc/httpd/alias/ -h 'System Trust' Certificate Nickname Trust Attributes SSL,S/MIME,JAR/XPI System Trust:TESTRELM.TEST IPA CA CT,C,C [root@ibm-x3650m4-02-vm-03 ~]# ipa-restore /var/lib/ipa/backup/ipa-full-2014-11-10-03-53-26/ Directory Manager (existing master) password: Preparing restore from /var/lib/ipa/backup/ipa-full-2014-11-10-03-53-26/ on ibm-x3650m4-02-vm-03.testrelm.test Restoring data will overwrite existing live data. Continue to restore? [no]: yes Each master will individually need to be re-initialized or re-created from this one. The replication agreements on masters running IPA 3.1 or earlier will need to be manually re-enabled. See the man page for details. Disabling all replication. Stopping IPA services Restoring files Restoring from userRoot in TESTRELM-TEST Restoring from ipaca in TESTRELM-TEST Starting IPA services Command ''ipactl' 'start'' returned non-zero exit status 1 [root@ibm-x3650m4-02-vm-03 ~]# certutil -L -d /etc/httpd/alias/ -h 'System Trust' Certificate Nickname Trust Attributes SSL,S/MIME,JAR/XPI System Trust:TESTRELM.TEST IPA CA ,, [root@ibm-x3650m4-02-vm-03 ~]# rpm -q ipa-server 389-ds-base nss ipa-server-4.1.0-3.el7.x86_64 389-ds-base-1.3.3.1-6.el7.x86_64 nss-3.16.2-9.el7.x86_64 [root@ibm-x3650m4-02-vm-03 ~]#
OK, we found where the bug is, now we have to find out why it happens: 1) Do any of the files /etc/pki/ca-trust/source/ipa.p11-kit or /etc/pki/ca-trust/source/anchors/ipa-ca.crt exist before the restore? If they do, please attach them. 2) Do any of the files /etc/pki/ca-trust/source/ipa.p11-kit or /etc/pki/ca-trust/source/anchors/ipa-ca.crt exist after the restore? If they do, please attach them. 3) Does running update-ca-trust after the restore fix the problem?
Created attachment 955782 [details] ipa.p11-kit file attached along with console log
Could you please also try if running "update-ca-trust" fixes the problem, as requested in 3)?
Sorry. forgot to include this as i ran this with earlier setup. Again ran with new setup also. update-ca-trust not fixing the problem. [root@ibm-x3650m4-02-vm-03 ~]# update-ca-trust [root@ibm-x3650m4-02-vm-03 ~]# systemctl status httpd.service httpd.service - The Apache HTTP Server Loaded: loaded (/usr/lib/systemd/system/httpd.service; disabled) Active: failed (Result: exit-code) since Mon 2014-11-10 06:22:52 EST; 23min ago ... ... [root@ibm-x3650m4-02-vm-03 ~]# [root@ibm-x3650m4-02-vm-03 ~]# systemctl restart httpd.service Job for httpd.service failed. See 'systemctl status httpd.service' and 'journalctl -xn' for details. [root@ibm-x3650m4-02-vm-03 ~]# [root@ibm-x3650m4-02-vm-03 ~]# systemctl status httpd.service httpd.service - The Apache HTTP Server Loaded: loaded (/usr/lib/systemd/system/httpd.service; disabled) Active: failed (Result: exit-code) since Mon 2014-11-10 06:46:53 EST; 1s ago .. ... [root@ibm-x3650m4-02-vm-03 ~]# [root@ibm-x3650m4-02-vm-03 ~]# tail -n 8 /var/log/httpd/error_log [Mon Nov 10 06:22:52.503472 2014] [core:notice] [pid 8342] SELinux policy enabled; httpd running as context system_u:system_r:httpd_t:s0 [Mon Nov 10 06:22:52.504730 2014] [suexec:notice] [pid 8342] AH01232: suEXEC mechanism enabled (wrapper: /usr/sbin/suexec) [Mon Nov 10 06:22:52.745698 2014] [:error] [pid 8342] SSL Library Error: -8172 Certificate is signed by an untrusted issuer [Mon Nov 10 06:22:52.745725 2014] [:error] [pid 8342] Unable to verify certificate 'Server-Cert'. Add "NSSEnforceValidCerts off" to nss.conf so the server can start until the problem can be resolved. [Mon Nov 10 06:46:53.195510 2014] [core:notice] [pid 8412] SELinux policy enabled; httpd running as context system_u:system_r:httpd_t:s0 [Mon Nov 10 06:46:53.196445 2014] [suexec:notice] [pid 8412] AH01232: suEXEC mechanism enabled (wrapper: /usr/sbin/suexec) [Mon Nov 10 06:46:53.371430 2014] [:error] [pid 8412] SSL Library Error: -8172 Certificate is signed by an untrusted issuer [Mon Nov 10 06:46:53.371464 2014] [:error] [pid 8412] Unable to verify certificate 'Server-Cert'. Add "NSSEnforceValidCerts off" to nss.conf so the server can start until the problem can be resolved. [root@ibm-x3650m4-02-vm-03 ~]#
The "TESTRELM.TEST IPA CA" cert in /etc/httpd/alias and /etc/pki/ca-trust/source/ipa.p11-kit do not match, which is why you are getting empty trust flags for it. This is because ipa.p11-kit is not backed up nor restored and thus wrong version of it is used after restore.
Upstream ticket: https://fedorahosted.org/freeipa/ticket/4711
Fixed upstream master: https://fedorahosted.org/freeipa/changeset/2639997dfee43d66e94ef9b5441289816c465e7d ipa-4-1: https://fedorahosted.org/freeipa/changeset/7c2aad17da8bd5f50b9c1409f91c413bc454ce28
Verified. IPA version: ------------ [root@dhcp207-1 ~]# rpm -q ipa-server ipa-server-4.1.0-7.el7.x86_64 [root@dhcp207-1 ~]# [root@dhcp207-1 ~]# ipa-restore -p xxxxxxxx -U /var/lib/ipa/backup/ipa-full-2014-11-24-16-26-33/ Preparing restore from /var/lib/ipa/backup/ipa-full-2014-11-24-16-26-33/ on dhcp207-1.testrelm.test Each master will individually need to be re-initialized or re-created from this one. The replication agreements on masters running IPA 3.1 or earlier will need to be manually re-enabled. See the man page for details. Disabling all replication. Stopping IPA services Systemwide CA database updated. Restoring files Systemwide CA database updated. Restoring from userRoot in TESTRELM-TEST Restoring from ipaca in TESTRELM-TEST Starting IPA services Restarting SSSD The ipa-restore command was successful [root@dhcp207-1 ~]#
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHSA-2015-0442.html