Bug 1481388 – IPA certificate auto renewal failed at unable to connect to LDAP server with authentication failure

Bugzilla will be upgraded to version 5.0. The upgrade date is tentatively scheduled for 2 December 2018, pending final testing and feedback.

Bug 1481388 - IPA certificate auto renewal failed at unable to connect to LDAP server with authentication failure

Summary:	IPA certificate auto renewal failed at unable to connect to LDAP server with ...

Status:	CLOSED ERRATA

Aliases:	None

Product:	Red Hat Enterprise Linux 7
Classification:	Red Hat
Component:	selinux-policy (Show other bugs)
Sub Component:
Version:	7.4
Hardware:	x86_64 Linux

Priority	high Severity high
Target Milestone:	rc
Target Release:	---
Assigned To:	Lukas Vrabec
QA Contact:	Milos Malik
Docs Contact:

URL:
Whiteboard:
Keywords:	ZStream

Depends On:
Blocks:	1486552 1515503
	Show dependency tree / graph

Reported:	2017-08-14 15:48 EDT by Xiyang Dong
Modified:	2018-04-10 08:40 EDT (History)
CC List:	13 users (show)

See Also:
Fixed In Version:	selinux-policy-3.13.1-172.el7
Doc Type:	Bug Fix
Doc Text:	Previously, the selinux-policy package missed certain rules related to Identity Management (IdM). As a consequence, the IPA certificate auto-renewal procedure failed and the Tomcat service was not able to connect to the LDAP server. This update adds the SELinux allow rules that enable IdM to auto-renew certificates when running SELinux in enforcing mode, and the described problem no longer occurs.
Story Points:	---
Clone Of:
Clones:	1486552 1515503 (view as bug list)
Environment:
Last Closed:	2018-04-10 08:38:21 EDT
Type:	Bug
Regression:	---
Mount Type:	---
Documentation:	---
CRM:
Verified Versions:
Category:	---
oVirt Team:	---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---

Attachments	(Terms of Use)
/var/log/pki/pki-tomcat/ca/debug (11.17 MB, text/plain) 2017-08-14 15:59 EDT, Xiyang Dong	no flags	Details
/var/log/audit/audit.log (100.50 KB, text/plain) 2017-08-14 17:06 EDT, Xiyang Dong	no flags	Details
/var/log/messages (2.51 MB, text/plain) 2017-08-14 17:07 EDT, Xiyang Dong	no flags	Details
/var/log/pki/pki-tomcat/ca/debug_Z-stream (10.97 MB, text/plain) 2017-09-24 22:14 EDT, Xiyang Dong	no flags	Details
/var/log/audit/audit.log_Z-stream (72.76 KB, text/plain) 2017-09-24 22:17 EDT, Xiyang Dong	no flags	Details
/var/log/messages_Z-stream (634.21 KB, text/plain) 2017-09-24 22:19 EDT, Xiyang Dong	no flags	Details
audit2why_Z-stream (7.22 KB, text/plain) 2017-09-24 22:28 EDT, Xiyang Dong	no flags	Details
ausearch_Z-stream (5.97 KB, text/plain) 2017-09-24 22:30 EDT, Xiyang Dong	no flags	Details
Add an attachment (proposed patch, testcase, etc.)

External Trackers
Tracker	ID	Priority	Status	Summary	Last Updated
Red Hat Product Errata	RHBA-2018:0763	None	None	None	2018-04-10 08:40 EDT

Groups: None (edit)

Description Xiyang Dong 2017-08-14 15:48:07 EDT

Description of problem:
IPA certificate auto renewal failed at ca-error: 
ca-error: Server at "http://bkr-hv03-guest23.testrelm.test:8080/ca/ee/ca/profileSubmit" replied: Failed to connect LDAP server Could not connect to LDAP server host bkr-hv03-guest23.testrelm.test port 636 Error netscape.ldap.LDAPException: Authentication failed (48)

Version-Release number of selected component (if applicable):
# rpm -q ipa-server
ipa-server-4.5.0-21.el7_4.1.x86_64

How reproducible:
Always

Steps to Reproduce:
1. Install ipa server
2. Change date prior to cert expire
3. Check cert renewal status

Actual results:
Cert autorenewal failed

Expected results:
Cert autorenewal passes

Additional info:
# date
Mon Aug 14 15:06:01 EDT 2017
# getcert list | egrep "status|expires|Request|subject|ca-error"
Request ID '20170814185636':
	status: MONITORING
	subject: CN=IPA RA,O=TESTRELM.TEST
	expires: 2019-08-04 18:56:40 UTC
Request ID '20170814185733':
	status: MONITORING
	subject: CN=CA Audit,O=TESTRELM.TEST
	expires: 2019-08-04 18:56:19 UTC
Request ID '20170814185734':
	status: MONITORING
	subject: CN=OCSP Subsystem,O=TESTRELM.TEST
	expires: 2019-08-04 18:56:18 UTC
Request ID '20170814185735':
	status: MONITORING
	subject: CN=CA Subsystem,O=TESTRELM.TEST
	expires: 2019-08-04 18:56:18 UTC
Request ID '20170814185736':
	status: MONITORING
	subject: CN=Certificate Authority,O=TESTRELM.TEST
	expires: 2037-08-14 18:56:18 UTC
Request ID '20170814185737':
	status: MONITORING
	subject: CN=auto-hv-01-guest01.testrelm.test,O=TESTRELM.TEST
	expires: 2019-08-04 18:56:18 UTC
Request ID '20170814185802':
	status: MONITORING
	subject: CN=auto-hv-01-guest01.testrelm.test,O=TESTRELM.TEST
	expires: 2019-08-15 18:58:03 UTC
Request ID '20170814185827':
	status: MONITORING
	subject: CN=auto-hv-01-guest01.testrelm.test,O=TESTRELM.TEST
	expires: 2019-08-15 18:58:28 UTC
Request ID '20170814185839':
	status: MONITORING
	subject: CN=auto-hv-01-guest01.testrelm.test,O=TESTRELM.TEST
	expires: 2019-08-15 18:58:40 UTC
# date -s "715 days";sleep 300;getcert list | egrep "status|expires|Request|subject|ca-error"
Tue Jul 30 15:07:04 EDT 2019
Request ID '20170814185636':
	status: MONITORING
	subject: CN=IPA RA,O=TESTRELM.TEST
	expires: 2021-07-19 19:10:30 UTC
Request ID '20170814185733':
	status: MONITORING
	subject: CN=CA Audit,O=TESTRELM.TEST
	expires: 2021-07-19 19:11:30 UTC
Request ID '20170814185734':
	status: MONITORING
	subject: CN=OCSP Subsystem,O=TESTRELM.TEST
	expires: 2021-07-19 19:11:19 UTC
Request ID '20170814185735':
	status: MONITORING
	subject: CN=CA Subsystem,O=TESTRELM.TEST
	expires: 2021-07-19 19:10:49 UTC
Request ID '20170814185736':
	status: MONITORING
	subject: CN=Certificate Authority,O=TESTRELM.TEST
	expires: 2037-08-14 18:56:18 UTC
Request ID '20170814185737':
	status: MONITORING
	subject: CN=auto-hv-01-guest01.testrelm.test,O=TESTRELM.TEST
	expires: 2021-07-19 19:10:29 UTC
Request ID '20170814185802':
	status: MONITORING
	subject: CN=auto-hv-01-guest01.testrelm.test,O=TESTRELM.TEST
	expires: 2021-07-30 19:10:27 UTC
Request ID '20170814185827':
	status: MONITORING
	subject: CN=auto-hv-01-guest01.testrelm.test,O=TESTRELM.TEST
	expires: 2021-07-30 19:10:19 UTC
Request ID '20170814185839':
	status: MONITORING
	subject: CN=auto-hv-01-guest01.testrelm.test,O=TESTRELM.TEST
	expires: 2021-07-30 19:10:13 UTC
# date -s "715 days";sleep 300;getcert list | egrep "status|expires|Request|subject|ca-error"
Wed Jul 14 15:15:23 EDT 2021
Request ID '20170814185636':
	status: MONITORING
	ca-error: Server at "http://auto-hv-01-guest01.testrelm.test:8080/ca/ee/ca/profileSubmit" replied: Failed to connect LDAP server Could not connect to LDAP server host auto-hv-01-guest01.testrelm.test port 636 Error netscape.ldap.LDAPException: Authentication failed (48)
	subject: CN=IPA RA,O=TESTRELM.TEST
	expires: 2021-07-19 19:10:30 UTC
Request ID '20170814185733':
	status: MONITORING
	ca-error: Server at "http://auto-hv-01-guest01.testrelm.test:8080/ca/ee/ca/profileSubmit" replied: Failed to connect LDAP server Could not connect to LDAP server host auto-hv-01-guest01.testrelm.test port 636 Error netscape.ldap.LDAPException: Authentication failed (48)
	subject: CN=CA Audit,O=TESTRELM.TEST
	expires: 2021-07-19 19:11:30 UTC
Request ID '20170814185734':
	status: MONITORING
	ca-error: Server at "http://auto-hv-01-guest01.testrelm.test:8080/ca/ee/ca/profileSubmit" replied: Failed to connect LDAP server Could not connect to LDAP server host auto-hv-01-guest01.testrelm.test port 636 Error netscape.ldap.LDAPException: Authentication failed (48)
	subject: CN=OCSP Subsystem,O=TESTRELM.TEST
	expires: 2021-07-19 19:11:19 UTC
Request ID '20170814185735':
	status: MONITORING
	ca-error: Server at "http://auto-hv-01-guest01.testrelm.test:8080/ca/ee/ca/profileSubmit" replied: Failed to connect LDAP server Could not connect to LDAP server host auto-hv-01-guest01.testrelm.test port 636 Error netscape.ldap.LDAPException: Authentication failed (48)
	subject: CN=CA Subsystem,O=TESTRELM.TEST
	expires: 2021-07-19 19:10:49 UTC
Request ID '20170814185736':
	status: MONITORING
	subject: CN=Certificate Authority,O=TESTRELM.TEST
	expires: 2037-08-14 18:56:18 UTC
Request ID '20170814185737':
	status: MONITORING
	ca-error: Server at "http://auto-hv-01-guest01.testrelm.test:8080/ca/ee/ca/profileSubmit" replied: Failed to connect LDAP server Could not connect to LDAP server host auto-hv-01-guest01.testrelm.test port 636 Error netscape.ldap.LDAPException: Authentication failed (48)
	subject: CN=auto-hv-01-guest01.testrelm.test,O=TESTRELM.TEST
	expires: 2021-07-19 19:10:29 UTC
Request ID '20170814185802':
	status: MONITORING
	subject: CN=auto-hv-01-guest01.testrelm.test,O=TESTRELM.TEST
	expires: 2023-07-15 19:20:36 UTC
Request ID '20170814185827':
	status: CA_UNREACHABLE
	ca-error: Server at https://auto-hv-01-guest01.testrelm.test/ipa/xml failed request, will retry: 4016 (RPC failed at server.  Failed to authenticate to CA REST API).
	subject: CN=auto-hv-01-guest01.testrelm.test,O=TESTRELM.TEST
	expires: 2021-07-30 19:10:19 UTC
Request ID '20170814185839':
	status: MONITORING
	subject: CN=auto-hv-01-guest01.testrelm.test,O=TESTRELM.TEST
	expires: 2023-07-15 19:20:28 UTC

Comment 2 Xiyang Dong 2017-08-14 15:59 EDT

Created attachment 1313327 [details]
/var/log/pki/pki-tomcat/ca/debug

Comment 3 Rob Crittenden 2017-08-14 16:01:45 EDT

I'd check SELinux output and the journal to be sure that during the first renewal the certificates were updated properly in o=ipaca

Were I to guess the RA agent cert is out-of-sync with what is in LDAP.

A few improvement suggestions to the test:

- print date again after the sleep(300)
- if you can, kinit admin && ipa cert-show 1 to exercise the IPA <-> dogtag connection

Comment 4 Xiyang Dong 2017-08-14 16:57:31 EDT

[root@auto-hv-01-guest01 ~]# date
Mon Aug 14 16:42:02 EDT 2017
[root@auto-hv-01-guest01 ~]# kinit admin
Password for admin@TESTRELM.TEST: 
[root@auto-hv-01-guest01 ~]# ipa cert-show 1
  Issuing CA: ipa
  Certificate: MIIDkTCCAnmgAwIBAgIBATANBgkqhkiG9w0BAQsFADA4MRYwFAYDVQQKDA1URVNUUkVMTS5URVNUMR4wHAYDVQQDDBVDZXJ0aWZpY2F0ZSBBdXRob3JpdHkwHhcNMTcwODE0MjAyNjAwWhcNMzcwODE0MjAyNjAwWjA4MRYwFAYDVQQKDA1URVNUUkVMTS5URVNUMR4wHAYDVQQDDBVDZXJ0aWZpY2F0ZSBBdXRob3JpdHkwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQC3PMBLxaNZPt0i2jBX7ya3HBcT+L5aQPvDoRBSXahoVvPGkB1QHHEF6LRgIMVKfiCfjR0UgtzjHScNa5qjq7iMQyS2iGYfXDUtWuhOuU/FMv78VpUJbjtlhsyeSaO44pJakaMYQ30EOEXAXtdKPqtUvgOqEOfCwG7D95PUysXiYvR/jYC0CJhC80m3+46+9FuSsDb2kPixinXZJ9snqHn8g+0WBHaYLk9gUhC17kzrZyZgENPJeRtK8rMU274YDUA7337hfAB079sV86jfko9pt0Q5yIpXsD75p6Sk4pX5d70VL+erMcWKYE2pZ2rMt/dBAi/8ZK2Mucdg9hoOOFApAgMBAAGjgaUwgaIwHwYDVR0jBBgwFoAUkZ/t+MOYNgO92/ICn8fDs6rN7XUwDwYDVR0TAQH/BAUwAwEB/zAOBgNVHQ8BAf8EBAMCAcYwHQYDVR0OBBYEFJGf7fjDmDYDvdvyAp/Hw7Oqze11MD8GCCsGAQUFBwEBBDMwMTAvBggrBgEFBQcwAYYjaHR0cDovL2lwYS1jYS50ZXN0cmVsbS50ZXN0L2NhL29jc3AwDQYJKoZIhvcNAQELBQADggEBAK5VDKvnhHUK2CKUt/XQD/VlthKcASFuCa9vNfUhdqn0aeLC6lrbV/LzWSSlnMyiwfoCY/Xerc6YDmejyyJCL08NSobfcyUtYpDsun5REBlpnewCw/9riQtaDUfxXgNyv1mu6EjTSYNN3QOQywRRrwQulMnzk64gsPjWwdCinQ34cX7QayquWmEbpip9fV9FpSxnA1Cjeqh23+YAgd+XRWAAvjaiEMiMUgEkQKX9ANsgxVUy18DnnHPVshGCNLVqs6MV7Uc43yfrn2UCPke9V190qvWAhyeou4rsUU7AiPmtWPyyvJOpTl1gh1Jb1AuKTXhPAyCPhZh+QtXXfI5tpKA=
  Subject: CN=Certificate Authority,O=TESTRELM.TEST
  Issuer: CN=Certificate Authority,O=TESTRELM.TEST
  Not Before: Mon Aug 14 20:26:00 2017 UTC
  Not After: Fri Aug 14 20:26:00 2037 UTC
  Serial number: 1
  Serial number (hex): 0x1
  Revoked: False
[root@auto-hv-01-guest01 ~]# getcert list | egrep "status|expires|Request|subject|ca-error"
Request ID '20170814202634':
	status: MONITORING
	subject: CN=IPA RA,O=TESTRELM.TEST
	expires: 2019-08-04 20:26:36 UTC
Request ID '20170814202739':
	status: MONITORING
	subject: CN=CA Audit,O=TESTRELM.TEST
	expires: 2019-08-04 20:26:05 UTC
Request ID '20170814202740':
	status: MONITORING
	subject: CN=OCSP Subsystem,O=TESTRELM.TEST
	expires: 2019-08-04 20:26:03 UTC
Request ID '20170814202741':
	status: MONITORING
	subject: CN=CA Subsystem,O=TESTRELM.TEST
	expires: 2019-08-04 20:26:04 UTC
Request ID '20170814202742':
	status: MONITORING
	subject: CN=Certificate Authority,O=TESTRELM.TEST
	expires: 2037-08-14 20:26:00 UTC
Request ID '20170814202743':
	status: MONITORING
	subject: CN=auto-hv-01-guest01.testrelm.test,O=TESTRELM.TEST
	expires: 2019-08-04 20:26:03 UTC
Request ID '20170814202821':
	status: MONITORING
	subject: CN=auto-hv-01-guest01.testrelm.test,O=TESTRELM.TEST
	expires: 2019-08-15 20:28:22 UTC
Request ID '20170814202911':
	status: MONITORING
	subject: CN=auto-hv-01-guest01.testrelm.test,O=TESTRELM.TEST
	expires: 2019-08-15 20:29:12 UTC
Request ID '20170814202927':
	status: MONITORING
	subject: CN=auto-hv-01-guest01.testrelm.test,O=TESTRELM.TEST
	expires: 2019-08-15 20:29:28 UTC
[root@auto-hv-01-guest01 ~]# date -s "715 days";sleep 300;date
Tue Jul 30 16:43:24 EDT 2019
Tue Jul 30 16:48:24 EDT 2019
[root@auto-hv-01-guest01 ~]# getcert list | egrep "status|expires|Request|subject|ca-error"
Request ID '20170814202634':
	status: MONITORING
	subject: CN=IPA RA,O=TESTRELM.TEST
	expires: 2021-07-19 20:46:14 UTC
Request ID '20170814202739':
	status: MONITORING
	subject: CN=CA Audit,O=TESTRELM.TEST
	expires: 2021-07-19 20:46:55 UTC
Request ID '20170814202740':
	status: MONITORING
	subject: CN=OCSP Subsystem,O=TESTRELM.TEST
	expires: 2021-07-19 20:46:04 UTC
Request ID '20170814202741':
	status: MONITORING
	subject: CN=CA Subsystem,O=TESTRELM.TEST
	expires: 2021-07-19 20:46:24 UTC
Request ID '20170814202742':
	status: MONITORING
	subject: CN=Certificate Authority,O=TESTRELM.TEST
	expires: 2037-08-14 20:26:00 UTC
Request ID '20170814202743':
	status: MONITORING
	subject: CN=auto-hv-01-guest01.testrelm.test,O=TESTRELM.TEST
	expires: 2021-07-19 20:45:54 UTC
Request ID '20170814202821':
	status: MONITORING
	subject: CN=auto-hv-01-guest01.testrelm.test,O=TESTRELM.TEST
	expires: 2021-07-30 20:45:59 UTC
Request ID '20170814202911':
	status: MONITORING
	subject: CN=auto-hv-01-guest01.testrelm.test,O=TESTRELM.TEST
	expires: 2021-07-30 20:45:50 UTC
Request ID '20170814202927':
	status: MONITORING
	subject: CN=auto-hv-01-guest01.testrelm.test,O=TESTRELM.TEST
	expires: 2021-07-30 20:45:40 UTC
[root@auto-hv-01-guest01 ~]# kinit admin
Password for admin@TESTRELM.TEST: 
Password expired.  You must change it now.
Enter new password: 
Enter it again: 
[root@auto-hv-01-guest01 ~]# ipa cert-show 1
  Issuing CA: ipa
  Certificate: MIIDkTCCAnmgAwIBAgIBATANBgkqhkiG9w0BAQsFADA4MRYwFAYDVQQKDA1URVNUUkVMTS5URVNUMR4wHAYDVQQDDBVDZXJ0aWZpY2F0ZSBBdXRob3JpdHkwHhcNMTcwODE0MjAyNjAwWhcNMzcwODE0MjAyNjAwWjA4MRYwFAYDVQQKDA1URVNUUkVMTS5URVNUMR4wHAYDVQQDDBVDZXJ0aWZpY2F0ZSBBdXRob3JpdHkwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQC3PMBLxaNZPt0i2jBX7ya3HBcT+L5aQPvDoRBSXahoVvPGkB1QHHEF6LRgIMVKfiCfjR0UgtzjHScNa5qjq7iMQyS2iGYfXDUtWuhOuU/FMv78VpUJbjtlhsyeSaO44pJakaMYQ30EOEXAXtdKPqtUvgOqEOfCwG7D95PUysXiYvR/jYC0CJhC80m3+46+9FuSsDb2kPixinXZJ9snqHn8g+0WBHaYLk9gUhC17kzrZyZgENPJeRtK8rMU274YDUA7337hfAB079sV86jfko9pt0Q5yIpXsD75p6Sk4pX5d70VL+erMcWKYE2pZ2rMt/dBAi/8ZK2Mucdg9hoOOFApAgMBAAGjgaUwgaIwHwYDVR0jBBgwFoAUkZ/t+MOYNgO92/ICn8fDs6rN7XUwDwYDVR0TAQH/BAUwAwEB/zAOBgNVHQ8BAf8EBAMCAcYwHQYDVR0OBBYEFJGf7fjDmDYDvdvyAp/Hw7Oqze11MD8GCCsGAQUFBwEBBDMwMTAvBggrBgEFBQcwAYYjaHR0cDovL2lwYS1jYS50ZXN0cmVsbS50ZXN0L2NhL29jc3AwDQYJKoZIhvcNAQELBQADggEBAK5VDKvnhHUK2CKUt/XQD/VlthKcASFuCa9vNfUhdqn0aeLC6lrbV/LzWSSlnMyiwfoCY/Xerc6YDmejyyJCL08NSobfcyUtYpDsun5REBlpnewCw/9riQtaDUfxXgNyv1mu6EjTSYNN3QOQywRRrwQulMnzk64gsPjWwdCinQ34cX7QayquWmEbpip9fV9FpSxnA1Cjeqh23+YAgd+XRWAAvjaiEMiMUgEkQKX9ANsgxVUy18DnnHPVshGCNLVqs6MV7Uc43yfrn2UCPke9V190qvWAhyeou4rsUU7AiPmtWPyyvJOpTl1gh1Jb1AuKTXhPAyCPhZh+QtXXfI5tpKA=
  Subject: CN=Certificate Authority,O=TESTRELM.TEST
  Issuer: CN=Certificate Authority,O=TESTRELM.TEST
  Not Before: Mon Aug 14 20:26:00 2017 UTC
  Not After: Fri Aug 14 20:26:00 2037 UTC
  Serial number: 1
  Serial number (hex): 0x1
  Revoked: False
[root@auto-hv-01-guest01 ~]# date -s "715 days";sleep 300;date
Wed Jul 14 16:49:21 EDT 2021
Wed Jul 14 16:54:21 EDT 2021
[root@auto-hv-01-guest01 ~]# getcert list | egrep "status|expires|Request|subject|ca-error"
Request ID '20170814202634':
	status: MONITORING
	ca-error: Server at "http://auto-hv-01-guest01.testrelm.test:8080/ca/ee/ca/profileSubmit" replied: Failed to connect LDAP server Could not connect to LDAP server host auto-hv-01-guest01.testrelm.test port 636 Error netscape.ldap.LDAPException: Authentication failed (48)
	subject: CN=IPA RA,O=TESTRELM.TEST
	expires: 2021-07-19 20:46:14 UTC
Request ID '20170814202739':
	status: MONITORING
	ca-error: Server at "http://auto-hv-01-guest01.testrelm.test:8080/ca/ee/ca/profileSubmit" replied: Failed to connect LDAP server Could not connect to LDAP server host auto-hv-01-guest01.testrelm.test port 636 Error netscape.ldap.LDAPException: Authentication failed (48)
	subject: CN=CA Audit,O=TESTRELM.TEST
	expires: 2021-07-19 20:46:55 UTC
Request ID '20170814202740':
	status: MONITORING
	ca-error: Server at "http://auto-hv-01-guest01.testrelm.test:8080/ca/ee/ca/profileSubmit" replied: Failed to connect LDAP server Could not connect to LDAP server host auto-hv-01-guest01.testrelm.test port 636 Error netscape.ldap.LDAPException: Authentication failed (48)
	subject: CN=OCSP Subsystem,O=TESTRELM.TEST
	expires: 2021-07-19 20:46:04 UTC
Request ID '20170814202741':
	status: MONITORING
	ca-error: Server at "http://auto-hv-01-guest01.testrelm.test:8080/ca/ee/ca/profileSubmit" replied: Failed to connect LDAP server Could not connect to LDAP server host auto-hv-01-guest01.testrelm.test port 636 Error netscape.ldap.LDAPException: Authentication failed (48)
	subject: CN=CA Subsystem,O=TESTRELM.TEST
	expires: 2021-07-19 20:46:24 UTC
Request ID '20170814202742':
	status: MONITORING
	subject: CN=Certificate Authority,O=TESTRELM.TEST
	expires: 2037-08-14 20:26:00 UTC
Request ID '20170814202743':
	status: MONITORING
	ca-error: Server at "http://auto-hv-01-guest01.testrelm.test:8080/ca/ee/ca/profileSubmit" replied: Failed to connect LDAP server Could not connect to LDAP server host auto-hv-01-guest01.testrelm.test port 636 Error netscape.ldap.LDAPException: Authentication failed (48)
	subject: CN=auto-hv-01-guest01.testrelm.test,O=TESTRELM.TEST
	expires: 2021-07-19 20:45:54 UTC
Request ID '20170814202821':
	status: MONITORING
	subject: CN=auto-hv-01-guest01.testrelm.test,O=TESTRELM.TEST
	expires: 2023-07-15 20:53:47 UTC
Request ID '20170814202911':
	status: MONITORING
	subject: CN=auto-hv-01-guest01.testrelm.test,O=TESTRELM.TEST
	expires: 2023-07-15 20:53:37 UTC
Request ID '20170814202927':
	status: MONITORING
	subject: CN=auto-hv-01-guest01.testrelm.test,O=TESTRELM.TEST
	expires: 2023-07-15 20:53:29 UTC
[root@auto-hv-01-guest01 ~]# kinit admin
Password for admin@TESTRELM.TEST: 
Password expired.  You must change it now.
Enter new password: 
Enter it again: 
[root@auto-hv-01-guest01 ~]# ipa cert-show 1
ipa: ERROR: Certificate operation cannot be completed: EXCEPTION (Invalid Credential.)

Comment 5 Xiyang Dong 2017-08-14 17:06 EDT

Created attachment 1313341 [details]
/var/log/audit/audit.log

Comment 6 Xiyang Dong 2017-08-14 17:07 EDT

Created attachment 1313342 [details]
/var/log/messages

Comment 7 Rob Crittenden 2017-08-14 17:17:23 EDT

Turns out my request for the add'l date command was not needed because I misread things originally :-(

The cert-show clearly demonstrates that the RA cert works after the first renewal but not the second.

Is this a regression or a new test?

From the messages file it looks like tomcat is having issues. I'm not sure if that is related or not

Jul 30 15:11:38 localhost renew_ca_cert: Traceback (most recent call last):
 File "/usr/libexec/ipa/certmonger/renew_ca_cert", line 218, in <module>
    main()
  File "/usr/libexec/ipa/certmonger/renew_ca_cert", line 212, in main
    _main()
  File "/usr/libexec/ipa/certmonger/renew_ca_cert", line 82, in _main
    ca.update_cert_config(nickname, cert)
  File "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", line 1156, in update_cert_config
    directives[nickname], cert, paths.CA_CS_CFG_PATH)
  File "/usr/lib/python2.7/site-packages/ipaserver/install/dogtaginstance.py", line 360, in update_cert_cs_cfg
    with stopped_service('pki-tomcatd', 'pki-tomcat'):
  File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "/usr/lib/python2.7/site-packages/ipaserver/install/installutils.py", line 1129, in stopped_service
    service_obj.stop(instance_name)
  File "/usr/lib/python2.7/site-packages/ipaplatform/base/services.py", line 283, in stop
    ipautil.run(args, skip_output=not capture_output)
  File "/usr/lib/python2.7/site-packages/ipapython/ipautil.py", line 511, in run
    raise CalledProcessError(p.returncode, arg_string, str(output))
CalledProcessError: Command '/bin/systemctl stop pki-tomcatd@pki-tomcat.service' returned non-zero exit status 4

Sadly I don't see anything in debug that echo's this failure at the same time.

Comment 21 Xiyang Dong 2017-09-24 22:14 EDT

Created attachment 1330340 [details]
/var/log/pki/pki-tomcat/ca/debug_Z-stream

Comment 22 Xiyang Dong 2017-09-24 22:17 EDT

Created attachment 1330341 [details]
/var/log/audit/audit.log_Z-stream

Comment 23 Xiyang Dong 2017-09-24 22:19 EDT

Created attachment 1330342 [details]
/var/log/messages_Z-stream

Comment 24 Xiyang Dong 2017-09-24 22:28 EDT

Created attachment 1330343 [details]
audit2why_Z-stream

Comment 25 Xiyang Dong 2017-09-24 22:30 EDT

Created attachment 1330344 [details]
ausearch_Z-stream

Comment 28 Xiyang Dong 2017-09-25 17:25:45 EDT

Thanks Florence and Lukas.
Verified on # rpm -qa ipa-server certmonger selinux-policy 
certmonger-0.78.4-3.el7.x86_64
ipa-server-4.5.0-20.el7.x86_64
selinux-policy-3.13.1-172.el7.noarch
# date
Mon Sep 25 17:03:14 EDT 2017
# kinit admin
Password for admin@TESTRELM.TEST:
# ipa cert-show 1
  Issuing CA: ipa
  Certificate: MIIDkTCCAnmgAwIBAgIBATANBgkqhkiG9w0BAQsFADA4MRYwFAYDVQQKDA1URVNUUkVMTS5URVNUMR4wHAYDVQQDDBVDZXJ0aWZpY2F0ZSBBdXRob3JpdHkwHhcNMTcwOTI1MTM1NDAxWhcNMzcwOTI1MTM1NDAxWjA4MRYwFAYDVQQKDA1URVNUUkVMTS5URVNUMR4wHAYDVQQDDBVDZXJ0aWZpY2F0ZSBBdXRob3JpdHkwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDY2jv3bFDdhyYEO9YlLnaTPcK2GAZcWSfamgowWio024UvfhjOb0jndM0ko6HdFCWTXTHvONylTE/HCseqCGNIMIw970CLPp7hbuZ/TMm2PIIbbXqzBq3r4bk1JJTISVdxKs4lnyTiJihH3cDlzLMoWu7DxPHP0yoywSDWdyyljrbKuO7Lu04Z3ZnSvHelbzRUdCNEyPyAnAA941J44m/jwVg4xQZ8R/M24HpVtlB5Oi15PNL21W2W7cFQ01d4s+c/Ab7kOBv37r6zxHAfBdV8hKpIlpcV1rS1mTy2wKGU33y+YeQVL1p+Yjoq4M0G7mpojGeFdRc9/1VZFyroNLyLAgMBAAGjgaUwgaIwHwYDVR0jBBgwFoAUR9v2Y1VesiOALKfJWj3qDlSwGVMwDwYDVR0TAQH/BAUwAwEB/zAOBgNVHQ8BAf8EBAMCAcYwHQYDVR0OBBYEFEfb9mNVXrIjgCynyVo96g5UsBlTMD8GCCsGAQUFBwEBBDMwMTAvBggrBgEFBQcwAYYjaHR0cDovL2lwYS1jYS50ZXN0cmVsbS50ZXN0L2NhL29jc3AwDQYJKoZIhvcNAQELBQADggEBAMPZkWas54cMhbWK+RXh7DbR2lR3V1AFMatra4Gw7qtfowSa1SCoxrwNuBpGnI1oxgWcVMk1Gbqe1cT85DBdjEnRk0vyJ/Ja5fIrLA0TvP/41dnN+IveGfL/k1xlpdRf+yVex5uJBs0eTs4Ohqf6So3AqLgru5ELiOjU8PerjFDp9J3vCVXc60nUFWIgt5pFvxkfJDAC23+eF/tZkfV2MncM3tXcpyC8ulkaZP2HwhUEa6MQ4tVhmabGYotQE5XAGhlKGdq5BV4m0KdZnrR0YTHuec5szw+SxfHea7zwpL1DzIrfuemSvwNjR3rso5MmD/pcxmWeGdzc9c+H+ysG0oU=
  Subject: CN=Certificate Authority,O=TESTRELM.TEST
  Issuer: CN=Certificate Authority,O=TESTRELM.TEST
  Not Before: Mon Sep 25 13:54:01 2017 UTC
  Not After: Fri Sep 25 13:54:01 2037 UTC
  Serial number: 1
  Serial number (hex): 0x1
  Revoked: False
# getcert list | egrep "status|expires|Request|subject|ca-error"
Request ID '20170925135424':
	status: MONITORING
	subject: CN=IPA RA,O=TESTRELM.TEST
	expires: 2019-09-15 13:54:25 UTC
Request ID '20170925135504':
	status: MONITORING
	subject: CN=CA Audit,O=TESTRELM.TEST
	expires: 2019-09-15 13:54:04 UTC
Request ID '20170925135505':
	status: MONITORING
	subject: CN=OCSP Subsystem,O=TESTRELM.TEST
	expires: 2019-09-15 13:54:03 UTC
Request ID '20170925135506':
	status: MONITORING
	subject: CN=CA Subsystem,O=TESTRELM.TEST
	expires: 2019-09-15 13:54:04 UTC
Request ID '20170925135507':
	status: MONITORING
	subject: CN=Certificate Authority,O=TESTRELM.TEST
	expires: 2037-09-25 13:54:01 UTC
Request ID '20170925135508':
	status: MONITORING
	subject: CN=mgmt6.testrelm.test,O=TESTRELM.TEST
	expires: 2019-09-15 13:54:03 UTC
Request ID '20170925135538':
	status: MONITORING
	subject: CN=mgmt6.testrelm.test,O=TESTRELM.TEST
	expires: 2019-09-26 13:55:38 UTC
Request ID '20170925135618':
	status: MONITORING
	subject: CN=mgmt6.testrelm.test,O=TESTRELM.TEST
	expires: 2019-09-26 13:56:19 UTC
Request ID '20170925135631':
	status: MONITORING
	subject: CN=mgmt6.testrelm.test,O=TESTRELM.TEST
	expires: 2019-09-26 13:56:31 UTC
# date -s "715 days";sleep 300
Tue Sep 10 17:05:33 EDT 2019
# getcert list | egrep "status|expires|Request|subject|ca-error"
Request ID '20170925135424':
	status: MONITORING
	subject: CN=IPA RA,O=TESTRELM.TEST
	expires: 2021-08-30 21:07:09 UTC
Request ID '20170925135504':
	status: MONITORING
	subject: CN=CA Audit,O=TESTRELM.TEST
	expires: 2021-08-30 21:07:27 UTC
Request ID '20170925135505':
	status: MONITORING
	subject: CN=OCSP Subsystem,O=TESTRELM.TEST
	expires: 2021-08-30 21:07:28 UTC
Request ID '20170925135506':
	status: MONITORING
	subject: CN=CA Subsystem,O=TESTRELM.TEST
	expires: 2021-08-30 21:07:08 UTC
Request ID '20170925135507':
	status: MONITORING
	subject: CN=Certificate Authority,O=TESTRELM.TEST
	expires: 2037-09-25 13:54:01 UTC
Request ID '20170925135508':
	status: MONITORING
	subject: CN=mgmt6.testrelm.test,O=TESTRELM.TEST
	expires: 2021-08-30 21:07:38 UTC
Request ID '20170925135538':
	status: MONITORING
	subject: CN=mgmt6.testrelm.test,O=TESTRELM.TEST
	expires: 2021-09-10 21:07:12 UTC
Request ID '20170925135618':
	status: MONITORING
	subject: CN=mgmt6.testrelm.test,O=TESTRELM.TEST
	expires: 2021-09-10 21:07:22 UTC
Request ID '20170925135631':
	status: MONITORING
	subject: CN=mgmt6.testrelm.test,O=TESTRELM.TEST
	expires: 2021-09-10 21:07:04 UTC
# kinit admin
Password for admin@TESTRELM.TEST: 
Password expired.  You must change it now.
Enter new password: 
Enter it again: 
# ipa cert-show 1
  Issuing CA: ipa
  Certificate: MIIDkTCCAnmgAwIBAgIBATANBgkqhkiG9w0BAQsFADA4MRYwFAYDVQQKDA1URVNUUkVMTS5URVNUMR4wHAYDVQQDDBVDZXJ0aWZpY2F0ZSBBdXRob3JpdHkwHhcNMTcwOTI1MTM1NDAxWhcNMzcwOTI1MTM1NDAxWjA4MRYwFAYDVQQKDA1URVNUUkVMTS5URVNUMR4wHAYDVQQDDBVDZXJ0aWZpY2F0ZSBBdXRob3JpdHkwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDY2jv3bFDdhyYEO9YlLnaTPcK2GAZcWSfamgowWio024UvfhjOb0jndM0ko6HdFCWTXTHvONylTE/HCseqCGNIMIw970CLPp7hbuZ/TMm2PIIbbXqzBq3r4bk1JJTISVdxKs4lnyTiJihH3cDlzLMoWu7DxPHP0yoywSDWdyyljrbKuO7Lu04Z3ZnSvHelbzRUdCNEyPyAnAA941J44m/jwVg4xQZ8R/M24HpVtlB5Oi15PNL21W2W7cFQ01d4s+c/Ab7kOBv37r6zxHAfBdV8hKpIlpcV1rS1mTy2wKGU33y+YeQVL1p+Yjoq4M0G7mpojGeFdRc9/1VZFyroNLyLAgMBAAGjgaUwgaIwHwYDVR0jBBgwFoAUR9v2Y1VesiOALKfJWj3qDlSwGVMwDwYDVR0TAQH/BAUwAwEB/zAOBgNVHQ8BAf8EBAMCAcYwHQYDVR0OBBYEFEfb9mNVXrIjgCynyVo96g5UsBlTMD8GCCsGAQUFBwEBBDMwMTAvBggrBgEFBQcwAYYjaHR0cDovL2lwYS1jYS50ZXN0cmVsbS50ZXN0L2NhL29jc3AwDQYJKoZIhvcNAQELBQADggEBAMPZkWas54cMhbWK+RXh7DbR2lR3V1AFMatra4Gw7qtfowSa1SCoxrwNuBpGnI1oxgWcVMk1Gbqe1cT85DBdjEnRk0vyJ/Ja5fIrLA0TvP/41dnN+IveGfL/k1xlpdRf+yVex5uJBs0eTs4Ohqf6So3AqLgru5ELiOjU8PerjFDp9J3vCVXc60nUFWIgt5pFvxkfJDAC23+eF/tZkfV2MncM3tXcpyC8ulkaZP2HwhUEa6MQ4tVhmabGYotQE5XAGhlKGdq5BV4m0KdZnrR0YTHuec5szw+SxfHea7zwpL1DzIrfuemSvwNjR3rso5MmD/pcxmWeGdzc9c+H+ysG0oU=
  Subject: CN=Certificate Authority,O=TESTRELM.TEST
  Issuer: CN=Certificate Authority,O=TESTRELM.TEST
  Not Before: Mon Sep 25 13:54:01 2017 UTC
  Not After: Fri Sep 25 13:54:01 2037 UTC
  Serial number: 1
  Serial number (hex): 0x1
  Revoked: False
# date -s "715 days";sleep 300
Wed Aug 25 17:13:45 EDT 2021
[root@mgmt6 ~]# getcert list | egrep "status|expires|Request|subject|ca-error"
Request ID '20170925135424':
	status: MONITORING
	subject: CN=IPA RA,O=TESTRELM.TEST
	expires: 2023-08-15 21:17:09 UTC
Request ID '20170925135504':
	status: MONITORING
	subject: CN=CA Audit,O=TESTRELM.TEST
	expires: 2023-08-15 21:17:10 UTC
Request ID '20170925135505':
	status: MONITORING
	subject: CN=OCSP Subsystem,O=TESTRELM.TEST
	expires: 2023-08-15 21:17:40 UTC
Request ID '20170925135506':
	status: MONITORING
	subject: CN=CA Subsystem,O=TESTRELM.TEST
	expires: 2023-08-15 21:17:20 UTC
Request ID '20170925135507':
	status: MONITORING
	subject: CN=Certificate Authority,O=TESTRELM.TEST
	expires: 2037-09-25 13:54:01 UTC
Request ID '20170925135508':
	status: POST_SAVED_CERT
	subject: CN=mgmt6.testrelm.test,O=TESTRELM.TEST
	expires: 2023-08-15 21:17:20 UTC
Request ID '20170925135538':
	status: MONITORING
	subject: CN=mgmt6.testrelm.test,O=TESTRELM.TEST
	expires: 2023-08-26 21:17:14 UTC
Request ID '20170925135618':
	status: MONITORING
	subject: CN=mgmt6.testrelm.test,O=TESTRELM.TEST
	expires: 2023-08-26 21:17:24 UTC
Request ID '20170925135631':
	status: MONITORING
	subject: CN=mgmt6.testrelm.test,O=TESTRELM.TEST
	expires: 2023-08-26 21:17:05 UTC
[root@mgmt6 ~]# getcert list | egrep "status|expires|Request|subject|ca-error"
Request ID '20170925135424':
	status: MONITORING
	subject: CN=IPA RA,O=TESTRELM.TEST
	expires: 2023-08-15 21:17:09 UTC
Request ID '20170925135504':
	status: MONITORING
	subject: CN=CA Audit,O=TESTRELM.TEST
	expires: 2023-08-15 21:17:10 UTC
Request ID '20170925135505':
	status: MONITORING
	subject: CN=OCSP Subsystem,O=TESTRELM.TEST
	expires: 2023-08-15 21:17:40 UTC
Request ID '20170925135506':
	status: MONITORING
	subject: CN=CA Subsystem,O=TESTRELM.TEST
	expires: 2023-08-15 21:17:20 UTC
Request ID '20170925135507':
	status: MONITORING
	subject: CN=Certificate Authority,O=TESTRELM.TEST
	expires: 2037-09-25 13:54:01 UTC
Request ID '20170925135508':
	status: MONITORING
	subject: CN=mgmt6.testrelm.test,O=TESTRELM.TEST
	expires: 2023-08-15 21:17:20 UTC
Request ID '20170925135538':
	status: MONITORING
	subject: CN=mgmt6.testrelm.test,O=TESTRELM.TEST
	expires: 2023-08-26 21:17:14 UTC
Request ID '20170925135618':
	status: MONITORING
	subject: CN=mgmt6.testrelm.test,O=TESTRELM.TEST
	expires: 2023-08-26 21:17:24 UTC
Request ID '20170925135631':
	status: MONITORING
	subject: CN=mgmt6.testrelm.test,O=TESTRELM.TEST
	expires: 2023-08-26 21:17:05 UTC
# kinit admin
Password for admin@TESTRELM.TEST: 
Password expired.  You must change it now.
Enter new password: 
Enter it again:
# ipa cert-show 1
  Issuing CA: ipa
  Certificate: MIIDkTCCAnmgAwIBAgIBATANBgkqhkiG9w0BAQsFADA4MRYwFAYDVQQKDA1URVNUUkVMTS5URVNUMR4wHAYDVQQDDBVDZXJ0aWZpY2F0ZSBBdXRob3JpdHkwHhcNMTcwOTI1MTM1NDAxWhcNMzcwOTI1MTM1NDAxWjA4MRYwFAYDVQQKDA1URVNUUkVMTS5URVNUMR4wHAYDVQQDDBVDZXJ0aWZpY2F0ZSBBdXRob3JpdHkwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDY2jv3bFDdhyYEO9YlLnaTPcK2GAZcWSfamgowWio024UvfhjOb0jndM0ko6HdFCWTXTHvONylTE/HCseqCGNIMIw970CLPp7hbuZ/TMm2PIIbbXqzBq3r4bk1JJTISVdxKs4lnyTiJihH3cDlzLMoWu7DxPHP0yoywSDWdyyljrbKuO7Lu04Z3ZnSvHelbzRUdCNEyPyAnAA941J44m/jwVg4xQZ8R/M24HpVtlB5Oi15PNL21W2W7cFQ01d4s+c/Ab7kOBv37r6zxHAfBdV8hKpIlpcV1rS1mTy2wKGU33y+YeQVL1p+Yjoq4M0G7mpojGeFdRc9/1VZFyroNLyLAgMBAAGjgaUwgaIwHwYDVR0jBBgwFoAUR9v2Y1VesiOALKfJWj3qDlSwGVMwDwYDVR0TAQH/BAUwAwEB/zAOBgNVHQ8BAf8EBAMCAcYwHQYDVR0OBBYEFEfb9mNVXrIjgCynyVo96g5UsBlTMD8GCCsGAQUFBwEBBDMwMTAvBggrBgEFBQcwAYYjaHR0cDovL2lwYS1jYS50ZXN0cmVsbS50ZXN0L2NhL29jc3AwDQYJKoZIhvcNAQELBQADggEBAMPZkWas54cMhbWK+RXh7DbR2lR3V1AFMatra4Gw7qtfowSa1SCoxrwNuBpGnI1oxgWcVMk1Gbqe1cT85DBdjEnRk0vyJ/Ja5fIrLA0TvP/41dnN+IveGfL/k1xlpdRf+yVex5uJBs0eTs4Ohqf6So3AqLgru5ELiOjU8PerjFDp9J3vCVXc60nUFWIgt5pFvxkfJDAC23+eF/tZkfV2MncM3tXcpyC8ulkaZP2HwhUEa6MQ4tVhmabGYotQE5XAGhlKGdq5BV4m0KdZnrR0YTHuec5szw+SxfHea7zwpL1DzIrfuemSvwNjR3rso5MmD/pcxmWeGdzc9c+H+ysG0oU=
  Subject: CN=Certificate Authority,O=TESTRELM.TEST
  Issuer: CN=Certificate Authority,O=TESTRELM.TEST
  Not Before: Mon Sep 25 13:54:01 2017 UTC
  Not After: Fri Sep 25 13:54:01 2037 UTC
  Serial number: 1
  Serial number (hex): 0x1
  Revoked: False

Comment 37 errata-xmlrpc 2018-04-10 08:38:21 EDT

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:0763

Note You need to log in before you can comment on or make changes to this bug.