1481388 – IPA certificate auto renewal failed at unable to connect to LDAP server with authentication failure

RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1481388 - IPA certificate auto renewal failed at unable to connect to LDAP server with authentication failure

Summary: IPA certificate auto renewal failed at unable to connect to LDAP server with ...

Keywords:
Status:	CLOSED ERRATA
Alias:	None
Product:	Red Hat Enterprise Linux 7
Classification:	Red Hat
Component:	selinux-policy
Sub Component:
Version:	7.4
Hardware:	x86_64
OS:	Linux
Priority:	high
Severity:	high
Target Milestone:	rc
Target Release:	---
Assignee:	Lukas Vrabec
QA Contact:	Milos Malik
Docs Contact:
URL:
Whiteboard:
Depends On:
Blocks:	1486552 1515503
TreeView+	depends on / blocked

Reported:	2017-08-14 19:48 UTC by Xiyang Dong
Modified:	2018-04-10 12:40 UTC (History)
CC List:	13 users (show)
Fixed In Version:	selinux-policy-3.13.1-172.el7
Doc Type:	Bug Fix
Doc Text:	Previously, the selinux-policy package missed certain rules related to Identity Management (IdM). As a consequence, the IPA certificate auto-renewal procedure failed and the Tomcat service was not able to connect to the LDAP server. This update adds the SELinux allow rules that enable IdM to auto-renew certificates when running SELinux in enforcing mode, and the described problem no longer occurs.
Clone Of:
Clones:	1486552 1515503 (view as bug list)
Environment:
Last Closed:	2018-04-10 12:38:21 UTC
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)
/var/log/pki/pki-tomcat/ca/debug (11.17 MB, text/plain) 2017-08-14 19:59 UTC, Xiyang Dong	no flags	Details
/var/log/audit/audit.log (100.50 KB, text/plain) 2017-08-14 21:06 UTC, Xiyang Dong	no flags	Details
/var/log/messages (2.51 MB, text/plain) 2017-08-14 21:07 UTC, Xiyang Dong	no flags	Details
/var/log/pki/pki-tomcat/ca/debug_Z-stream (10.97 MB, text/plain) 2017-09-25 02:14 UTC, Xiyang Dong	no flags	Details
/var/log/audit/audit.log_Z-stream (72.76 KB, text/plain) 2017-09-25 02:17 UTC, Xiyang Dong	no flags	Details
/var/log/messages_Z-stream (634.21 KB, text/plain) 2017-09-25 02:19 UTC, Xiyang Dong	no flags	Details
audit2why_Z-stream (7.22 KB, text/plain) 2017-09-25 02:28 UTC, Xiyang Dong	no flags	Details
ausearch_Z-stream (5.97 KB, text/plain) 2017-09-25 02:30 UTC, Xiyang Dong	no flags	Details
View All

Links
System	ID	Private	Priority	Status	Summary	Last Updated
Red Hat Product Errata	RHBA-2018:0763	0	None	None	None	2018-04-10 12:40:51 UTC

Description Xiyang Dong 2017-08-14 19:48:07 UTC

Description of problem:
IPA certificate auto renewal failed at ca-error: 
ca-error: Server at "http://bkr-hv03-guest23.testrelm.test:8080/ca/ee/ca/profileSubmit" replied: Failed to connect LDAP server Could not connect to LDAP server host bkr-hv03-guest23.testrelm.test port 636 Error netscape.ldap.LDAPException: Authentication failed (48)

Version-Release number of selected component (if applicable):
# rpm -q ipa-server
ipa-server-4.5.0-21.el7_4.1.x86_64

How reproducible:
Always

Steps to Reproduce:
1. Install ipa server
2. Change date prior to cert expire
3. Check cert renewal status

Actual results:
Cert autorenewal failed

Expected results:
Cert autorenewal passes

Additional info:
# date
Mon Aug 14 15:06:01 EDT 2017
# getcert list | egrep "status|expires|Request|subject|ca-error"
Request ID '20170814185636':
	status: MONITORING
	subject: CN=IPA RA,O=TESTRELM.TEST
	expires: 2019-08-04 18:56:40 UTC
Request ID '20170814185733':
	status: MONITORING
	subject: CN=CA Audit,O=TESTRELM.TEST
	expires: 2019-08-04 18:56:19 UTC
Request ID '20170814185734':
	status: MONITORING
	subject: CN=OCSP Subsystem,O=TESTRELM.TEST
	expires: 2019-08-04 18:56:18 UTC
Request ID '20170814185735':
	status: MONITORING
	subject: CN=CA Subsystem,O=TESTRELM.TEST
	expires: 2019-08-04 18:56:18 UTC
Request ID '20170814185736':
	status: MONITORING
	subject: CN=Certificate Authority,O=TESTRELM.TEST
	expires: 2037-08-14 18:56:18 UTC
Request ID '20170814185737':
	status: MONITORING
	subject: CN=auto-hv-01-guest01.testrelm.test,O=TESTRELM.TEST
	expires: 2019-08-04 18:56:18 UTC
Request ID '20170814185802':
	status: MONITORING
	subject: CN=auto-hv-01-guest01.testrelm.test,O=TESTRELM.TEST
	expires: 2019-08-15 18:58:03 UTC
Request ID '20170814185827':
	status: MONITORING
	subject: CN=auto-hv-01-guest01.testrelm.test,O=TESTRELM.TEST
	expires: 2019-08-15 18:58:28 UTC
Request ID '20170814185839':
	status: MONITORING
	subject: CN=auto-hv-01-guest01.testrelm.test,O=TESTRELM.TEST
	expires: 2019-08-15 18:58:40 UTC
# date -s "715 days";sleep 300;getcert list | egrep "status|expires|Request|subject|ca-error"
Tue Jul 30 15:07:04 EDT 2019
Request ID '20170814185636':
	status: MONITORING
	subject: CN=IPA RA,O=TESTRELM.TEST
	expires: 2021-07-19 19:10:30 UTC
Request ID '20170814185733':
	status: MONITORING
	subject: CN=CA Audit,O=TESTRELM.TEST
	expires: 2021-07-19 19:11:30 UTC
Request ID '20170814185734':
	status: MONITORING
	subject: CN=OCSP Subsystem,O=TESTRELM.TEST
	expires: 2021-07-19 19:11:19 UTC
Request ID '20170814185735':
	status: MONITORING
	subject: CN=CA Subsystem,O=TESTRELM.TEST
	expires: 2021-07-19 19:10:49 UTC
Request ID '20170814185736':
	status: MONITORING
	subject: CN=Certificate Authority,O=TESTRELM.TEST
	expires: 2037-08-14 18:56:18 UTC
Request ID '20170814185737':
	status: MONITORING
	subject: CN=auto-hv-01-guest01.testrelm.test,O=TESTRELM.TEST
	expires: 2021-07-19 19:10:29 UTC
Request ID '20170814185802':
	status: MONITORING
	subject: CN=auto-hv-01-guest01.testrelm.test,O=TESTRELM.TEST
	expires: 2021-07-30 19:10:27 UTC
Request ID '20170814185827':
	status: MONITORING
	subject: CN=auto-hv-01-guest01.testrelm.test,O=TESTRELM.TEST
	expires: 2021-07-30 19:10:19 UTC
Request ID '20170814185839':
	status: MONITORING
	subject: CN=auto-hv-01-guest01.testrelm.test,O=TESTRELM.TEST
	expires: 2021-07-30 19:10:13 UTC
# date -s "715 days";sleep 300;getcert list | egrep "status|expires|Request|subject|ca-error"
Wed Jul 14 15:15:23 EDT 2021
Request ID '20170814185636':
	status: MONITORING
	ca-error: Server at "http://auto-hv-01-guest01.testrelm.test:8080/ca/ee/ca/profileSubmit" replied: Failed to connect LDAP server Could not connect to LDAP server host auto-hv-01-guest01.testrelm.test port 636 Error netscape.ldap.LDAPException: Authentication failed (48)
	subject: CN=IPA RA,O=TESTRELM.TEST
	expires: 2021-07-19 19:10:30 UTC
Request ID '20170814185733':
	status: MONITORING
	ca-error: Server at "http://auto-hv-01-guest01.testrelm.test:8080/ca/ee/ca/profileSubmit" replied: Failed to connect LDAP server Could not connect to LDAP server host auto-hv-01-guest01.testrelm.test port 636 Error netscape.ldap.LDAPException: Authentication failed (48)
	subject: CN=CA Audit,O=TESTRELM.TEST
	expires: 2021-07-19 19:11:30 UTC
Request ID '20170814185734':
	status: MONITORING
	ca-error: Server at "http://auto-hv-01-guest01.testrelm.test:8080/ca/ee/ca/profileSubmit" replied: Failed to connect LDAP server Could not connect to LDAP server host auto-hv-01-guest01.testrelm.test port 636 Error netscape.ldap.LDAPException: Authentication failed (48)
	subject: CN=OCSP Subsystem,O=TESTRELM.TEST
	expires: 2021-07-19 19:11:19 UTC
Request ID '20170814185735':
	status: MONITORING
	ca-error: Server at "http://auto-hv-01-guest01.testrelm.test:8080/ca/ee/ca/profileSubmit" replied: Failed to connect LDAP server Could not connect to LDAP server host auto-hv-01-guest01.testrelm.test port 636 Error netscape.ldap.LDAPException: Authentication failed (48)
	subject: CN=CA Subsystem,O=TESTRELM.TEST
	expires: 2021-07-19 19:10:49 UTC
Request ID '20170814185736':
	status: MONITORING
	subject: CN=Certificate Authority,O=TESTRELM.TEST
	expires: 2037-08-14 18:56:18 UTC
Request ID '20170814185737':
	status: MONITORING
	ca-error: Server at "http://auto-hv-01-guest01.testrelm.test:8080/ca/ee/ca/profileSubmit" replied: Failed to connect LDAP server Could not connect to LDAP server host auto-hv-01-guest01.testrelm.test port 636 Error netscape.ldap.LDAPException: Authentication failed (48)
	subject: CN=auto-hv-01-guest01.testrelm.test,O=TESTRELM.TEST
	expires: 2021-07-19 19:10:29 UTC
Request ID '20170814185802':
	status: MONITORING
	subject: CN=auto-hv-01-guest01.testrelm.test,O=TESTRELM.TEST
	expires: 2023-07-15 19:20:36 UTC
Request ID '20170814185827':
	status: CA_UNREACHABLE
	ca-error: Server at https://auto-hv-01-guest01.testrelm.test/ipa/xml failed request, will retry: 4016 (RPC failed at server.  Failed to authenticate to CA REST API).
	subject: CN=auto-hv-01-guest01.testrelm.test,O=TESTRELM.TEST
	expires: 2021-07-30 19:10:19 UTC
Request ID '20170814185839':
	status: MONITORING
	subject: CN=auto-hv-01-guest01.testrelm.test,O=TESTRELM.TEST
	expires: 2023-07-15 19:20:28 UTC

Comment 2 Xiyang Dong 2017-08-14 19:59:47 UTC

Created attachment 1313327 [details]
/var/log/pki/pki-tomcat/ca/debug

Comment 3 Rob Crittenden 2017-08-14 20:01:45 UTC

I'd check SELinux output and the journal to be sure that during the first renewal the certificates were updated properly in o=ipaca

Were I to guess the RA agent cert is out-of-sync with what is in LDAP.

A few improvement suggestions to the test:

- print date again after the sleep(300)
- if you can, kinit admin && ipa cert-show 1 to exercise the IPA <-> dogtag connection

Comment 4 Xiyang Dong 2017-08-14 20:57:31 UTC

[root@auto-hv-01-guest01 ~]# date
Mon Aug 14 16:42:02 EDT 2017
[root@auto-hv-01-guest01 ~]# kinit admin
Password for admin: 
[root@auto-hv-01-guest01 ~]# ipa cert-show 1
  Issuing CA: ipa
  Certificate: MIIDkTCCAnmgAwIBAgIBATANBgkqhkiG9w0BAQsFADA4MRYwFAYDVQQKDA1URVNUUkVMTS5URVNUMR4wHAYDVQQDDBVDZXJ0aWZpY2F0ZSBBdXRob3JpdHkwHhcNMTcwODE0MjAyNjAwWhcNMzcwODE0MjAyNjAwWjA4MRYwFAYDVQQKDA1URVNUUkVMTS5URVNUMR4wHAYDVQQDDBVDZXJ0aWZpY2F0ZSBBdXRob3JpdHkwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQC3PMBLxaNZPt0i2jBX7ya3HBcT+L5aQPvDoRBSXahoVvPGkB1QHHEF6LRgIMVKfiCfjR0UgtzjHScNa5qjq7iMQyS2iGYfXDUtWuhOuU/FMv78VpUJbjtlhsyeSaO44pJakaMYQ30EOEXAXtdKPqtUvgOqEOfCwG7D95PUysXiYvR/jYC0CJhC80m3+46+9FuSsDb2kPixinXZJ9snqHn8g+0WBHaYLk9gUhC17kzrZyZgENPJeRtK8rMU274YDUA7337hfAB079sV86jfko9pt0Q5yIpXsD75p6Sk4pX5d70VL+erMcWKYE2pZ2rMt/dBAi/8ZK2Mucdg9hoOOFApAgMBAAGjgaUwgaIwHwYDVR0jBBgwFoAUkZ/t+MOYNgO92/ICn8fDs6rN7XUwDwYDVR0TAQH/BAUwAwEB/zAOBgNVHQ8BAf8EBAMCAcYwHQYDVR0OBBYEFJGf7fjDmDYDvdvyAp/Hw7Oqze11MD8GCCsGAQUFBwEBBDMwMTAvBggrBgEFBQcwAYYjaHR0cDovL2lwYS1jYS50ZXN0cmVsbS50ZXN0L2NhL29jc3AwDQYJKoZIhvcNAQELBQADggEBAK5VDKvnhHUK2CKUt/XQD/VlthKcASFuCa9vNfUhdqn0aeLC6lrbV/LzWSSlnMyiwfoCY/Xerc6YDmejyyJCL08NSobfcyUtYpDsun5REBlpnewCw/9riQtaDUfxXgNyv1mu6EjTSYNN3QOQywRRrwQulMnzk64gsPjWwdCinQ34cX7QayquWmEbpip9fV9FpSxnA1Cjeqh23+YAgd+XRWAAvjaiEMiMUgEkQKX9ANsgxVUy18DnnHPVshGCNLVqs6MV7Uc43yfrn2UCPke9V190qvWAhyeou4rsUU7AiPmtWPyyvJOpTl1gh1Jb1AuKTXhPAyCPhZh+QtXXfI5tpKA=
  Subject: CN=Certificate Authority,O=TESTRELM.TEST
  Issuer: CN=Certificate Authority,O=TESTRELM.TEST
  Not Before: Mon Aug 14 20:26:00 2017 UTC
  Not After: Fri Aug 14 20:26:00 2037 UTC
  Serial number: 1
  Serial number (hex): 0x1
  Revoked: False
[root@auto-hv-01-guest01 ~]# getcert list | egrep "status|expires|Request|subject|ca-error"
Request ID '20170814202634':
	status: MONITORING
	subject: CN=IPA RA,O=TESTRELM.TEST
	expires: 2019-08-04 20:26:36 UTC
Request ID '20170814202739':
	status: MONITORING
	subject: CN=CA Audit,O=TESTRELM.TEST
	expires: 2019-08-04 20:26:05 UTC
Request ID '20170814202740':
	status: MONITORING
	subject: CN=OCSP Subsystem,O=TESTRELM.TEST
	expires: 2019-08-04 20:26:03 UTC
Request ID '20170814202741':
	status: MONITORING
	subject: CN=CA Subsystem,O=TESTRELM.TEST
	expires: 2019-08-04 20:26:04 UTC
Request ID '20170814202742':
	status: MONITORING
	subject: CN=Certificate Authority,O=TESTRELM.TEST
	expires: 2037-08-14 20:26:00 UTC
Request ID '20170814202743':
	status: MONITORING
	subject: CN=auto-hv-01-guest01.testrelm.test,O=TESTRELM.TEST
	expires: 2019-08-04 20:26:03 UTC
Request ID '20170814202821':
	status: MONITORING
	subject: CN=auto-hv-01-guest01.testrelm.test,O=TESTRELM.TEST
	expires: 2019-08-15 20:28:22 UTC
Request ID '20170814202911':
	status: MONITORING
	subject: CN=auto-hv-01-guest01.testrelm.test,O=TESTRELM.TEST
	expires: 2019-08-15 20:29:12 UTC
Request ID '20170814202927':
	status: MONITORING
	subject: CN=auto-hv-01-guest01.testrelm.test,O=TESTRELM.TEST
	expires: 2019-08-15 20:29:28 UTC
[root@auto-hv-01-guest01 ~]# date -s "715 days";sleep 300;date
Tue Jul 30 16:43:24 EDT 2019
Tue Jul 30 16:48:24 EDT 2019
[root@auto-hv-01-guest01 ~]# getcert list | egrep "status|expires|Request|subject|ca-error"
Request ID '20170814202634':
	status: MONITORING
	subject: CN=IPA RA,O=TESTRELM.TEST
	expires: 2021-07-19 20:46:14 UTC
Request ID '20170814202739':
	status: MONITORING
	subject: CN=CA Audit,O=TESTRELM.TEST
	expires: 2021-07-19 20:46:55 UTC
Request ID '20170814202740':
	status: MONITORING
	subject: CN=OCSP Subsystem,O=TESTRELM.TEST
	expires: 2021-07-19 20:46:04 UTC
Request ID '20170814202741':
	status: MONITORING
	subject: CN=CA Subsystem,O=TESTRELM.TEST
	expires: 2021-07-19 20:46:24 UTC
Request ID '20170814202742':
	status: MONITORING
	subject: CN=Certificate Authority,O=TESTRELM.TEST
	expires: 2037-08-14 20:26:00 UTC
Request ID '20170814202743':
	status: MONITORING
	subject: CN=auto-hv-01-guest01.testrelm.test,O=TESTRELM.TEST
	expires: 2021-07-19 20:45:54 UTC
Request ID '20170814202821':
	status: MONITORING
	subject: CN=auto-hv-01-guest01.testrelm.test,O=TESTRELM.TEST
	expires: 2021-07-30 20:45:59 UTC
Request ID '20170814202911':
	status: MONITORING
	subject: CN=auto-hv-01-guest01.testrelm.test,O=TESTRELM.TEST
	expires: 2021-07-30 20:45:50 UTC
Request ID '20170814202927':
	status: MONITORING
	subject: CN=auto-hv-01-guest01.testrelm.test,O=TESTRELM.TEST
	expires: 2021-07-30 20:45:40 UTC
[root@auto-hv-01-guest01 ~]# kinit admin
Password for admin: 
Password expired.  You must change it now.
Enter new password: 
Enter it again: 
[root@auto-hv-01-guest01 ~]# ipa cert-show 1
  Issuing CA: ipa
  Certificate: MIIDkTCCAnmgAwIBAgIBATANBgkqhkiG9w0BAQsFADA4MRYwFAYDVQQKDA1URVNUUkVMTS5URVNUMR4wHAYDVQQDDBVDZXJ0aWZpY2F0ZSBBdXRob3JpdHkwHhcNMTcwODE0MjAyNjAwWhcNMzcwODE0MjAyNjAwWjA4MRYwFAYDVQQKDA1URVNUUkVMTS5URVNUMR4wHAYDVQQDDBVDZXJ0aWZpY2F0ZSBBdXRob3JpdHkwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQC3PMBLxaNZPt0i2jBX7ya3HBcT+L5aQPvDoRBSXahoVvPGkB1QHHEF6LRgIMVKfiCfjR0UgtzjHScNa5qjq7iMQyS2iGYfXDUtWuhOuU/FMv78VpUJbjtlhsyeSaO44pJakaMYQ30EOEXAXtdKPqtUvgOqEOfCwG7D95PUysXiYvR/jYC0CJhC80m3+46+9FuSsDb2kPixinXZJ9snqHn8g+0WBHaYLk9gUhC17kzrZyZgENPJeRtK8rMU274YDUA7337hfAB079sV86jfko9pt0Q5yIpXsD75p6Sk4pX5d70VL+erMcWKYE2pZ2rMt/dBAi/8ZK2Mucdg9hoOOFApAgMBAAGjgaUwgaIwHwYDVR0jBBgwFoAUkZ/t+MOYNgO92/ICn8fDs6rN7XUwDwYDVR0TAQH/BAUwAwEB/zAOBgNVHQ8BAf8EBAMCAcYwHQYDVR0OBBYEFJGf7fjDmDYDvdvyAp/Hw7Oqze11MD8GCCsGAQUFBwEBBDMwMTAvBggrBgEFBQcwAYYjaHR0cDovL2lwYS1jYS50ZXN0cmVsbS50ZXN0L2NhL29jc3AwDQYJKoZIhvcNAQELBQADggEBAK5VDKvnhHUK2CKUt/XQD/VlthKcASFuCa9vNfUhdqn0aeLC6lrbV/LzWSSlnMyiwfoCY/Xerc6YDmejyyJCL08NSobfcyUtYpDsun5REBlpnewCw/9riQtaDUfxXgNyv1mu6EjTSYNN3QOQywRRrwQulMnzk64gsPjWwdCinQ34cX7QayquWmEbpip9fV9FpSxnA1Cjeqh23+YAgd+XRWAAvjaiEMiMUgEkQKX9ANsgxVUy18DnnHPVshGCNLVqs6MV7Uc43yfrn2UCPke9V190qvWAhyeou4rsUU7AiPmtWPyyvJOpTl1gh1Jb1AuKTXhPAyCPhZh+QtXXfI5tpKA=
  Subject: CN=Certificate Authority,O=TESTRELM.TEST
  Issuer: CN=Certificate Authority,O=TESTRELM.TEST
  Not Before: Mon Aug 14 20:26:00 2017 UTC
  Not After: Fri Aug 14 20:26:00 2037 UTC
  Serial number: 1
  Serial number (hex): 0x1
  Revoked: False
[root@auto-hv-01-guest01 ~]# date -s "715 days";sleep 300;date
Wed Jul 14 16:49:21 EDT 2021
Wed Jul 14 16:54:21 EDT 2021
[root@auto-hv-01-guest01 ~]# getcert list | egrep "status|expires|Request|subject|ca-error"
Request ID '20170814202634':
	status: MONITORING
	ca-error: Server at "http://auto-hv-01-guest01.testrelm.test:8080/ca/ee/ca/profileSubmit" replied: Failed to connect LDAP server Could not connect to LDAP server host auto-hv-01-guest01.testrelm.test port 636 Error netscape.ldap.LDAPException: Authentication failed (48)
	subject: CN=IPA RA,O=TESTRELM.TEST
	expires: 2021-07-19 20:46:14 UTC
Request ID '20170814202739':
	status: MONITORING
	ca-error: Server at "http://auto-hv-01-guest01.testrelm.test:8080/ca/ee/ca/profileSubmit" replied: Failed to connect LDAP server Could not connect to LDAP server host auto-hv-01-guest01.testrelm.test port 636 Error netscape.ldap.LDAPException: Authentication failed (48)
	subject: CN=CA Audit,O=TESTRELM.TEST
	expires: 2021-07-19 20:46:55 UTC
Request ID '20170814202740':
	status: MONITORING
	ca-error: Server at "http://auto-hv-01-guest01.testrelm.test:8080/ca/ee/ca/profileSubmit" replied: Failed to connect LDAP server Could not connect to LDAP server host auto-hv-01-guest01.testrelm.test port 636 Error netscape.ldap.LDAPException: Authentication failed (48)
	subject: CN=OCSP Subsystem,O=TESTRELM.TEST
	expires: 2021-07-19 20:46:04 UTC
Request ID '20170814202741':
	status: MONITORING
	ca-error: Server at "http://auto-hv-01-guest01.testrelm.test:8080/ca/ee/ca/profileSubmit" replied: Failed to connect LDAP server Could not connect to LDAP server host auto-hv-01-guest01.testrelm.test port 636 Error netscape.ldap.LDAPException: Authentication failed (48)
	subject: CN=CA Subsystem,O=TESTRELM.TEST
	expires: 2021-07-19 20:46:24 UTC
Request ID '20170814202742':
	status: MONITORING
	subject: CN=Certificate Authority,O=TESTRELM.TEST
	expires: 2037-08-14 20:26:00 UTC
Request ID '20170814202743':
	status: MONITORING
	ca-error: Server at "http://auto-hv-01-guest01.testrelm.test:8080/ca/ee/ca/profileSubmit" replied: Failed to connect LDAP server Could not connect to LDAP server host auto-hv-01-guest01.testrelm.test port 636 Error netscape.ldap.LDAPException: Authentication failed (48)
	subject: CN=auto-hv-01-guest01.testrelm.test,O=TESTRELM.TEST
	expires: 2021-07-19 20:45:54 UTC
Request ID '20170814202821':
	status: MONITORING
	subject: CN=auto-hv-01-guest01.testrelm.test,O=TESTRELM.TEST
	expires: 2023-07-15 20:53:47 UTC
Request ID '20170814202911':
	status: MONITORING
	subject: CN=auto-hv-01-guest01.testrelm.test,O=TESTRELM.TEST
	expires: 2023-07-15 20:53:37 UTC
Request ID '20170814202927':
	status: MONITORING
	subject: CN=auto-hv-01-guest01.testrelm.test,O=TESTRELM.TEST
	expires: 2023-07-15 20:53:29 UTC
[root@auto-hv-01-guest01 ~]# kinit admin
Password for admin: 
Password expired.  You must change it now.
Enter new password: 
Enter it again: 
[root@auto-hv-01-guest01 ~]# ipa cert-show 1
ipa: ERROR: Certificate operation cannot be completed: EXCEPTION (Invalid Credential.)

Comment 5 Xiyang Dong 2017-08-14 21:06:52 UTC

Created attachment 1313341 [details]
/var/log/audit/audit.log

Comment 6 Xiyang Dong 2017-08-14 21:07:18 UTC

Created attachment 1313342 [details]
/var/log/messages

Comment 7 Rob Crittenden 2017-08-14 21:17:23 UTC

Turns out my request for the add'l date command was not needed because I misread things originally :-(

The cert-show clearly demonstrates that the RA cert works after the first renewal but not the second.

Is this a regression or a new test?

From the messages file it looks like tomcat is having issues. I'm not sure if that is related or not

Jul 30 15:11:38 localhost renew_ca_cert: Traceback (most recent call last):
 File "/usr/libexec/ipa/certmonger/renew_ca_cert", line 218, in <module>
    main()
  File "/usr/libexec/ipa/certmonger/renew_ca_cert", line 212, in main
    _main()
  File "/usr/libexec/ipa/certmonger/renew_ca_cert", line 82, in _main
    ca.update_cert_config(nickname, cert)
  File "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", line 1156, in update_cert_config
    directives[nickname], cert, paths.CA_CS_CFG_PATH)
  File "/usr/lib/python2.7/site-packages/ipaserver/install/dogtaginstance.py", line 360, in update_cert_cs_cfg
    with stopped_service('pki-tomcatd', 'pki-tomcat'):
  File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "/usr/lib/python2.7/site-packages/ipaserver/install/installutils.py", line 1129, in stopped_service
    service_obj.stop(instance_name)
  File "/usr/lib/python2.7/site-packages/ipaplatform/base/services.py", line 283, in stop
    ipautil.run(args, skip_output=not capture_output)
  File "/usr/lib/python2.7/site-packages/ipapython/ipautil.py", line 511, in run
    raise CalledProcessError(p.returncode, arg_string, str(output))
CalledProcessError: Command '/bin/systemctl stop pki-tomcatd' returned non-zero exit status 4

Sadly I don't see anything in debug that echo's this failure at the same time.

Comment 21 Xiyang Dong 2017-09-25 02:14:31 UTC

Created attachment 1330340 [details]
/var/log/pki/pki-tomcat/ca/debug_Z-stream

Comment 22 Xiyang Dong 2017-09-25 02:17:39 UTC

Created attachment 1330341 [details]
/var/log/audit/audit.log_Z-stream

Comment 23 Xiyang Dong 2017-09-25 02:19:10 UTC

Created attachment 1330342 [details]
/var/log/messages_Z-stream

Comment 24 Xiyang Dong 2017-09-25 02:28:52 UTC

Created attachment 1330343 [details]
audit2why_Z-stream

Comment 25 Xiyang Dong 2017-09-25 02:30:05 UTC

Created attachment 1330344 [details]
ausearch_Z-stream

Comment 28 Xiyang Dong 2017-09-25 21:25:45 UTC

Thanks Florence and Lukas.
Verified on # rpm -qa ipa-server certmonger selinux-policy 
certmonger-0.78.4-3.el7.x86_64
ipa-server-4.5.0-20.el7.x86_64
selinux-policy-3.13.1-172.el7.noarch
# date
Mon Sep 25 17:03:14 EDT 2017
# kinit admin
Password for admin:
# ipa cert-show 1
  Issuing CA: ipa
  Certificate: MIIDkTCCAnmgAwIBAgIBATANBgkqhkiG9w0BAQsFADA4MRYwFAYDVQQKDA1URVNUUkVMTS5URVNUMR4wHAYDVQQDDBVDZXJ0aWZpY2F0ZSBBdXRob3JpdHkwHhcNMTcwOTI1MTM1NDAxWhcNMzcwOTI1MTM1NDAxWjA4MRYwFAYDVQQKDA1URVNUUkVMTS5URVNUMR4wHAYDVQQDDBVDZXJ0aWZpY2F0ZSBBdXRob3JpdHkwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDY2jv3bFDdhyYEO9YlLnaTPcK2GAZcWSfamgowWio024UvfhjOb0jndM0ko6HdFCWTXTHvONylTE/HCseqCGNIMIw970CLPp7hbuZ/TMm2PIIbbXqzBq3r4bk1JJTISVdxKs4lnyTiJihH3cDlzLMoWu7DxPHP0yoywSDWdyyljrbKuO7Lu04Z3ZnSvHelbzRUdCNEyPyAnAA941J44m/jwVg4xQZ8R/M24HpVtlB5Oi15PNL21W2W7cFQ01d4s+c/Ab7kOBv37r6zxHAfBdV8hKpIlpcV1rS1mTy2wKGU33y+YeQVL1p+Yjoq4M0G7mpojGeFdRc9/1VZFyroNLyLAgMBAAGjgaUwgaIwHwYDVR0jBBgwFoAUR9v2Y1VesiOALKfJWj3qDlSwGVMwDwYDVR0TAQH/BAUwAwEB/zAOBgNVHQ8BAf8EBAMCAcYwHQYDVR0OBBYEFEfb9mNVXrIjgCynyVo96g5UsBlTMD8GCCsGAQUFBwEBBDMwMTAvBggrBgEFBQcwAYYjaHR0cDovL2lwYS1jYS50ZXN0cmVsbS50ZXN0L2NhL29jc3AwDQYJKoZIhvcNAQELBQADggEBAMPZkWas54cMhbWK+RXh7DbR2lR3V1AFMatra4Gw7qtfowSa1SCoxrwNuBpGnI1oxgWcVMk1Gbqe1cT85DBdjEnRk0vyJ/Ja5fIrLA0TvP/41dnN+IveGfL/k1xlpdRf+yVex5uJBs0eTs4Ohqf6So3AqLgru5ELiOjU8PerjFDp9J3vCVXc60nUFWIgt5pFvxkfJDAC23+eF/tZkfV2MncM3tXcpyC8ulkaZP2HwhUEa6MQ4tVhmabGYotQE5XAGhlKGdq5BV4m0KdZnrR0YTHuec5szw+SxfHea7zwpL1DzIrfuemSvwNjR3rso5MmD/pcxmWeGdzc9c+H+ysG0oU=
  Subject: CN=Certificate Authority,O=TESTRELM.TEST
  Issuer: CN=Certificate Authority,O=TESTRELM.TEST
  Not Before: Mon Sep 25 13:54:01 2017 UTC
  Not After: Fri Sep 25 13:54:01 2037 UTC
  Serial number: 1
  Serial number (hex): 0x1
  Revoked: False
# getcert list | egrep "status|expires|Request|subject|ca-error"
Request ID '20170925135424':
	status: MONITORING
	subject: CN=IPA RA,O=TESTRELM.TEST
	expires: 2019-09-15 13:54:25 UTC
Request ID '20170925135504':
	status: MONITORING
	subject: CN=CA Audit,O=TESTRELM.TEST
	expires: 2019-09-15 13:54:04 UTC
Request ID '20170925135505':
	status: MONITORING
	subject: CN=OCSP Subsystem,O=TESTRELM.TEST
	expires: 2019-09-15 13:54:03 UTC
Request ID '20170925135506':
	status: MONITORING
	subject: CN=CA Subsystem,O=TESTRELM.TEST
	expires: 2019-09-15 13:54:04 UTC
Request ID '20170925135507':
	status: MONITORING
	subject: CN=Certificate Authority,O=TESTRELM.TEST
	expires: 2037-09-25 13:54:01 UTC
Request ID '20170925135508':
	status: MONITORING
	subject: CN=mgmt6.testrelm.test,O=TESTRELM.TEST
	expires: 2019-09-15 13:54:03 UTC
Request ID '20170925135538':
	status: MONITORING
	subject: CN=mgmt6.testrelm.test,O=TESTRELM.TEST
	expires: 2019-09-26 13:55:38 UTC
Request ID '20170925135618':
	status: MONITORING
	subject: CN=mgmt6.testrelm.test,O=TESTRELM.TEST
	expires: 2019-09-26 13:56:19 UTC
Request ID '20170925135631':
	status: MONITORING
	subject: CN=mgmt6.testrelm.test,O=TESTRELM.TEST
	expires: 2019-09-26 13:56:31 UTC
# date -s "715 days";sleep 300
Tue Sep 10 17:05:33 EDT 2019
# getcert list | egrep "status|expires|Request|subject|ca-error"
Request ID '20170925135424':
	status: MONITORING
	subject: CN=IPA RA,O=TESTRELM.TEST
	expires: 2021-08-30 21:07:09 UTC
Request ID '20170925135504':
	status: MONITORING
	subject: CN=CA Audit,O=TESTRELM.TEST
	expires: 2021-08-30 21:07:27 UTC
Request ID '20170925135505':
	status: MONITORING
	subject: CN=OCSP Subsystem,O=TESTRELM.TEST
	expires: 2021-08-30 21:07:28 UTC
Request ID '20170925135506':
	status: MONITORING
	subject: CN=CA Subsystem,O=TESTRELM.TEST
	expires: 2021-08-30 21:07:08 UTC
Request ID '20170925135507':
	status: MONITORING
	subject: CN=Certificate Authority,O=TESTRELM.TEST
	expires: 2037-09-25 13:54:01 UTC
Request ID '20170925135508':
	status: MONITORING
	subject: CN=mgmt6.testrelm.test,O=TESTRELM.TEST
	expires: 2021-08-30 21:07:38 UTC
Request ID '20170925135538':
	status: MONITORING
	subject: CN=mgmt6.testrelm.test,O=TESTRELM.TEST
	expires: 2021-09-10 21:07:12 UTC
Request ID '20170925135618':
	status: MONITORING
	subject: CN=mgmt6.testrelm.test,O=TESTRELM.TEST
	expires: 2021-09-10 21:07:22 UTC
Request ID '20170925135631':
	status: MONITORING
	subject: CN=mgmt6.testrelm.test,O=TESTRELM.TEST
	expires: 2021-09-10 21:07:04 UTC
# kinit admin
Password for admin: 
Password expired.  You must change it now.
Enter new password: 
Enter it again: 
# ipa cert-show 1
  Issuing CA: ipa
  Certificate: MIIDkTCCAnmgAwIBAgIBATANBgkqhkiG9w0BAQsFADA4MRYwFAYDVQQKDA1URVNUUkVMTS5URVNUMR4wHAYDVQQDDBVDZXJ0aWZpY2F0ZSBBdXRob3JpdHkwHhcNMTcwOTI1MTM1NDAxWhcNMzcwOTI1MTM1NDAxWjA4MRYwFAYDVQQKDA1URVNUUkVMTS5URVNUMR4wHAYDVQQDDBVDZXJ0aWZpY2F0ZSBBdXRob3JpdHkwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDY2jv3bFDdhyYEO9YlLnaTPcK2GAZcWSfamgowWio024UvfhjOb0jndM0ko6HdFCWTXTHvONylTE/HCseqCGNIMIw970CLPp7hbuZ/TMm2PIIbbXqzBq3r4bk1JJTISVdxKs4lnyTiJihH3cDlzLMoWu7DxPHP0yoywSDWdyyljrbKuO7Lu04Z3ZnSvHelbzRUdCNEyPyAnAA941J44m/jwVg4xQZ8R/M24HpVtlB5Oi15PNL21W2W7cFQ01d4s+c/Ab7kOBv37r6zxHAfBdV8hKpIlpcV1rS1mTy2wKGU33y+YeQVL1p+Yjoq4M0G7mpojGeFdRc9/1VZFyroNLyLAgMBAAGjgaUwgaIwHwYDVR0jBBgwFoAUR9v2Y1VesiOALKfJWj3qDlSwGVMwDwYDVR0TAQH/BAUwAwEB/zAOBgNVHQ8BAf8EBAMCAcYwHQYDVR0OBBYEFEfb9mNVXrIjgCynyVo96g5UsBlTMD8GCCsGAQUFBwEBBDMwMTAvBggrBgEFBQcwAYYjaHR0cDovL2lwYS1jYS50ZXN0cmVsbS50ZXN0L2NhL29jc3AwDQYJKoZIhvcNAQELBQADggEBAMPZkWas54cMhbWK+RXh7DbR2lR3V1AFMatra4Gw7qtfowSa1SCoxrwNuBpGnI1oxgWcVMk1Gbqe1cT85DBdjEnRk0vyJ/Ja5fIrLA0TvP/41dnN+IveGfL/k1xlpdRf+yVex5uJBs0eTs4Ohqf6So3AqLgru5ELiOjU8PerjFDp9J3vCVXc60nUFWIgt5pFvxkfJDAC23+eF/tZkfV2MncM3tXcpyC8ulkaZP2HwhUEa6MQ4tVhmabGYotQE5XAGhlKGdq5BV4m0KdZnrR0YTHuec5szw+SxfHea7zwpL1DzIrfuemSvwNjR3rso5MmD/pcxmWeGdzc9c+H+ysG0oU=
  Subject: CN=Certificate Authority,O=TESTRELM.TEST
  Issuer: CN=Certificate Authority,O=TESTRELM.TEST
  Not Before: Mon Sep 25 13:54:01 2017 UTC
  Not After: Fri Sep 25 13:54:01 2037 UTC
  Serial number: 1
  Serial number (hex): 0x1
  Revoked: False
# date -s "715 days";sleep 300
Wed Aug 25 17:13:45 EDT 2021
[root@mgmt6 ~]# getcert list | egrep "status|expires|Request|subject|ca-error"
Request ID '20170925135424':
	status: MONITORING
	subject: CN=IPA RA,O=TESTRELM.TEST
	expires: 2023-08-15 21:17:09 UTC
Request ID '20170925135504':
	status: MONITORING
	subject: CN=CA Audit,O=TESTRELM.TEST
	expires: 2023-08-15 21:17:10 UTC
Request ID '20170925135505':
	status: MONITORING
	subject: CN=OCSP Subsystem,O=TESTRELM.TEST
	expires: 2023-08-15 21:17:40 UTC
Request ID '20170925135506':
	status: MONITORING
	subject: CN=CA Subsystem,O=TESTRELM.TEST
	expires: 2023-08-15 21:17:20 UTC
Request ID '20170925135507':
	status: MONITORING
	subject: CN=Certificate Authority,O=TESTRELM.TEST
	expires: 2037-09-25 13:54:01 UTC
Request ID '20170925135508':
	status: POST_SAVED_CERT
	subject: CN=mgmt6.testrelm.test,O=TESTRELM.TEST
	expires: 2023-08-15 21:17:20 UTC
Request ID '20170925135538':
	status: MONITORING
	subject: CN=mgmt6.testrelm.test,O=TESTRELM.TEST
	expires: 2023-08-26 21:17:14 UTC
Request ID '20170925135618':
	status: MONITORING
	subject: CN=mgmt6.testrelm.test,O=TESTRELM.TEST
	expires: 2023-08-26 21:17:24 UTC
Request ID '20170925135631':
	status: MONITORING
	subject: CN=mgmt6.testrelm.test,O=TESTRELM.TEST
	expires: 2023-08-26 21:17:05 UTC
[root@mgmt6 ~]# getcert list | egrep "status|expires|Request|subject|ca-error"
Request ID '20170925135424':
	status: MONITORING
	subject: CN=IPA RA,O=TESTRELM.TEST
	expires: 2023-08-15 21:17:09 UTC
Request ID '20170925135504':
	status: MONITORING
	subject: CN=CA Audit,O=TESTRELM.TEST
	expires: 2023-08-15 21:17:10 UTC
Request ID '20170925135505':
	status: MONITORING
	subject: CN=OCSP Subsystem,O=TESTRELM.TEST
	expires: 2023-08-15 21:17:40 UTC
Request ID '20170925135506':
	status: MONITORING
	subject: CN=CA Subsystem,O=TESTRELM.TEST
	expires: 2023-08-15 21:17:20 UTC
Request ID '20170925135507':
	status: MONITORING
	subject: CN=Certificate Authority,O=TESTRELM.TEST
	expires: 2037-09-25 13:54:01 UTC
Request ID '20170925135508':
	status: MONITORING
	subject: CN=mgmt6.testrelm.test,O=TESTRELM.TEST
	expires: 2023-08-15 21:17:20 UTC
Request ID '20170925135538':
	status: MONITORING
	subject: CN=mgmt6.testrelm.test,O=TESTRELM.TEST
	expires: 2023-08-26 21:17:14 UTC
Request ID '20170925135618':
	status: MONITORING
	subject: CN=mgmt6.testrelm.test,O=TESTRELM.TEST
	expires: 2023-08-26 21:17:24 UTC
Request ID '20170925135631':
	status: MONITORING
	subject: CN=mgmt6.testrelm.test,O=TESTRELM.TEST
	expires: 2023-08-26 21:17:05 UTC
# kinit admin
Password for admin: 
Password expired.  You must change it now.
Enter new password: 
Enter it again:
# ipa cert-show 1
  Issuing CA: ipa
  Certificate: MIIDkTCCAnmgAwIBAgIBATANBgkqhkiG9w0BAQsFADA4MRYwFAYDVQQKDA1URVNUUkVMTS5URVNUMR4wHAYDVQQDDBVDZXJ0aWZpY2F0ZSBBdXRob3JpdHkwHhcNMTcwOTI1MTM1NDAxWhcNMzcwOTI1MTM1NDAxWjA4MRYwFAYDVQQKDA1URVNUUkVMTS5URVNUMR4wHAYDVQQDDBVDZXJ0aWZpY2F0ZSBBdXRob3JpdHkwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDY2jv3bFDdhyYEO9YlLnaTPcK2GAZcWSfamgowWio024UvfhjOb0jndM0ko6HdFCWTXTHvONylTE/HCseqCGNIMIw970CLPp7hbuZ/TMm2PIIbbXqzBq3r4bk1JJTISVdxKs4lnyTiJihH3cDlzLMoWu7DxPHP0yoywSDWdyyljrbKuO7Lu04Z3ZnSvHelbzRUdCNEyPyAnAA941J44m/jwVg4xQZ8R/M24HpVtlB5Oi15PNL21W2W7cFQ01d4s+c/Ab7kOBv37r6zxHAfBdV8hKpIlpcV1rS1mTy2wKGU33y+YeQVL1p+Yjoq4M0G7mpojGeFdRc9/1VZFyroNLyLAgMBAAGjgaUwgaIwHwYDVR0jBBgwFoAUR9v2Y1VesiOALKfJWj3qDlSwGVMwDwYDVR0TAQH/BAUwAwEB/zAOBgNVHQ8BAf8EBAMCAcYwHQYDVR0OBBYEFEfb9mNVXrIjgCynyVo96g5UsBlTMD8GCCsGAQUFBwEBBDMwMTAvBggrBgEFBQcwAYYjaHR0cDovL2lwYS1jYS50ZXN0cmVsbS50ZXN0L2NhL29jc3AwDQYJKoZIhvcNAQELBQADggEBAMPZkWas54cMhbWK+RXh7DbR2lR3V1AFMatra4Gw7qtfowSa1SCoxrwNuBpGnI1oxgWcVMk1Gbqe1cT85DBdjEnRk0vyJ/Ja5fIrLA0TvP/41dnN+IveGfL/k1xlpdRf+yVex5uJBs0eTs4Ohqf6So3AqLgru5ELiOjU8PerjFDp9J3vCVXc60nUFWIgt5pFvxkfJDAC23+eF/tZkfV2MncM3tXcpyC8ulkaZP2HwhUEa6MQ4tVhmabGYotQE5XAGhlKGdq5BV4m0KdZnrR0YTHuec5szw+SxfHea7zwpL1DzIrfuemSvwNjR3rso5MmD/pcxmWeGdzc9c+H+ysG0oU=
  Subject: CN=Certificate Authority,O=TESTRELM.TEST
  Issuer: CN=Certificate Authority,O=TESTRELM.TEST
  Not Before: Mon Sep 25 13:54:01 2017 UTC
  Not After: Fri Sep 25 13:54:01 2037 UTC
  Serial number: 1
  Serial number (hex): 0x1
  Revoked: False

Comment 37 errata-xmlrpc 2018-04-10 12:38:21 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:0763

Note You need to log in before you can comment on or make changes to this bug.