Bug 1623668
Summary: | Replica install: certmonger sometimes fails [rhel-7.5.z] | ||
---|---|---|---|
Product: | Red Hat Enterprise Linux 7 | Reporter: | Jaroslav Reznik <jreznik> |
Component: | ipa | Assignee: | IPA Maintainers <ipa-maint> |
Status: | CLOSED ERRATA | QA Contact: | ipa-qe <ipa-qe> |
Severity: | urgent | Docs Contact: | Filip Hanzelka <fhanzelk> |
Priority: | urgent | ||
Version: | 7.5 | CC: | fhanzelk, frenaud, ipa-maint, ndehadra, pvoborni, rcritten, tscherf |
Target Milestone: | rc | Keywords: | ZStream |
Target Release: | --- | ||
Hardware: | Unspecified | ||
OS: | Unspecified | ||
Whiteboard: | |||
Fixed In Version: | ipa-4.5.4-10.el7_5.4.2 | Doc Type: | If docs needed, set a value |
Doc Text: | Story Points: | --- | |
Clone Of: | 1623113 | Environment: | |
Last Closed: | 2018-09-25 19:07:13 UTC | Type: | --- |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: | |||
Bug Depends On: | 1623113 | ||
Bug Blocks: |
Description
Jaroslav Reznik
2018-08-29 20:44:30 UTC
ipa-server-version: ipa-server-4.5.4-10.el7_5.4.3.x86_64 Tested the bug with following observations: 1. Setup IPA-Master 2. Install 3 or more replicas simultaneously ( In my case I installed 5 replicas in parallel / simultaneously) 3. Run following command on REPLICA # grep -rn "CA_REJECTED\|CA_UNREACHABLE\|RuntimeError" /var/log/ipareplica-install.log 4. Try creating a new cert request on any replica Observations: 1. While replica installation is in progress noticed message across all replicas: Configuring Kerberos KDC (krb5kdc) [1/1]: installing X509 Certificate for PKINIT Full PKINIT configuration did not succeed The setup will only install bits essential to the server functionality You can enable PKINIT after the setup completed using 'ipa-pkinit-manage' Done configuring Kerberos KDC (krb5kdc). Applying LDAP updates Upgrading IPA:. Estimated time: 1 minute 30 seconds [1/9]: stopping directory server For above issue BZ1623486, is already logged. 2. After step2, replica installation is successful on all the systems. 3. After step 3, following error message is received (RPC failed at server. an internal error has occurred).) [root@vm-idm-028 ~]# grep -rn "CA_REJECTED\|CA_UNREACHABLE\|RuntimeError" /var/log/ipareplica-install.log 3788:2018-09-04T07:41:05Z DEBUG certmonger request is in state dbus.String(u'CA_UNREACHABLE', variant_level=1) 3789:2018-09-04T07:41:05Z DEBUG Cert request 20180904074045 failed: CA_UNREACHABLE (Server at https://vm-idm-028.testrelm.test/ipa/xml failed request, will retry: 903 (RPC failed at server. an internal error has occurred).) 3792:2018-09-04T07:41:20Z DEBUG certmonger request is in state dbus.String(u'CA_UNREACHABLE', variant_level=1) 3793:2018-09-04T07:41:20Z DEBUG Cert request 20180904074045 failed: CA_UNREACHABLE (Server at https://vm-idm-028.testrelm.test/ipa/xml failed request, will retry: 903 (RPC failed at server. an internal error has occurred).) 3796:2018-09-04T07:41:35Z DEBUG certmonger request is in state dbus.String(u'CA_UNREACHABLE', variant_level=1) 3797:2018-09-04T07:41:35Z DEBUG Cert request 20180904074045 failed: CA_UNREACHABLE (Server at https://vm-idm-028.testrelm.test/ipa/xml failed request, will retry: 903 (RPC failed at server. an internal error has occurred).) 3800:2018-09-04T07:41:50Z DEBUG certmonger request is in state dbus.String(u'CA_UNREACHABLE', variant_level=1) 3801:2018-09-04T07:41:50Z DEBUG Cert request 20180904074045 failed: CA_UNREACHABLE (Server at https://vm-idm-028.testrelm.test/ipa/xml failed request, will retry: 903 (RPC failed at server. an internal error has occurred).) 3804:2018-09-04T07:42:05Z DEBUG certmonger request is in state dbus.String(u'CA_UNREACHABLE', variant_level=1) 3805:2018-09-04T07:42:05Z DEBUG Cert request 20180904074045 failed: CA_UNREACHABLE (Server at https://vm-idm-028.testrelm.test/ipa/xml failed request, will retry: 903 (RPC failed at server. an internal error has occurred).) 3808:2018-09-04T07:42:20Z DEBUG certmonger request is in state dbus.String(u'CA_UNREACHABLE', variant_level=1) 3809:2018-09-04T07:42:20Z DEBUG Cert request 20180904074045 failed: CA_UNREACHABLE (Server at https://vm-idm-028.testrelm.test/ipa/xml failed request, will retry: 903 (RPC failed at server. an internal error has occurred).) 3812:2018-09-04T07:42:35Z DEBUG certmonger request is in state dbus.String(u'CA_UNREACHABLE', variant_level=1) 3813:2018-09-04T07:42:35Z DEBUG Cert request 20180904074045 failed: CA_UNREACHABLE (Server at https://vm-idm-028.testrelm.test/ipa/xml failed request, will retry: 903 (RPC failed at server. an internal error has occurred).) 3816:2018-09-04T07:42:51Z DEBUG certmonger request is in state dbus.String(u'CA_UNREACHABLE', variant_level=1) 3817:2018-09-04T07:42:51Z DEBUG Cert request 20180904074045 failed: CA_UNREACHABLE (Server at https://vm-idm-028.testrelm.test/ipa/xml failed request, will retry: 903 (RPC failed at server. an internal error has occurred).) 3820:2018-09-04T07:43:06Z DEBUG certmonger request is in state dbus.String(u'CA_UNREACHABLE', variant_level=1) 3821:2018-09-04T07:43:06Z DEBUG Cert request 20180904074045 failed: CA_UNREACHABLE (Server at https://vm-idm-028.testrelm.test/ipa/xml failed request, will retry: 903 (RPC failed at server. an internal error has occurred).) 3824:2018-09-04T07:43:21Z DEBUG certmonger request is in state dbus.String(u'CA_UNREACHABLE', variant_level=1) 3825:2018-09-04T07:43:21Z DEBUG Cert request 20180904074045 failed: CA_UNREACHABLE (Server at https://vm-idm-028.testrelm.test/ipa/xml failed request, will retry: 903 (RPC failed at server. an internal error has occurred).) 3828:2018-09-04T07:43:36Z DEBUG certmonger request is in state dbus.String(u'CA_UNREACHABLE', variant_level=1) 3829:2018-09-04T07:43:36Z DEBUG Cert request 20180904074045 failed: CA_UNREACHABLE (Server at https://vm-idm-028.testrelm.test/ipa/xml failed request, will retry: 903 (RPC failed at server. an internal error has occurred).) 3832:2018-09-04T07:43:51Z DEBUG certmonger request is in state dbus.String(u'CA_UNREACHABLE', variant_level=1) 3833:2018-09-04T07:43:51Z DEBUG Cert request 20180904074045 failed: CA_UNREACHABLE (Server at https://vm-idm-028.testrelm.test/ipa/xml failed request, will retry: 903 (RPC failed at server. an internal error has occurred).) 3836:2018-09-04T07:44:06Z DEBUG certmonger request is in state dbus.String(u'CA_UNREACHABLE', variant_level=1) 3837:2018-09-04T07:44:06Z DEBUG Cert request 20180904074045 failed: CA_UNREACHABLE (Server at https://vm-idm-028.testrelm.test/ipa/xml failed request, will retry: 903 (RPC failed at server. an internal error has occurred).) 3840:2018-09-04T07:44:21Z DEBUG certmonger request is in state dbus.String(u'CA_UNREACHABLE', variant_level=1) 3841:2018-09-04T07:44:21Z DEBUG Cert request 20180904074045 failed: CA_UNREACHABLE (Server at https://vm-idm-028.testrelm.test/ipa/xml failed request, will retry: 903 (RPC failed at server. an internal error has occurred).) 3844:2018-09-04T07:44:36Z DEBUG certmonger request is in state dbus.String(u'CA_UNREACHABLE', variant_level=1) 3845:2018-09-04T07:44:36Z DEBUG Cert request 20180904074045 failed: CA_UNREACHABLE (Server at https://vm-idm-028.testrelm.test/ipa/xml failed request, will retry: 903 (RPC failed at server. an internal error has occurred).) 3848:2018-09-04T07:44:51Z DEBUG certmonger request is in state dbus.String(u'CA_UNREACHABLE', variant_level=1) 3849:2018-09-04T07:44:51Z DEBUG Cert request 20180904074045 failed: CA_UNREACHABLE (Server at https://vm-idm-028.testrelm.test/ipa/xml failed request, will retry: 903 (RPC failed at server. an internal error has occurred).) 3852:2018-09-04T07:45:06Z DEBUG certmonger request is in state dbus.String(u'CA_UNREACHABLE', variant_level=1) 3853:2018-09-04T07:45:06Z DEBUG Cert request 20180904074045 failed: CA_UNREACHABLE (Server at https://vm-idm-028.testrelm.test/ipa/xml failed request, will retry: 903 (RPC failed at server. an internal error has occurred).) 3856:2018-09-04T07:45:21Z DEBUG certmonger request is in state dbus.String(u'CA_UNREACHABLE', variant_level=1) 3857:2018-09-04T07:45:21Z DEBUG Cert request 20180904074045 failed: CA_UNREACHABLE (Server at https://vm-idm-028.testrelm.test/ipa/xml failed request, will retry: 903 (RPC failed at server. an internal error has occurred).) 3860:2018-09-04T07:45:37Z DEBUG certmonger request is in state dbus.String(u'CA_UNREACHABLE', variant_level=1) 3861:2018-09-04T07:45:37Z DEBUG Cert request 20180904074045 failed: CA_UNREACHABLE (Server at https://vm-idm-028.testrelm.test/ipa/xml failed request, will retry: 903 (RPC failed at server. an internal error has occurred).) 3864:2018-09-04T07:45:52Z DEBUG certmonger request is in state dbus.String(u'CA_UNREACHABLE', variant_level=1) 3865:2018-09-04T07:45:52Z DEBUG Cert request 20180904074045 failed: CA_UNREACHABLE (Server at https://vm-idm-028.testrelm.test/ipa/xml failed request, will retry: 903 (RPC failed at server. an internal error has occurred).) 3867:2018-09-04T07:45:52Z WARNING PKINIT certificate request failed: Certificate issuance failed (CA_UNREACHABLE: Server at https://vm-idm-028.testrelm.test/ipa/xml failed request, will retry: 903 (RPC failed at server. an internal error has occurred).) 4. After step4, Creation of new certs results in CA_REJECTED status. [root@vm-idm-028 ~]# rpm -q ipa-server ipa-server-4.5.4-10.el7_5.4.3.x86_64 [root@vm-idm-028 ~]# ipa dnsrecord-add testrelm.test www --a-rec 192.168.0.101 Record name: www A record: 192.168.0.101 [root@vm-idm-028 ~]# ipa host-add www.testrelm.test ------------------------------ Added host "www.testrelm.test" ------------------------------ Host name: www.testrelm.test Principal name: host/www.testrelm.test Principal alias: host/www.testrelm.test Password: False Keytab: False Managed by: www.testrelm.test [root@vm-idm-028 ~]# #ipa-getcert request -r -f /etc/pki/tls/certs/www.testrelm.test.crt -k /etc/pki/tls/private/www.testrelm.test.key -N CN=www.testrelm.test -D www.testrelm.test -K HTTP/www.testrelm.test [root@vm-idm-028 ~]# ipa service-add HTTP/www.testrelm.test ---------------------------------------------------- Added service "HTTP/www.testrelm.test" ---------------------------------------------------- Principal name: HTTP/www.testrelm.test Principal alias: HTTP/www.testrelm.test Managed by: www.testrelm.test [root@vm-idm-028 ~]# ipa-getcert request -r -f /etc/pki/tls/certs/www.testrelm.test.crt -k /etc/pki/tls/private/www.testrelm.test.key -N CN=www.testrelm.test -D www.testrelm.test -K HTTP/www.testrelm.test New signing request "20180904090332" added. [root@vm-idm-028 ~]# ipa-getcert list Number of certificates and requests being tracked: 11. Request ID '20180904073448': status: MONITORING stuck: no key pair storage: type=NSSDB,location='/etc/dirsrv/slapd-TESTRELM-TEST',nickname='Server-Cert',token='NSS Certificate DB',pinfile='/etc/dirsrv/slapd-TESTRELM-TEST/pwdfile.txt' certificate: type=NSSDB,location='/etc/dirsrv/slapd-TESTRELM-TEST',nickname='Server-Cert',token='NSS Certificate DB' CA: IPA issuer: CN=Certificate Authority,O=TESTRELM.TEST subject: CN=vm-idm-028.testrelm.test,O=TESTRELM.TEST expires: 2020-09-04 07:34:49 UTC dns: vm-idm-028.testrelm.test principal name: ldap/vm-idm-028.testrelm.test key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: post-save command: /usr/libexec/ipa/certmonger/restart_dirsrv TESTRELM-TEST track: yes auto-renew: yes Request ID '20180904073517': status: MONITORING stuck: no key pair storage: type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt' certificate: type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS Certificate DB' CA: IPA issuer: CN=Certificate Authority,O=TESTRELM.TEST subject: CN=vm-idm-028.testrelm.test,O=TESTRELM.TEST expires: 2020-09-04 07:35:18 UTC dns: vm-idm-028.testrelm.test principal name: HTTP/vm-idm-028.testrelm.test key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: post-save command: /usr/libexec/ipa/certmonger/restart_httpd track: yes auto-renew: yes Request ID '20180904090332': status: CA_REJECTED ca-error: Server at https://vm-idm-028.testrelm.test/ipa/xml denied our request, giving up: 2100 (RPC failed at server. Insufficient access: Insufficient 'write' privilege to the 'userCertificate' attribute of entry 'krbprincipalname=HTTP/www.testrelm.test,cn=services,cn=accounts,dc=testrelm,dc=test'.). stuck: yes key pair storage: type=FILE,location='/etc/pki/tls/private/www.testrelm.test.key' certificate: type=FILE,location='/etc/pki/tls/certs/www.testrelm.test.crt' CA: IPA issuer: subject: expires: unknown pre-save command: post-save command: track: yes auto-renew: yes [root@vm-idm-028 ~]# Thus on the basis of above observations in step3 and step4, marking the status of bug to "ASSIGNED". Hi Nikhil, the procedure is missing a step allowing the replica to request cert for the service HTTP/www.testrelm.test: $ ipa service-add-host --hosts=<replica> HTTP/www.testrelm.test Could you retry with the above step right before the ipa-getcert request? (In reply to Florence Blanc-Renaud from comment #4) > Hi Nikhil, > > the procedure is missing a step allowing the replica to request cert for the > service HTTP/www.testrelm.test: > > $ ipa service-add-host --hosts=<replica> HTTP/www.testrelm.test > > Could you retry with the above step right before the ipa-getcert request? Following the steps on REPLICA, I am no more seeing status as 'CA_REJECTED' but now the status is 'MONITORING' #REPLICA [root@vm-idm-034 ~]# rpm -q ipa-server ipa-server-4.5.4-10.el7_5.4.3.x86_64 [root@vm-idm-034 ~]# ipa dnsrecord-add testrelm.test www --a-rec 192.168.0.101 Record name: www A record: 192.168.0.101 [root@vm-idm-034 ~]# ipa host-add www.testrelm.test ------------------------------ Added host "www.testrelm.test" ------------------------------ Host name: www.testrelm.test Principal name: host/www.testrelm.test Principal alias: host/www.testrelm.test Password: False Keytab: False Managed by: www.testrelm.test [root@vm-idm-034 ~]# hostname vm-idm-034.testrelm.test [root@vm-idm-034 ~]# ipa service-add HTTP/www.testrelm.test ---------------------------------------------------- Added service "HTTP/www.testrelm.test" ---------------------------------------------------- Principal name: HTTP/www.testrelm.test Principal alias: HTTP/www.testrelm.test Managed by: www.testrelm.test [root@vm-idm-034 ~]# ipa service-add-host --hosts=`hostname` HTTP/www.testrelm.test Principal name: HTTP/www.testrelm.test Principal alias: HTTP/www.testrelm.test Managed by: www.testrelm.test, vm-idm-034.testrelm.test ------------------------- Number of members added 1 ------------------------- [root@vm-idm-034 ~]# ipa-getcert request -r -f /etc/pki/tls/certs/www.testrelm.test.crt -k /etc/pki/tls/private/www.testrelm.test.key -N CN=www.testrelm.test -D www.testrelm.test -K HTTP/www.testrelm.test New signing request "20180905080952" added. [root@vm-idm-034 ~]# ipa-getcert list Number of certificates and requests being tracked: 10. Request ID '20180904093627': status: MONITORING stuck: no key pair storage: type=NSSDB,location='/etc/dirsrv/slapd-TESTRELM-TEST',nickname='Server-Cert',token='NSS Certificate DB',pinfile='/etc/dirsrv/slapd-TESTRELM-TEST/pwdfile.txt' certificate: type=NSSDB,location='/etc/dirsrv/slapd-TESTRELM-TEST',nickname='Server-Cert',token='NSS Certificate DB' CA: IPA issuer: CN=Certificate Authority,O=TESTRELM.TEST subject: CN=vm-idm-034.testrelm.test,O=TESTRELM.TEST expires: 2020-09-04 09:36:29 UTC dns: vm-idm-034.testrelm.test principal name: ldap/vm-idm-034.testrelm.test key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: post-save command: /usr/libexec/ipa/certmonger/restart_dirsrv TESTRELM-TEST track: yes auto-renew: yes Request ID '20180904093702': status: MONITORING stuck: no key pair storage: type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt' certificate: type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS Certificate DB' CA: IPA issuer: CN=Certificate Authority,O=TESTRELM.TEST subject: CN=vm-idm-034.testrelm.test,O=TESTRELM.TEST expires: 2020-09-04 09:37:03 UTC dns: vm-idm-034.testrelm.test principal name: HTTP/vm-idm-034.testrelm.test key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: post-save command: /usr/libexec/ipa/certmonger/restart_httpd track: yes auto-renew: yes Request ID '20180905080952': status: MONITORING stuck: no key pair storage: type=FILE,location='/etc/pki/tls/private/www.testrelm.test.key' certificate: type=FILE,location='/etc/pki/tls/certs/www.testrelm.test.crt' CA: IPA issuer: CN=Certificate Authority,O=TESTRELM.TEST subject: CN=www.testrelm.test,O=TESTRELM.TEST expires: 2020-09-05 08:09:55 UTC dns: www.testrelm.test principal name: HTTP/www.testrelm.test key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: post-save command: track: yes auto-renew: yes [root@vm-idm-034 ~]# As ipa-getcert is now successful, moving back to ON_QA ipa-server-version: ipa-server-4.5.4-10.el7_5.4.3.x86_64 Verified the bug with following observations: 1. Setup IPA-Master 2. Install 3 or more replicas simultaneously ( In my case I installed 5 replicas in parallel / simultaneously) 3. Run following command on REPLICA # grep -rn "CA_REJECTED\|CA_UNREACHABLE\|RuntimeError" /var/log/ipareplica-install.log 4. Try creating a new cert request on any replica Observations: 1. While replica installation is in progress noticed message across all replicas: Configuring Kerberos KDC (krb5kdc) [1/1]: installing X509 Certificate for PKINIT Full PKINIT configuration did not succeed The setup will only install bits essential to the server functionality You can enable PKINIT after the setup completed using 'ipa-pkinit-manage' Done configuring Kerberos KDC (krb5kdc). Applying LDAP updates Upgrading IPA:. Estimated time: 1 minute 30 seconds [1/9]: stopping directory server For above issue BZ1623486, is already logged. 2. After step2, replica installation is successful on all the systems. 3. After step 3, following error message is received (RPC failed at server. an internal error has occurred).), these errors are observed due to step1 and it will be tracked in separate bug BZ1623486 Console: While Configuring Kerberos KDC (krb5kdc) Step, following message is received: [1/1]: installing X509 Certificate for PKINIT Full PKINIT configuration did not succeed The setup will only install bits essential to the server functionality You can enable PKINIT after the setup completed using 'ipa-pkinit-manage' Done configuring Kerberos KDC (krb5kdc). Applying LDAP updates Upgrading IPA:. Estimated time: 1 minute 30 seconds replica-install.log: ------------------------ 2018-09-05T08:48:39Z DEBUG Configuring Kerberos KDC (krb5kdc) 2018-09-05T08:48:39Z DEBUG [1/1]: installing X509 Certificate for PKINIT 2018-09-05T08:48:40Z DEBUG certmonger request is in state dbus.String(u'NEWLY_ADDED_READING_KEYINFO', variant_level=1) 2018-09-05T08:48:45Z DEBUG certmonger request is in state dbus.String(u'SUBMITTING', variant_level=1) 2018-09-05T08:48:50Z DEBUG certmonger request is in state dbus.String(u'SUBMITTING', variant_level=1) 2018-09-05T08:48:55Z DEBUG certmonger request is in state dbus.String(u'CA_UNREACHABLE', variant_level=1) 2018-09-05T08:48:55Z DEBUG Cert request 20180905084839 failed: CA_UNREACHABLE (Server at https://vm-idm-034.testrelm.test/ipa/xml failed request, will retry: 903 (RPC failed at server. an internal error has occurred).) 2018-09-05T08:48:55Z DEBUG Sleep and resubmit cert request 20180905084839 2018-09-05T08:49:05Z DEBUG certmonger request is in state dbus.String(u'GENERATING_CSR', variant_level=1) 2018-09-05T08:49:10Z DEBUG certmonger request is in state dbus.String(u'CA_UNREACHABLE', variant_level=1) 2018-09-05T08:49:10Z DEBUG Cert request 20180905084839 failed: CA_UNREACHABLE (Server at https://vm-idm-034.testrelm.test/ipa/xml failed request, will retry: 903 (RPC failed at server. an internal error has occurred).) 2018-09-05T08:49:10Z DEBUG Sleep and resubmit cert request 20180905084839 2018-09-05T08:49:20Z DEBUG certmonger request is in state dbus.String(u'GENERATING_CSR', variant_level=1) 2018-09-05T08:49:25Z DEBUG certmonger request is in state dbus.String(u'CA_UNREACHABLE', variant_level=1) 2018-09-05T08:49:25Z DEBUG Cert request 20180905084839 failed: CA_UNREACHABLE (Server at https://vm-idm-034.testrelm.test/ipa/xml failed request, will retry: 903 (RPC failed at server. an internal error has occurred).) 2018-09-05T08:49:25Z DEBUG Sleep and resubmit cert request 20180905084839 2018-09-05T08:49:35Z DEBUG certmonger request is in state dbus.String(u'GENERATING_CSR', variant_level=1) 2018-09-05T08:49:40Z DEBUG certmonger request is in state dbus.String(u'CA_UNREACHABLE', variant_level=1) 2018-09-05T08:49:40Z DEBUG Cert request 20180905084839 failed: CA_UNREACHABLE (Server at https://vm-idm-034.testrelm.test/ipa/xml failed request, will retry: 903 (RPC failed at server. an internal error has occurred).) 2018-09-05T08:49:40Z DEBUG Sleep and resubmit cert request 20180905084839 2018-09-05T08:49:50Z DEBUG certmonger request is in state dbus.String(u'GENERATING_CSR', variant_level=1) 2018-09-05T08:49:55Z DEBUG certmonger request is in state dbus.String(u'CA_UNREACHABLE', variant_level=1) 2018-09-05T08:49:55Z DEBUG Cert request 20180905084839 failed: CA_UNREACHABLE (Server at https://vm-idm-034.testrelm.test/ipa/xml failed request, will retry: 903 (RPC failed at server. an internal error has occurred).) 2018-09-05T08:49:55Z DEBUG Sleep and resubmit cert request 20180905084839 2018-09-05T08:50:05Z DEBUG certmonger request is in state dbus.String(u'GENERATING_CSR', variant_level=1) 2018-09-05T08:50:10Z DEBUG certmonger request is in state dbus.String(u'CA_UNREACHABLE', variant_level=1) 2018-09-05T08:50:10Z DEBUG Cert request 20180905084839 failed: CA_UNREACHABLE (Server at https://vm-idm-034.testrelm.test/ipa/xml failed request, will retry: 903 (RPC failed at server. an internal error has occurred).) 2018-09-05T08:50:10Z DEBUG Sleep and resubmit cert request 20180905084839 2018-09-05T08:50:20Z DEBUG certmonger request is in state dbus.String(u'GENERATING_CSR', variant_level=1) 2018-09-05T08:50:25Z DEBUG certmonger request is in state dbus.String(u'CA_UNREACHABLE', variant_level=1) 2018-09-05T08:50:25Z DEBUG Cert request 20180905084839 failed: CA_UNREACHABLE (Server at https://vm-idm-034.testrelm.test/ipa/xml failed request, will retry: 903 (RPC failed at server. an internal error has occurred).) 2018-09-05T08:50:25Z DEBUG Sleep and resubmit cert request 20180905084839 2018-09-05T08:50:35Z DEBUG certmonger request is in state dbus.String(u'GENERATING_CSR', variant_level=1) 2018-09-05T08:50:40Z DEBUG certmonger request is in state dbus.String(u'CA_UNREACHABLE', variant_level=1) 2018-09-05T08:50:40Z DEBUG Cert request 20180905084839 failed: CA_UNREACHABLE (Server at https://vm-idm-034.testrelm.test/ipa/xml failed request, will retry: 903 (RPC failed at server. an internal error has occurred).) 2018-09-05T08:50:40Z DEBUG Sleep and resubmit cert request 20180905084839 2018-09-05T08:50:50Z DEBUG certmonger request is in state dbus.String(u'GENERATING_CSR', variant_level=1) 2018-09-05T08:50:55Z DEBUG certmonger request is in state dbus.String(u'CA_UNREACHABLE', variant_level=1) 2018-09-05T08:50:55Z DEBUG Cert request 20180905084839 failed: CA_UNREACHABLE (Server at https://vm-idm-034.testrelm.test/ipa/xml failed request, will retry: 903 (RPC failed at server. an internal error has occurred).) 2018-09-05T08:50:55Z DEBUG Sleep and resubmit cert request 20180905084839 2018-09-05T08:51:05Z DEBUG certmonger request is in state dbus.String(u'GENERATING_CSR', variant_level=1) 2018-09-05T08:51:10Z DEBUG certmonger request is in state dbus.String(u'CA_UNREACHABLE', variant_level=1) 2018-09-05T08:51:10Z DEBUG Cert request 20180905084839 failed: CA_UNREACHABLE (Server at https://vm-idm-034.testrelm.test/ipa/xml failed request, will retry: 903 (RPC failed at server. an internal error has occurred).) 2018-09-05T08:51:10Z DEBUG Sleep and resubmit cert request 20180905084839 2018-09-05T08:51:20Z DEBUG certmonger request is in state dbus.String(u'GENERATING_CSR', variant_level=1) 2018-09-05T08:51:25Z DEBUG certmonger request is in state dbus.String(u'CA_UNREACHABLE', variant_level=1) 2018-09-05T08:51:25Z DEBUG Cert request 20180905084839 failed: CA_UNREACHABLE (Server at https://vm-idm-034.testrelm.test/ipa/xml failed request, will retry: 903 (RPC failed at server. an internal error has occurred).) 2018-09-05T08:51:25Z DEBUG Sleep and resubmit cert request 20180905084839 2018-09-05T08:51:36Z DEBUG certmonger request is in state dbus.String(u'GENERATING_CSR', variant_level=1) 2018-09-05T08:51:41Z DEBUG certmonger request is in state dbus.String(u'CA_UNREACHABLE', variant_level=1) 2018-09-05T08:51:41Z DEBUG Cert request 20180905084839 failed: CA_UNREACHABLE (Server at https://vm-idm-034.testrelm.test/ipa/xml failed request, will retry: 903 (RPC failed at server. an internal error has occurred).) 2018-09-05T08:51:41Z DEBUG Sleep and resubmit cert request 20180905084839 2018-09-05T08:51:51Z DEBUG certmonger request is in state dbus.String(u'GENERATING_CSR', variant_level=1) 2018-09-05T08:51:56Z DEBUG certmonger request is in state dbus.String(u'CA_UNREACHABLE', variant_level=1) 2018-09-05T08:51:56Z DEBUG Cert request 20180905084839 failed: CA_UNREACHABLE (Server at https://vm-idm-034.testrelm.test/ipa/xml failed request, will retry: 903 (RPC failed at server. an internal error has occurred).) 2018-09-05T08:51:56Z DEBUG Sleep and resubmit cert request 20180905084839 2018-09-05T08:52:06Z DEBUG certmonger request is in state dbus.String(u'GENERATING_CSR', variant_level=1) 2018-09-05T08:52:11Z DEBUG certmonger request is in state dbus.String(u'CA_UNREACHABLE', variant_level=1) 2018-09-05T08:52:11Z DEBUG Cert request 20180905084839 failed: CA_UNREACHABLE (Server at https://vm-idm-034.testrelm.test/ipa/xml failed request, will retry: 903 (RPC failed at server. an internal error has occurred).) 2018-09-05T08:52:11Z DEBUG Sleep and resubmit cert request 20180905084839 2018-09-05T08:52:21Z DEBUG certmonger request is in state dbus.String(u'GENERATING_CSR', variant_level=1) 2018-09-05T08:52:26Z DEBUG certmonger request is in state dbus.String(u'CA_UNREACHABLE', variant_level=1) 2018-09-05T08:52:26Z DEBUG Cert request 20180905084839 failed: CA_UNREACHABLE (Server at https://vm-idm-034.testrelm.test/ipa/xml failed request, will retry: 903 (RPC failed at server. an internal error has occurred).) 2018-09-05T08:52:26Z DEBUG Sleep and resubmit cert request 20180905084839 2018-09-05T08:52:36Z DEBUG certmonger request is in state dbus.String(u'GENERATING_CSR', variant_level=1) 2018-09-05T08:52:41Z DEBUG certmonger request is in state dbus.String(u'CA_UNREACHABLE', variant_level=1) 2018-09-05T08:52:41Z DEBUG Cert request 20180905084839 failed: CA_UNREACHABLE (Server at https://vm-idm-034.testrelm.test/ipa/xml failed request, will retry: 903 (RPC failed at server. an internal error has occurred).) 2018-09-05T08:52:41Z DEBUG Sleep and resubmit cert request 20180905084839 2018-09-05T08:52:51Z DEBUG certmonger request is in state dbus.String(u'GENERATING_CSR', variant_level=1) 2018-09-05T08:52:56Z DEBUG certmonger request is in state dbus.String(u'CA_UNREACHABLE', variant_level=1) 2018-09-05T08:52:56Z DEBUG Cert request 20180905084839 failed: CA_UNREACHABLE (Server at https://vm-idm-034.testrelm.test/ipa/xml failed request, will retry: 903 (RPC failed at server. an internal error has occurred).) 2018-09-05T08:52:56Z DEBUG Sleep and resubmit cert request 20180905084839 2018-09-05T08:53:06Z DEBUG certmonger request is in state dbus.String(u'GENERATING_CSR', variant_level=1) 2018-09-05T08:53:11Z DEBUG certmonger request is in state dbus.String(u'CA_UNREACHABLE', variant_level=1) 2018-09-05T08:53:11Z DEBUG Cert request 20180905084839 failed: CA_UNREACHABLE (Server at https://vm-idm-034.testrelm.test/ipa/xml failed request, will retry: 903 (RPC failed at server. an internal error has occurred).) 2018-09-05T08:53:11Z DEBUG Sleep and resubmit cert request 20180905084839 2018-09-05T08:53:21Z DEBUG certmonger request is in state dbus.String(u'GENERATING_CSR', variant_level=1) 2018-09-05T08:53:26Z DEBUG certmonger request is in state dbus.String(u'CA_UNREACHABLE', variant_level=1) 2018-09-05T08:53:26Z DEBUG Cert request 20180905084839 failed: CA_UNREACHABLE (Server at https://vm-idm-034.testrelm.test/ipa/xml failed request, will retry: 903 (RPC failed at server. an internal error has occurred).) 2018-09-05T08:53:26Z DEBUG Sleep and resubmit cert request 20180905084839 2018-09-05T08:53:36Z DEBUG certmonger request is in state dbus.String(u'GENERATING_CSR', variant_level=1) 2018-09-05T08:53:41Z DEBUG certmonger request is in state dbus.String(u'CA_UNREACHABLE', variant_level=1) 2018-09-05T08:53:41Z DEBUG Cert request 20180905084839 failed: CA_UNREACHABLE (Server at https://vm-idm-034.testrelm.test/ipa/xml failed request, will retry: 903 (RPC failed at server. an internal error has occurred).) 2018-09-05T08:53:41Z DEBUG Request 20180905084839 reached resubmit dead line 2018-09-05T08:53:41Z WARNING PKINIT certificate request failed: Certificate issuance failed (CA_UNREACHABLE: Server at https://vm-idm-034.testrelm.test/ipa/xml failed request, will retry: 903 (RPC failed at server. an internal error has occurred).) 2018-09-05T08:53:41Z WARNING Failed to configure PKINIT 2018-09-05T08:53:41Z DEBUG Full PKINIT configuration did not succeed 2018-09-05T08:53:41Z DEBUG The setup will only install bits essential to the server functionality 2018-09-05T08:53:41Z DEBUG You can enable PKINIT after the setup completed using 'ipa-pkinit-manage' 2018-09-05T08:53:41Z DEBUG certmonger request is in state dbus.String(u'NEWLY_ADDED_READING_KEYINFO', variant_level=1) 2018-09-05T08:53:46Z DEBUG certmonger request is in state dbus.String(u'MONITORING', variant_level=1) 2018-09-05T08:53:46Z DEBUG Cert request 20180905085341 was successful 2018-09-05T08:53:46Z DEBUG duration: 306 seconds 2018-09-05T08:53:46Z DEBUG Done configuring Kerberos KDC (krb5kdc). 2018-09-05T08:53:46Z DEBUG Starting external process 2018-09-05T08:53:46Z DEBUG args=/bin/systemctl restart krb5kdc.service 2018-09-05T08:53:46Z DEBUG Process finished, return code=0 2018-09-05T08:53:46Z DEBUG stdout= 2018-09-05T08:53:46Z DEBUG stderr= 2018-09-05T08:53:46Z DEBUG Starting external process 2018-09-05T08:53:46Z DEBUG args=/bin/systemctl is-active krb5kdc.service 2018-09-05T08:53:46Z DEBUG Process finished, return code=0 2018-09-05T08:53:46Z DEBUG stdout=active 2018-09-05T08:53:46Z DEBUG stderr= 2018-09-05T08:53:46Z DEBUG Applying LDAP updates 2018-09-05T08:53:46Z DEBUG Loading StateFile from '/var/lib/ipa/sysrestore/sysrestore.state' 2018-09-05T08:53:46Z DEBUG Loading Index file from '/var/lib/ipa/sysrestore/sysrestore.index' 2018-09-05T08:53:46Z DEBUG Starting external process 2018-09-05T08:53:46Z DEBUG args=/bin/systemctl is-active dirsrv 2018-09-05T08:53:46Z DEBUG Process finished, return code=0 2018-09-05T08:53:46Z DEBUG stdout=active 2018-09-05T08:53:46Z DEBUG stderr= 2018-09-05T08:53:46Z DEBUG Upgrading IPA:. Estimated time: 1 minute 30 seconds 4. After step4, Creation of new certs results in MONITORING status. #REPLICA [root@vm-idm-034 ~]# rpm -q ipa-server ipa-server-4.5.4-10.el7_5.4.3.x86_64 [root@vm-idm-034 ~]# ipa dnsrecord-add testrelm.test www --a-rec 192.168.0.101 Record name: www A record: 192.168.0.101 [root@vm-idm-034 ~]# ipa host-add www.testrelm.test ------------------------------ Added host "www.testrelm.test" ------------------------------ Host name: www.testrelm.test Principal name: host/www.testrelm.test Principal alias: host/www.testrelm.test Password: False Keytab: False Managed by: www.testrelm.test [root@vm-idm-034 ~]# hostname vm-idm-034.testrelm.test [root@vm-idm-034 ~]# ipa service-add HTTP/www.testrelm.test ---------------------------------------------------- Added service "HTTP/www.testrelm.test" ---------------------------------------------------- Principal name: HTTP/www.testrelm.test Principal alias: HTTP/www.testrelm.test Managed by: www.testrelm.test [root@vm-idm-034 ~]# ipa service-add-host --hosts=`hostname` HTTP/www.testrelm.test Principal name: HTTP/www.testrelm.test Principal alias: HTTP/www.testrelm.test Managed by: www.testrelm.test, vm-idm-034.testrelm.test ------------------------- Number of members added 1 ------------------------- [root@vm-idm-034 ~]# ipa-getcert request -r -f /etc/pki/tls/certs/www.testrelm.test.crt -k /etc/pki/tls/private/www.testrelm.test.key -N CN=www.testrelm.test -D www.testrelm.test -K HTTP/www.testrelm.test New signing request "20180905080952" added. [root@vm-idm-034 ~]# ipa-getcert list Number of certificates and requests being tracked: 10. Request ID '20180904093627': status: MONITORING stuck: no key pair storage: type=NSSDB,location='/etc/dirsrv/slapd-TESTRELM-TEST',nickname='Server-Cert',token='NSS Certificate DB',pinfile='/etc/dirsrv/slapd-TESTRELM-TEST/pwdfile.txt' certificate: type=NSSDB,location='/etc/dirsrv/slapd-TESTRELM-TEST',nickname='Server-Cert',token='NSS Certificate DB' CA: IPA issuer: CN=Certificate Authority,O=TESTRELM.TEST subject: CN=vm-idm-034.testrelm.test,O=TESTRELM.TEST expires: 2020-09-04 09:36:29 UTC dns: vm-idm-034.testrelm.test principal name: ldap/vm-idm-034.testrelm.test key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: post-save command: /usr/libexec/ipa/certmonger/restart_dirsrv TESTRELM-TEST track: yes auto-renew: yes Request ID '20180904093702': status: MONITORING stuck: no key pair storage: type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt' certificate: type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS Certificate DB' CA: IPA issuer: CN=Certificate Authority,O=TESTRELM.TEST subject: CN=vm-idm-034.testrelm.test,O=TESTRELM.TEST expires: 2020-09-04 09:37:03 UTC dns: vm-idm-034.testrelm.test principal name: HTTP/vm-idm-034.testrelm.test key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: post-save command: /usr/libexec/ipa/certmonger/restart_httpd track: yes auto-renew: yes Request ID '20180905080952': status: MONITORING stuck: no key pair storage: type=FILE,location='/etc/pki/tls/private/www.testrelm.test.key' certificate: type=FILE,location='/etc/pki/tls/certs/www.testrelm.test.crt' CA: IPA issuer: CN=Certificate Authority,O=TESTRELM.TEST subject: CN=www.testrelm.test,O=TESTRELM.TEST expires: 2020-09-05 08:09:55 UTC dns: www.testrelm.test principal name: HTTP/www.testrelm.test key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: post-save command: track: yes auto-renew: yes [root@vm-idm-034 ~]# Thus on the basis of above observation, marking the status of bug to 'VERIFIED' Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2018:2760 |