Bug 1774032

Summary: 'ipa-healthcheck --failures-only' when run on newly installed ipa-server lists ERROR for IPACertTracking check
Product: Red Hat Enterprise Linux 8 Reporter: Sudhir Menon <sumenon>
Component: ipa-healthcheckAssignee: Rob Crittenden <rcritten>
Status: CLOSED ERRATA QA Contact: ipa-qe <ipa-qe>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 8.2CC: fcami, ksiddiqu, ssidhaye
Target Milestone: rcKeywords: TestCaseProvided
Target Release: 8.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: ipa-healthcheck-0.4-3 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-04-28 15:44:12 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Sudhir Menon 2019-11-19 13:13:07 UTC
Description of problem: 'ipa-healthcheck --failures-only' when run on newly installed ipa-server lists ERROR for IPACertTracking check


Version-Release number of selected component (if applicable):
[root@master ~]# cat /etc/redhat-release 
Red Hat Enterprise Linux release 8.2 Beta (Ootpa)

[root@master ~]# rpm -q ipa-server
ipa-server-4.8.2-1.module+el8.2.0+4697+7171660c.x86_64

 
How reproducible: Always


Steps to Reproduce:
1. Install IPA Server and ipa-healthcheck
2. Run ipa-healthcheck --failures-only
3. Check the results.

Actual results:
[root@master ~]# ipa-healthcheck --failures-only
[
  {
    "source": "ipahealthcheck.ipa.certs",
    "check": "IPACertTracking",
    "result": "ERROR",
    "uuid": "1a21e9ab-6b88-40b1-a279-d5c4607e192a",
    "when": "20191119130346Z",
    "duration": "0.303834",
    "kw": {
      "msg": "Missing tracking for cert-database=/etc/pki/pki-tomcat/alias, cert-nickname=caSigningCert cert-pki-ca, ca-name=dogtag-ipa-ca-renew-agent, cert-presave-command=/usr/libexec/ipa/certmonger/stop_pkicad, cert-postsave-command=/usr/libexec/ipa/certmonger/renew_ca_cert \"caSigningCert cert-pki-ca\", template-profile=None"
    }
  },
  {
    "source": "ipahealthcheck.ipa.certs",
    "check": "IPACertTracking",
    "result": "WARNING",
   "uuid": "486c5905-b00b-4b74-9250-c4ef00f7665d",
    "when": "20191119130346Z",
    "duration": "0.425676",
    "kw": {
      "key": "20191118100649",
      "msg": "Unknown certmonger id 20191118100649"
    }
  }
]

Expected results: Logging this issue as bug, since the msg says that the tracking is missing for cert-database also there is a warning for unknown certmonger-id

Additional info:

Comment 1 Rob Crittenden 2019-11-19 17:15:30 UTC
Reproduced, investigating.

Comment 2 Rob Crittenden 2019-11-19 19:15:41 UTC
Upstream issue https://github.com/freeipa/freeipa-healthcheck/issues/86

Comment 3 Rob Crittenden 2019-11-19 19:19:12 UTC
The set of certs to track is hardcoded in healthcheck and there are two problems in this:

1. The caSigningCert is duplicated, one with an empty template-profile and one with the caCACert profile.
2. There is an error in the second which adds the caCACert such that it will never get added.

Drop this duplicated code and use the same call to retrieve the set of CA and KRA certs as the upgrade code.

Comment 4 Rob Crittenden 2019-11-19 19:24:04 UTC
https://github.com/freeipa/freeipa-healthcheck/pull/87

Comment 5 Rob Crittenden 2019-11-20 13:02:22 UTC
Fixed upstream:
master: fb76216e534a0ec850357e6ed0daec3ba7a161f4

Comment 8 Sumedh Sidhaye 2020-02-25 00:49:15 UTC
Steps to Reproduce:
1. Install IPA Server and ipa-healthcheck
2. Run ipa-healthcheck --failures-only
3. Check the results.


Reproducer:

[root@ci-vm-10-0-137-197 ~]# cat /etc/redhat-release 
Red Hat Enterprise Linux release 8.2 Beta (Ootpa)

[root@ci-vm-10-0-137-197 ~]# rpm -q ipa-server ipa-healthcheck
ipa-server-4.8.2-2.module+el8.2.0+4736+360582ce.x86_64
ipa-healthcheck-0.3-4.module+el8.1.0+4098+f286395e.noarch
[root@ci-vm-10-0-137-197 ~]# ipa-healthcheck --failures-only
[
  {
    "source": "ipahealthcheck.ipa.certs",
    "check": "IPACertTracking",
    "result": "ERROR",
    "uuid": "5736f224-0917-4f2f-9d80-b9a15f0e2d2f",
    "when": "20200224231758Z",
    "duration": "0.357015",
    "kw": {
      "msg": "Missing tracking for cert-database=/etc/pki/pki-tomcat/alias, cert-nickname=caSigningCert cert-pki-ca, ca-name=dogtag-ipa-ca-renew-agent, cert-presave-command=/usr/libexec/ipa/certmonger/stop_pkicad, cert-postsave-command=/usr/libexec/ipa/certmonger/renew_ca_cert \"caSigningCert cert-pki-ca\", template-profile=None"
    }
  },
  {
    "source": "ipahealthcheck.ipa.certs",
    "check": "IPACertTracking",
    "result": "WARNING",
    "uuid": "aea24076-a986-47cc-b78f-c04172d3af10",
    "when": "20200224231758Z",
    "duration": "0.487280",
    "kw": {
      "key": "20200224225723",
      "msg": "Unknown certmonger id 20200224225723"
    }
  }
]
[root@ci-vm-10-0-137-197 ~]# 




Fix:

[root@master ~]# rpm -q ipa-server ipa-healthcheck
ipa-server-4.8.4-6.module+el8.2.0+5773+68ace8c5.x86_64
ipa-healthcheck-0.4-4.module+el8.2.0+5489+95477d9f.noarch
[root@master ~]# cat /etc/redhat-release 
Red Hat Enterprise Linux release 8.2 Beta (Ootpa)
[root@master ~]# 


[root@master ~]# ipa-healthcheck --failures-only
Loading instance: pki-tomcat
Loading global Tomcat config: /etc/tomcat/tomcat.conf
Loading PKI Tomcat config: /usr/share/pki/etc/tomcat.conf
Loading instance Tomcat config: /etc/pki/pki-tomcat/tomcat.conf
Loading password config: /etc/pki/pki-tomcat/password.conf
Loading instance registry: /etc/sysconfig/pki/tomcat/pki-tomcat/pki-tomcat
Loading subsystem: ca
Loading subsystem config: /var/lib/pki/pki-tomcat/ca/conf/CS.cfg
Getting sslserver cert info for ca from CS.cfg
Getting subsystem cert info for ca from CS.cfg
Getting audit_signing cert info for ca from CS.cfg
Getting ocsp_signing cert info for ca from CS.cfg
Getting signing cert info for ca from CS.cfg
[]


Based on above observations marking Bugzilla verified.

Comment 9 Rob Crittenden 2020-04-06 16:55:33 UTC
upstream test case

master:
https://pagure.io/freeipa/c/3022bb5fd200ea3b22fb1600401e95aaee2006ea

Comment 11 Sumedh Sidhaye 2020-04-21 14:35:35 UTC
Automated test passing in CI

------------------------------ Captured log call -------------------------------
transport.py               391 INFO     RUN ['kinit', 'admin']
transport.py               513 DEBUG    RUN ['kinit', 'admin']
transport.py               558 DEBUG    Password for admin: 
transport.py               217 DEBUG    Exit code: 0
transport.py               391 INFO     RUN ['ipa', 'domainlevel-get']
transport.py               513 DEBUG    RUN ['ipa', 'domainlevel-get']
transport.py               558 DEBUG    -----------------------
transport.py               558 DEBUG    Current domain level: 1
transport.py               558 DEBUG    -----------------------
transport.py               217 DEBUG    Exit code: 0
transport.py               391 INFO     RUN ['ipa-kra-install', '-U', '-p', 'Secret.123']
transport.py               513 DEBUG    RUN ['ipa-kra-install', '-U', '-p', 'Secret.123']
transport.py               558 DEBUG    /usr/lib/python3.6/site-packages/ipaserver/install/dogtaginstance.py:72: The SecurityDomainClient.get_security_domain_info() has been deprecated (https://www.dogtagpki.org/wiki/PKI_10.8_Python_Changes).
transport.py               558 DEBUG    /usr/lib/python3.6/site-packages/ipaserver/install/dogtaginstance.py:85: The DomainInfo.systems has been deprecated (https://www.dogtagpki.org/wiki/PKI_10.8_Python_Changes).
transport.py               558 DEBUG    
transport.py               558 DEBUG    ===================================================================
transport.py               558 DEBUG    This program will setup Dogtag KRA for the IPA Server.
transport.py               558 DEBUG    
transport.py               558 DEBUG    
transport.py               558 DEBUG    Configuring KRA server (pki-tomcatd). Estimated time: 2 minutes
transport.py               558 DEBUG      [1/10]: configuring KRA instance
transport.py               558 DEBUG      [2/10]: create KRA agent
transport.py               558 DEBUG      [3/10]: enabling ephemeral requests
transport.py               558 DEBUG      [4/10]: restarting KRA
transport.py               558 DEBUG      [5/10]: configure certmonger for renewals
transport.py               558 DEBUG      [6/10]: configure certificate renewals
transport.py               558 DEBUG      [7/10]: configure HTTP to proxy connections
transport.py               558 DEBUG      [8/10]: add vault container
transport.py               558 DEBUG      [9/10]: apply LDAP updates
transport.py               558 DEBUG      [10/10]: enabling KRA instance
transport.py               558 DEBUG    Done configuring KRA server (pki-tomcatd).
transport.py               558 DEBUG    Restarting the directory server
transport.py               558 DEBUG    The ipa-kra-install command was successful
transport.py               217 DEBUG    Exit code: 0
__init__.py                261 INFO     Adding master.testrealm.test:/var/log/dirsrv/slapd-TESTREALM-TEST/errors to list of logs to collect
__init__.py                261 INFO     Adding master.testrealm.test:/var/log/dirsrv/slapd-TESTREALM-TEST/access to list of logs to collect
__init__.py                261 INFO     Adding master.testrealm.test:/var/log/ipaserver-install.log to list of logs to collect
__init__.py                261 INFO     Adding master.testrealm.test:/var/log/ipaserver-uninstall.log to list of logs to collect
__init__.py                261 INFO     Adding master.testrealm.test:/var/log/ipaclient-install.log to list of logs to collect
__init__.py                261 INFO     Adding master.testrealm.test:/var/log/ipaclient-uninstall.log to list of logs to collect
__init__.py                261 INFO     Adding master.testrealm.test:/var/log/ipareplica-install.log to list of logs to collect
__init__.py                261 INFO     Adding master.testrealm.test:/var/log/ipareplica-conncheck.log to list of logs to collect
__init__.py                261 INFO     Adding master.testrealm.test:/var/log/ipareplica-ca-install.log to list of logs to collect
__init__.py                261 INFO     Adding master.testrealm.test:/var/log/ipaserver-kra-install.log to list of logs to collect
__init__.py                261 INFO     Adding master.testrealm.test:/var/log/ipa-custodia.audit.log to list of logs to collect
__init__.py                261 INFO     Adding master.testrealm.test:/var/log/ipaclient-uninstall.log to list of logs to collect
__init__.py                261 INFO     Adding master.testrealm.test:/var/log/iparestore.log to list of logs to collect
__init__.py                261 INFO     Adding master.testrealm.test:/var/log/ipabackup.log to list of logs to collect
__init__.py                261 INFO     Adding master.testrealm.test:/var/log/kadmind.log to list of logs to collect
__init__.py                261 INFO     Adding master.testrealm.test:/var/log/krb5kdc.log to list of logs to collect
__init__.py                261 INFO     Adding master.testrealm.test:/var/log/httpd/error_log to list of logs to collect
__init__.py                261 INFO     Adding master.testrealm.test:/var/log/pki/ to list of logs to collect
__init__.py                261 INFO     Adding master.testrealm.test:/var/log/audit/audit.log to list of logs to collect
transport.py               391 INFO     RUN ['ipa-healthcheck', '--output-type', 'json', '--failures-only']
transport.py               513 DEBUG    RUN ['ipa-healthcheck', '--output-type', 'json', '--failures-only']
transport.py               558 DEBUG    Loading instance: pki-tomcat
transport.py               558 DEBUG    Loading global Tomcat config: /etc/tomcat/tomcat.conf
transport.py               558 DEBUG    Loading PKI Tomcat config: /usr/share/pki/etc/tomcat.conf
transport.py               558 DEBUG    Loading instance Tomcat config: /etc/pki/pki-tomcat/tomcat.conf
transport.py               558 DEBUG    Loading password config: /etc/pki/pki-tomcat/password.conf
transport.py               558 DEBUG    Loading instance registry: /etc/sysconfig/pki/tomcat/pki-tomcat/pki-tomcat
transport.py               558 DEBUG    Loading subsystem: ca
transport.py               558 DEBUG    Loading subsystem config: /var/lib/pki/pki-tomcat/ca/conf/CS.cfg
transport.py               558 DEBUG    Loading subsystem: kra
transport.py               558 DEBUG    Loading subsystem config: /var/lib/pki/pki-tomcat/kra/conf/CS.cfg
transport.py               558 DEBUG    Getting sslserver cert info for ca from CS.cfg
transport.py               558 DEBUG    Getting subsystem cert info for ca from CS.cfg
transport.py               558 DEBUG    Getting audit_signing cert info for ca from CS.cfg
transport.py               558 DEBUG    Getting ocsp_signing cert info for ca from CS.cfg
transport.py               558 DEBUG    Getting signing cert info for ca from CS.cfg
transport.py               558 DEBUG    Getting transport cert info for kra from CS.cfg
transport.py               558 DEBUG    Getting storage cert info for kra from CS.cfg
transport.py               558 DEBUG    Getting audit_signing cert info for kra from CS.cfg
transport.py               558 DEBUG    []
transport.py               217 DEBUG    Exit code: 0

Comment 13 errata-xmlrpc 2020-04-28 15:44:12 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2020:1640