Bug 1895197

Summary: improve IPA PKI susbsystem detection by other means than a directory presence, use pki-server subsystem-find [rhel-7.9.z]
Product: Red Hat Enterprise Linux 7 Reporter: Marc Sauton <msauton>
Component: ipaAssignee: Florence Blanc-Renaud <frenaud>
Status: CLOSED ERRATA QA Contact: ipa-qe <ipa-qe>
Severity: high Docs Contact:
Priority: high    
Version: 7.9CC: edewata, jreznik, ksiddiqu, pcech, rcritten, sigbjorn.lie, tscherf
Target Milestone: rcKeywords: TestCaseProvided, Triaged, ZStream
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: ipa-4.6.8-5.el7_9.4 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-03-16 13:56:37 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Marc Sauton 2020-11-06 01:23:58 UTC
Description of problem:

improve the way IPA detects PKI services, do not rely on files or directories, try to rely on the Dogtag tools or interfaces, example:

use
pki-server subsystem-find
from
base/server/python/pki/server/cli/subsystem.py
class SubsystemFindCLI(pki.cli.CLI):

pki-server subsystem-find
-----------------
2 entries matched
-----------------
  Subsystem ID: ca
  Instance ID: pki-tomcat
  Enabled: True

  Subsystem ID: kra
  Instance ID: pki-tomcat
  Enabled: True


instead of

/usr/lib/python2.7/site-packages/ipaserver/install/dogtaginstance.py

    def is_installed(self):
        """
        Determine if subsystem instance has been installed.
        Returns True/False
        """
        return os.path.exists(os.path.join(
            paths.VAR_LIB_PKI_TOMCAT_DIR, self.subsystem.lower()))


this directory path testing return an incorrect true status because for example, there is no "ipa-kra-install --uninstall" ( removed in 2018 / bz 1454444 ):
ipa-kra-install --uninstall 
ERROR: Standalone KRA uninstallation was removed in IPA 4.5 as it had never worked properly and only caused issues.


real world scenario example:

there is a RHEL-7 IPA replica deployed
that system is updated from RHEL-7.8 to RHEL-7.9
the update fails in IPA, leading to failing "ipactl restart", then to some serious other problems:


in this case, there was no PKI KRA subsystem configured listed from
ldapsearch -o ldif-wrap=no -LLLxD cn=directory\ manager -W -b "ou=Security Domain,o=ipaca"
nor from a
ipa config-show --all

and no ipara listed from
ldapsearch -o ldif-wrap=no -LLLxD cn=directory\ manager -W -b o=ipaca '(|(uid=ipara)(uid=ipakra))'

but we till had a directory
/var/lib/pki/pki-tomcat/kra/

so the test in
/usr/lib/python2.7/site-packages/ipaserver/install/dogtaginstance.py

incorrectly returned a true status in is_installed(self)

that all lead to a LDAP error on no ipakra user entry, then to a failed IPA update, then to a failed ipactl start, and a bad situation.


Version-Release number of selected component (if applicable):
RHEL-7
RHEL-8

How reproducible:


Steps to Reproduce:
1. IPA with CA, NO KRA
2. add a /var/lib/pki/pki-tomcat/kra/ directory
3. try to run the ipa console for
api.Updater[fix_kra_people_entry]()


Actual results:

2020-11-03T17:27:14Z DEBUG Executing upgrade plugin: fix_kra_people_entry
...
2020-11-03T17:27:14Z ERROR Upgrade failed with no such entry
2020-11-03T17:27:14Z DEBUG Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/ipaserver/install/upgradeinstance.py", line 274, in __upgrade
    self.modified = (ld.update(self.files) or self.modified)
  File "/usr/lib/python2.7/site-packages/ipaserver/install/ldapupdate.py", line 966, in update
    self._run_updates(all_updates)
...
  File "/usr/lib/python2.7/site-packages/ipaserver/install/plugins/fix_kra_people_entry.py", line 35, in execute
    entry = self.api.Backend.ldap2.get_entry(krainstance.KRA_AGENT_DN)

then I realize there is no KRA agent LDAP entry uid=ipakra,ou=people,o=kra,o=ipaca
because that RHEL IPA system has no KRA installed (but has a CA)

so why the failure in fix_kra_people_entry.py if there is no KRA ??

the IPA update uses "plug-ins" to update various elements, listed from
/usr/share/ipa/updates/90-post_upgrade_plugins.update

and one of them is
/usr/lib/python2.7/site-packages/ipaserver/install/plugins/fix_kra_people_entry.py

which calls
/usr/lib/python2.7/site-packages/ipaserver/install/dogtaginstance.py

which calls a function is_installed()

the problem is is_installed() simply check for a directory path to decide a PKI subsystem is installed and configured, which is not a sufficient way to detect the availability of a service.


Expected results:
yes


Additional info:

Comment 2 Florence Blanc-Renaud 2020-11-06 09:56:34 UTC
Notes for triage: looks like an easy fix. The "pki-server subsystem-show kra" command could be used.

If the KRA is configured, the output looks like:
# pki-server subsystem-show  kra
  Subsystem ID: kra
  Instance ID: pki-tomcat
  Enabled: True

If the KRA hasn't been configured, the output looks like:
# pki-server subsystem-show kra
ERROR: ERROR: No kra subsystem in instance pki-tomcat.
(return code = 1)

If the server doesn't have PKI configured at all (no CA, no KRA), the output is:
# pki-server subsystem-show kra
ERROR: Invalid instance pki-tomcat.
(return code 1)


Notes:
- pki-server does not require the services to be up. The command can be used even during an upgrade when some of the services are stopped.
- The command /usr/sbin/pki-server is delivered in the package pki-server. freeipa-server indirectly requires pki-server pkg through pki-ca and pki-kra packages, meaning that the command is always installed on a freeipa server.

Comment 4 Florence Blanc-Renaud 2020-11-25 08:45:58 UTC
Upstream ticket:
https://pagure.io/freeipa/issue/8596

Comment 6 Florence Blanc-Renaud 2020-12-09 08:57:35 UTC
Test case provided upstream in test_integration/test_upgrade.py::TestUpgrade::test_kra_detection

Comment 7 Florence Blanc-Renaud 2020-12-10 18:07:13 UTC
ipa-4-8:

    af830c0 Improve PKI subsystem detection
    7d47e37 ipatests: add test for PKI subsystem detection

ipa-4-9:

    cf30cc3 Improve PKI subsystem detection
    24f6a36 ipatests: add test for PKI subsystem detection

Comment 10 Florence Blanc-Renaud 2021-01-14 13:57:34 UTC
Test fixed:
Fixed upstream
master:
https://pagure.io/freeipa/c/6e0634bd72f60295b86a1561adf94f14d6948009

Comment 12 Florence Blanc-Renaud 2021-01-15 12:56:35 UTC
Test Fixed upstream
ipa-4-6:
https://pagure.io/freeipa/c/46a4e93da1034d596fa58ab4d1c4d0ac637ecebc

Comment 15 Kaleem 2021-02-10 08:21:28 UTC
Verified based on following details.

IPA Version:
=============

2021-02-10T07:12:25+0000 TASK [List installed IPA packages version] *************************************
2021-02-10T07:12:26+0000 ok: [master.testrelm.test] => (item=ipa-server) => 
2021-02-10T07:12:26+0000   msg:
2021-02-10T07:12:26+0000   - arch: x86_64
2021-02-10T07:12:26+0000     epoch: null
2021-02-10T07:12:26+0000     name: ipa-server
2021-02-10T07:12:26+0000     release: 5.el7_9.4
2021-02-10T07:12:26+0000     source: rpm
2021-02-10T07:12:26+0000     version: 4.6.8

snip from test-result log:
==========================
test "test_kra_detection" is executed successfully.

============================= test session starts ==============================
platform linux2 -- Python 2.7.5, pytest-3.10.1, py-1.10.0, pluggy-0.13.1 -- /usr/bin/python2
cachedir: .pytest_cache
metadata: {'Python': '2.7.5', 'Platform': 'Linux-3.10.0-1160.17.1.el7.x86_64-x86_64-with-redhat-7.9-Maipo', 'Packages': {'py': '1.10.0', 'pytest': '3.10.1', 'pluggy': '0.13.1'}, 'Plugins': {u'html': u'1.22.1', u'multihost': u'1.1', u'sourceorder': u'0.5', u'metadata': u'1.11.0'}}
rootdir: /usr/lib/python2.7/site-packages/ipatests, inifile:
plugins: metadata-1.11.0, html-1.22.1, multihost-1.1, sourceorder-0.5
collecting ... collected 3 items

test_integration/test_upgrade.py::TestUpgrade::test_invoke_upgrader PASSED [ 33%]
test_integration/test_upgrade.py::TestUpgrade::test_double_encoded_cacert PASSED [ 66%]
test_integration/test_upgrade.py::TestUpgrade::test_kra_detection PASSED [100%]

---------------- generated xml file: /home/cloud-user/junit.xml ----------------
----------- generated html file: file:///home/cloud-user/report.html -----------
========================== 3 passed in 618.53 seconds ==========================

Comment 19 errata-xmlrpc 2021-03-16 13:56:37 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: ipa security and bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:0860