Bug 1577108

Summary: Improve Custodia client and key distribution handling
Product: Red Hat Enterprise Linux 7 Reporter: Petr Vobornik <pvoborni>
Component: ipaAssignee: IPA Maintainers <ipa-maint>
Status: CLOSED ERRATA QA Contact: ipa-qe <ipa-qe>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 7.4CC: cheimes, frenaud, gkaihoro, ksiddiqu, msauton, ndehadra, pasik, pvoborni, rcritten, tscherf
Target Milestone: rcKeywords: ZStream
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: ipa-4.6.4-7.el7 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1579190 (view as bug list) Environment:
Last Closed: 2018-10-30 10:58:39 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1579190    

Description Petr Vobornik 2018-05-11 08:31:31 UTC
Cloned from upstream: https://pagure.io/freeipa/issue/7518

## Problem description
When a replica is installed, a new set of Custodia keys are create locally. The public keys are uploaded to the local 389-DS instance. Then Custodia waits until it sees the keys on its replication peer. For busy systems, replication can talk a while. In worst case, the installer runs into a timeout.

Further more, the installer creates multiple instances of the Custodia client object. In the replica promotion case with CA setup, some parts talk to a Custodia instance on replica master, other instances to Custodia instance on the CA replica master. Replica and CA replica can be different hosts.

## Fix
Installers now pass a single CustodiaInstance object around, instead of creating new instances on demand. In case of replica promotion with CA, the instance gets all secrets from a master with CA present. Before, an installer created multiple instances and may have requested CA key material from a different machine than DM password hash.

In case of Domain Level 1 and replica promotion, the CustodiaInstance no longer adds the keys to the local instance and waits for replication to other replica. Instead the installer directly uploads the new public keys to the remote 389-DS instance.

Without promotion, new Custodia public keys are still added to local 389-DS over LDAPI.

Comment 2 Petr Vobornik 2018-05-11 08:35:21 UTC
Upstream ticket:
https://pagure.io/freeipa/issue/7518

Comment 3 Petr Vobornik 2018-05-11 08:36:00 UTC
master:
    994f71a Use single Custodia instance in installers

ipa-4-6:
    a0cdeb6 Use single Custodia instance in installers

ipa-4-5:
    0e42fb9 Use single Custodia instance in installers

Comment 8 Christian Heimes 2018-06-10 16:38:19 UTC
The previous fix was incomplete and a new fix was pushed: Use one Custodia peer to retrieve all secrets

Fixed upstream
master:
https://pagure.io/freeipa/c/533307382ad8212567337793bd42991885769a58
ipa-4-6:
https://pagure.io/freeipa/c/a3d389095e90706972ec519af53416d83ed4fe82
ipa-4-5:
https://pagure.io/freeipa/c/d3c09a6de06d8ae19d05c0b96cb9c2a5b789b472

Comment 12 Florence Blanc-Renaud 2018-08-28 07:48:26 UTC
Fixed upstream:
master:
https://pagure.io/freeipa/c/530da69eadf5b73e4ca83252e3a370ed70354a39
ipa-4-6:
https://pagure.io/freeipa/c/b9beda34886d99374e5892cf50e32a8ff95c09f4
ipa-4-5:
https://pagure.io/freeipa/c/fd5f000d1379daf263711f1f34ad97ac75dfcde7

moving to POST as additional patches are required and need to be incorporated in a new rhel build.

Comment 14 Nikhil Dehadrai 2018-09-19 17:45:28 UTC
Version:
ipa-server-4.6.4-10.el7.x86_64
389-ds-base-1.3.8.4-15.el7.x86_64

Tested the bug with following observations using the comment#4 as reference:
1. Setup IPA server with CA
2. Setup REPLICA-1 against this IPA in step1, without CA. The setup of REPLICA-1 is successful.
3. Now stop custodia service on REPLICA-1.
4. Setup REPLICA-2 with CA (--setup-ca), and using REPLICA-1 as source. The setup of REPLICA-2 is successful.

Thus the issue mentioned in above comment#5 is no more observed.

Setup: (Line Topology)
-------------------------

MASTER ------> REPLICA-1 ----------> Replica-2


Console-log:
------------------

MASTER:
-------------
[root@auto-hv-02-guest06 ~]# rpm -q ipa-server
ipa-server-4.6.4-10.el7.x86_64
[root@auto-hv-02-guest06 ~]# ipactl status
Directory Service: RUNNING
krb5kdc Service: RUNNING
kadmin Service: RUNNING
named Service: RUNNING
httpd Service: RUNNING
ipa-custodia Service: RUNNING
ntpd Service: RUNNING
pki-tomcatd Service: RUNNING
ipa-otpd Service: RUNNING
ipa-dnskeysyncd Service: RUNNING
ipa: INFO: The ipactl command was successful
[root@auto-hv-02-guest06 ~]# tail -1 /var/log/ipaserver-install.log 
2018-09-19T16:44:48Z INFO The ipa-server-install command was successful
[root@auto-hv-02-guest06 ~]# kinit admin
Password for admin: 
[root@auto-hv-02-guest06 ~]# 

REPLICA-1:
-------------
# ipa-replica-install -U --setup-dns --forwarder=x.x.x.x --ip-address=x.x.x.x -P admin -w Secret123

[root@ipaqavme ~]# systemctl stop ipa-custodia
[root@ipaqavme ~]# ipactl status
Directory Service: RUNNING
krb5kdc Service: RUNNING
kadmin Service: RUNNING
named Service: RUNNING
httpd Service: RUNNING
ipa-custodia Service: STOPPED
ntpd Service: RUNNING
ipa-otpd Service: RUNNING
ipa-dnskeysyncd Service: RUNNING
ipa: INFO: The ipactl command was successful
[root@ipaqavme ~]# tail -1 /var/log/ipareplica-install.log 
2018-09-19T17:26:05Z INFO The ipa-replica-install command was successful
[root@ipaqavme ~]# kinit admin
Password for admin: 
[root@ipaqavme ~]#


REPLICA-2:
---------------------
# ipa-replica-install -U --setup-ca --setup-dns --forwarder=x.x.x.x --ip-address=x.x.x.x -P admin -w Secret123

[root@ipaqavmg ~]# tail -1 /var/log/ipareplica-install.log 
2018-09-19T17:43:54Z INFO The ipa-replica-install command was successful
[root@ipaqavmg ~]# ipactl status
Directory Service: RUNNING
krb5kdc Service: RUNNING
kadmin Service: RUNNING
named Service: RUNNING
httpd Service: RUNNING
ipa-custodia Service: RUNNING
ntpd Service: RUNNING
pki-tomcatd Service: RUNNING
ipa-otpd Service: RUNNING
ipa-dnskeysyncd Service: RUNNING
ipa: INFO: The ipactl command was successful
[root@ipaqavmg ~]# kinit admin
Password for admin: 
[root@ipaqavmg ~]# ipa-replica-manage list
auto-hv-02-guest06.testrelm.test: master
ipaqavmg.testrelm.test: master
ipaqavme.testrelm.test: master
[root@ipaqavmg ~]#

Thus on the basis of above observations, marking the status of bug to 'VERIFIED'.

Comment 16 errata-xmlrpc 2018-10-30 10:58:39 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:3187