Bug 1591824

Summary: Installation of replica against a specific master
Product: Red Hat Enterprise Linux 7 Reporter: Petr Vobornik <pvoborni>
Component: ipaAssignee: IPA Maintainers <ipa-maint>
Status: CLOSED ERRATA QA Contact: ipa-qe <ipa-qe>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 7.4CC: cheimes, frenaud, myusuf, ndehadra, pasik, pvoborni, rcritten, tscherf
Target Milestone: rcKeywords: ZStream
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: ipa-4.6.4-7.el7 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1623679 (view as bug list) Environment:
Last Closed: 2018-10-30 10:58:44 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1623679    

Description Petr Vobornik 2018-06-15 16:14:18 UTC
Cloned from upstream: https://pagure.io/freeipa/issue/7566

This can be viewed as both RFE and a bug.

### Request for enhancement
As an administrator, I want to install a replica(promote client) against a specific server so that it doesn't pick a replica which is unreachable(e.g. the replica is on different site) for the to-be-promoted client. It should not require communicating with other servers as they might be unreachable.


### Issue
A replica has several steps when it communicates with another master: getting secrets, creation of service principles, requests of certificates, replication of domain and ca suffixes. It is not guaranteed that all operations will be done against a single master. This can be a cause of race-conditions when something is created on one master, but replica communicates with another master where it was not yet replicated thus failing the installation. 

Also in environments with split topology (error state), the replica installation can be a CA server in disconnected CA suffix and thus fail.

#### Actual behavior
In some race conditions and error conditions, replica installation might fail.

#### Expected behavior
Replica installation will be more robust.

So the use cases are:

* installation of a replica in a network with some masters behind a firewall
* more robust installation, e.g. multiple replicas in parallel, in split topology

#### Version/Release/Distribution
   $ every IPA release till this date (May 28 2018 - unreleased 4.7), maybe after an introduction of replica promotion

#### Additional information:
A proposal for behavior:

* If the supplied master doesn't fulfill prerequisites - e.g. doesn't have CA/KRA server then installation should fail. 
* If no server is provided then replica should pick one automatically (e.g. similarly as now) and then behave as it was picked

Comment 2 Petr Vobornik 2018-06-15 16:14:31 UTC
Upstream ticket:
https://pagure.io/freeipa/issue/7566

Comment 4 Christian Heimes 2018-06-21 13:36:00 UTC
Related fix in master, 4.6 and 4.5:

* 2c471b529c4701b2d8b1e88a8186d0cda641fa90 Always set ca_host when installing replica

Comment 6 Petr Vobornik 2018-06-22 08:52:40 UTC
ipa-4-6:
    14519c2 Always set ca_host when installing replica

Comment 20 Tibor Dudlák 2018-08-28 07:41:38 UTC
New upstream patch has been added and needs to be included in a new RHEL build

Fixed upstream
master:
https://pagure.io/freeipa/c/6175672e8e11a5fb0a813ea11513efffb704a672

Comment 21 Tibor Dudlák 2018-08-28 08:38:00 UTC
Fixed upstream
ipa-4-6:
https://pagure.io/freeipa/c/c4481d71a9a57b89366b02f86f99fc84b5d9d320

Comment 28 Mohammad Rizwan 2018-09-05 11:18:33 UTC
version:
ipa-server-4.6.4-8.el7.x86_64

Steps:
1) Install master with CA
2) Install replica1 without CA
3) Stop ipa-custodia on replica1
   $  systemctl stop ipa-custodia.service
4) Install replica2 from replica1. Since replica1 doesn't have a CA, the installer on replica2 will fetch all secrets from master.

Actual result:

replica2 installed successfully from replica1.

Master:
~~~~~~~~
[root@master ~]# tail -1f /var/log/ipaserver-install.log 
2018-09-05T10:01:27Z INFO The ipa-server-install command was successful

Replica1:
~~~~~~~~~~
[root@replica1 ~]# tail -1f /var/log/ipareplica-install.log 
2018-09-05T10:15:40Z INFO The ipa-replica-install command was successful

[root@replica1 ~]# ipactl status
Directory Service: RUNNING
krb5kdc Service: RUNNING
kadmin Service: RUNNING
named Service: RUNNING
httpd Service: RUNNING
ipa-custodia Service: STOPPED
ntpd Service: RUNNING
ipa-otpd Service: RUNNING
ipa-dnskeysyncd Service: RUNNING
ipa: INFO: The ipactl command was successful

Replica2:
~~~~~~~~~~~

[root@replica2 ~]# /usr/sbin/ipa-replica-install -U --setup-dns --forwarder xx.xx.xx.xx --no-reverse --setup-ca --server replica1.testrelm.test --domain testrelm.test --admin-password Secret123 --principal admin
WARNING: conflicting time&date synchronization service 'chronyd' will
be disabled in favor of ntpd

Configuring client side components
Client hostname: replica2.testrelm.test
Realm: TESTRELM.TEST
DNS Domain: testrelm.test
IPA Server: replica1.testrelm.test
BaseDN: dc=testrelm,dc=test

Skipping synchronizing time with NTP server.
Successfully retrieved CA cert
    Subject:     CN=Certificate Authority,O=TESTRELM.TEST
    Issuer:      CN=Certificate Authority,O=TESTRELM.TEST
    Valid From:  2018-09-05 09:54:59
    Valid Until: 2038-09-05 09:54:59

[..]
Global DNS configuration in LDAP server is empty
You can use 'dnsconfig-mod' command to set global DNS options that
would override settings in local named.conf files

[root@replica2 ~]# tail -1f /var/log/ipareplica-install.log 
2018-09-05T10:47:43Z INFO The ipa-replica-install command was successful


[root@replica2 ~]# ipa server-role-find
-----------------------
18 server roles matched
-----------------------
  Server name: master.testrelm.test
  Role name: CA server
  Role status: enabled

  Server name: replica1.testrelm.test
  Role name: CA server
  Role status: absent

  Server name: replica2.testrelm.test
  Role name: CA server
  Role status: enabled

  Server name: master.testrelm.test
  Role name: DNS server
  Role status: enabled

  Server name: replica1.testrelm.test
  Role name: DNS server
  Role status: enabled

  Server name: replica2.testrelm.test
  Role name: DNS server
  Role status: enabled

  Server name: master.testrelm.test
  Role name: NTP server
  Role status: enabled

  Server name: replica1.testrelm.test
  Role name: NTP server
  Role status: enabled

  Server name: replica2.testrelm.test
  Role name: NTP server
  Role status: enabled

  Server name: master.testrelm.test
  Role name: AD trust agent
  Role status: absent

  Server name: replica1.testrelm.test
  Role name: AD trust agent
  Role status: absent

  Server name: replica2.testrelm.test
  Role name: AD trust agent
  Role status: absent

  Server name: master.testrelm.test
  Role name: KRA server
  Role status: absent

  Server name: replica1.testrelm.test
  Role name: KRA server
  Role status: absent

  Server name: replica2.testrelm.test
  Role name: KRA server
  Role status: absent

  Server name: master.testrelm.test
  Role name: AD trust controller
  Role status: absent

  Server name: replica1.testrelm.test
  Role name: AD trust controller
  Role status: absent

  Server name: replica2.testrelm.test
  Role name: AD trust controller
  Role status: absent
-----------------------------
Number of entries returned 18
-----------------------------


Thus based on above observations, marking the bug as verified.

Comment 31 errata-xmlrpc 2018-10-30 10:58:44 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:3187

Comment 32 Florence Blanc-Renaud 2019-10-07 06:11:05 UTC
Test added upstream in  ipatests/test_integration/test_installation.py::TestInstallReplicaAgainstSpecificServer

Fixed upstream
master:
https://pagure.io/freeipa/c/c2c1000e2d5481d4be377feb12588fdb09d12de0
https://pagure.io/freeipa/c/c77bbe7899577cb14b42625953f1b9a868e6f237

Comment 34 Florence Blanc-Renaud 2019-10-14 11:37:24 UTC
Test backported upstream:

Fixed upstream
ipa-4-6:
https://pagure.io/freeipa/c/f4dc0ee169689974020a4a77b8bb58b26f360369
https://pagure.io/freeipa/c/9b3855ec486990ecd08a9f3a0ca408425ee7fbf7

Comment 35 Florence Blanc-Renaud 2020-01-03 08:21:11 UTC
Test backported upstream
ipa-4-6:
https://pagure.io/freeipa/c/0d91a78ee409e66f96e7b2555ca33fb2128fdfa3