Bug 1591824

Summary:	Installation of replica against a specific master
Product:	Red Hat Enterprise Linux 7	Reporter:	Petr Vobornik <pvoborni>
Component:	ipa	Assignee:	IPA Maintainers <ipa-maint>
Status:	CLOSED ERRATA	QA Contact:	ipa-qe <ipa-qe>
Severity:	urgent	Docs Contact:
Priority:	urgent
Version:	7.4	CC:	cheimes, frenaud, myusuf, ndehadra, pasik, pvoborni, rcritten, tscherf
Target Milestone:	rc	Keywords:	ZStream
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	ipa-4.6.4-7.el7	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:
Clones:	1623679 (view as bug list)		Environment:
Last Closed:	2018-10-30 10:58:44 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1623679

Description Petr Vobornik 2018-06-15 16:14:18 UTC

Cloned from upstream: https://pagure.io/freeipa/issue/7566

This can be viewed as both RFE and a bug.

### Request for enhancement
As an administrator, I want to install a replica(promote client) against a specific server so that it doesn't pick a replica which is unreachable(e.g. the replica is on different site) for the to-be-promoted client. It should not require communicating with other servers as they might be unreachable.

### Issue
A replica has several steps when it communicates with another master: getting secrets, creation of service principles, requests of certificates, replication of domain and ca suffixes. It is not guaranteed that all operations will be done against a single master. This can be a cause of race-conditions when something is created on one master, but replica communicates with another master where it was not yet replicated thus failing the installation.

Also in environments with split topology (error state), the replica installation can be a CA server in disconnected CA suffix and thus fail.

#### Actual behavior
In some race conditions and error conditions, replica installation might fail.

#### Expected behavior
Replica installation will be more robust.

So the use cases are:

* installation of a replica in a network with some masters behind a firewall
* more robust installation, e.g. multiple replicas in parallel, in split topology

#### Version/Release/Distribution
$ every IPA release till this date (May 28 2018 - unreleased 4.7), maybe after an introduction of replica promotion

#### Additional information:
A proposal for behavior:

* If the supplied master doesn't fulfill prerequisites - e.g. doesn't have CA/KRA server then installation should fail.
* If no server is provided then replica should pick one automatically (e.g. similarly as now) and then behave as it was picked

Comment 2 Petr Vobornik 2018-06-15 16:14:31 UTC

Upstream ticket:
https://pagure.io/freeipa/issue/7566

Comment 4 Christian Heimes 2018-06-21 13:36:00 UTC

Related fix in master, 4.6 and 4.5:

* 2c471b529c4701b2d8b1e88a8186d0cda641fa90 Always set ca_host when installing replica

Comment 6 Petr Vobornik 2018-06-22 08:52:40 UTC

ipa-4-6:
    14519c2 Always set ca_host when installing replica

Comment 20 Tibor Dudlák 2018-08-28 07:41:38 UTC

New upstream patch has been added and needs to be included in a new RHEL build

Fixed upstream
master:
https://pagure.io/freeipa/c/6175672e8e11a5fb0a813ea11513efffb704a672

Comment 21 Tibor Dudlák 2018-08-28 08:38:00 UTC

Fixed upstream
ipa-4-6:
https://pagure.io/freeipa/c/c4481d71a9a57b89366b02f86f99fc84b5d9d320

Comment 28 Mohammad Rizwan 2018-09-05 11:18:33 UTC

version:
ipa-server-4.6.4-8.el7.x86_64

Steps:
1) Install master with CA
2) Install replica1 without CA
3) Stop ipa-custodia on replica1
   $  systemctl stop ipa-custodia.service
4) Install replica2 from replica1. Since replica1 doesn't have a CA, the installer on replica2 will fetch all secrets from master.

Actual result:

replica2 installed successfully from replica1.

Master:
~~~~~~~~
[root@master ~]# tail -1f /var/log/ipaserver-install.log 
2018-09-05T10:01:27Z INFO The ipa-server-install command was successful

Replica1:
~~~~~~~~~~
[root@replica1 ~]# tail -1f /var/log/ipareplica-install.log 
2018-09-05T10:15:40Z INFO The ipa-replica-install command was successful

[root@replica1 ~]# ipactl status
Directory Service: RUNNING
krb5kdc Service: RUNNING
kadmin Service: RUNNING
named Service: RUNNING
httpd Service: RUNNING
ipa-custodia Service: STOPPED
ntpd Service: RUNNING
ipa-otpd Service: RUNNING
ipa-dnskeysyncd Service: RUNNING
ipa: INFO: The ipactl command was successful

Replica2:
~~~~~~~~~~~

[root@replica2 ~]# /usr/sbin/ipa-replica-install -U --setup-dns --forwarder xx.xx.xx.xx --no-reverse --setup-ca --server replica1.testrelm.test --domain testrelm.test --admin-password Secret123 --principal admin
WARNING: conflicting time&date synchronization service 'chronyd' will
be disabled in favor of ntpd

Configuring client side components
Client hostname: replica2.testrelm.test
Realm: TESTRELM.TEST
DNS Domain: testrelm.test
IPA Server: replica1.testrelm.test
BaseDN: dc=testrelm,dc=test

Skipping synchronizing time with NTP server.
Successfully retrieved CA cert
    Subject:     CN=Certificate Authority,O=TESTRELM.TEST
    Issuer:      CN=Certificate Authority,O=TESTRELM.TEST
    Valid From:  2018-09-05 09:54:59
    Valid Until: 2038-09-05 09:54:59

[..]
Global DNS configuration in LDAP server is empty
You can use 'dnsconfig-mod' command to set global DNS options that
would override settings in local named.conf files

[root@replica2 ~]# tail -1f /var/log/ipareplica-install.log 
2018-09-05T10:47:43Z INFO The ipa-replica-install command was successful


[root@replica2 ~]# ipa server-role-find
-----------------------
18 server roles matched
-----------------------
  Server name: master.testrelm.test
  Role name: CA server
  Role status: enabled

  Server name: replica1.testrelm.test
  Role name: CA server
  Role status: absent

  Server name: replica2.testrelm.test
  Role name: CA server
  Role status: enabled

  Server name: master.testrelm.test
  Role name: DNS server
  Role status: enabled

  Server name: replica1.testrelm.test
  Role name: DNS server
  Role status: enabled

  Server name: replica2.testrelm.test
  Role name: DNS server
  Role status: enabled

  Server name: master.testrelm.test
  Role name: NTP server
  Role status: enabled

  Server name: replica1.testrelm.test
  Role name: NTP server
  Role status: enabled

  Server name: replica2.testrelm.test
  Role name: NTP server
  Role status: enabled

  Server name: master.testrelm.test
  Role name: AD trust agent
  Role status: absent

  Server name: replica1.testrelm.test
  Role name: AD trust agent
  Role status: absent

  Server name: replica2.testrelm.test
  Role name: AD trust agent
  Role status: absent

  Server name: master.testrelm.test
  Role name: KRA server
  Role status: absent

  Server name: replica1.testrelm.test
  Role name: KRA server
  Role status: absent

  Server name: replica2.testrelm.test
  Role name: KRA server
  Role status: absent

  Server name: master.testrelm.test
  Role name: AD trust controller
  Role status: absent

  Server name: replica1.testrelm.test
  Role name: AD trust controller
  Role status: absent

  Server name: replica2.testrelm.test
  Role name: AD trust controller
  Role status: absent
-----------------------------
Number of entries returned 18
-----------------------------


Thus based on above observations, marking the bug as verified.

Comment 29 Florence Blanc-Renaud 2018-09-06 12:25:49 UTC

Additional patches needed because of regression BZ 1623486

Fixed upstream
master:
https://pagure.io/freeipa/c/2a227c240fae802d3625805e0905a8ce71706b2f
https://pagure.io/freeipa/c/bcfd18f336d752483dffc048e1d9c0edac1628fd

ipa-4-5:
https://pagure.io/freeipa/c/2ff9684f14c14bcdf4a520c5e00cfe4030868143
https://pagure.io/freeipa/c/5b8531eb8f91c689cba1313dd2a7387f7bb5b5fa

ipa-4-6:
https://pagure.io/freeipa/c/e02041d9797c2478da27bace65bfc6853afcb638
https://pagure.io/freeipa/c/2a2fd0829e7b768974365b01ea540dc16e705199

ipa-4-7:
https://pagure.io/freeipa/c/09c78a1e07056eea1036d974bcdfd8c00a254733
https://pagure.io/freeipa/c/5ea8f8ae9d250b86d66d20df95293a71dc40eb46

Comment 31 errata-xmlrpc 2018-10-30 10:58:44 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:3187

Comment 32 Florence Blanc-Renaud 2019-10-07 06:11:05 UTC

Test added upstream in  ipatests/test_integration/test_installation.py::TestInstallReplicaAgainstSpecificServer

Fixed upstream
master:
https://pagure.io/freeipa/c/c2c1000e2d5481d4be377feb12588fdb09d12de0
https://pagure.io/freeipa/c/c77bbe7899577cb14b42625953f1b9a868e6f237

Comment 33 Florence Blanc-Renaud 2019-10-14 08:55:16 UTC

Test backported upstream:

Fixed upstream
ipa-4-8:
https://pagure.io/freeipa/c/b6134e86b377a2804efbfac1d78091a460898d0c
https://pagure.io/freeipa/c/b585e58b845ccecd48934f55c664a12b8ed06fc8

ipa-4-7:
https://pagure.io/freeipa/c/e12fa0b88371962e3684c6b932980c3ac0ab8e1d
https://pagure.io/freeipa/c/16c794d8a3d7d690883da5b29c5c04a203a2b8db

Comment 34 Florence Blanc-Renaud 2019-10-14 11:37:24 UTC

Test backported upstream:

Fixed upstream
ipa-4-6:
https://pagure.io/freeipa/c/f4dc0ee169689974020a4a77b8bb58b26f360369
https://pagure.io/freeipa/c/9b3855ec486990ecd08a9f3a0ca408425ee7fbf7

Comment 35 Florence Blanc-Renaud 2020-01-03 08:21:11 UTC

Test backported upstream
ipa-4-6:
https://pagure.io/freeipa/c/0d91a78ee409e66f96e7b2555ca33fb2128fdfa3