Bug 1594141

Summary:	Replication races in DogtagInstance.setup_admin
Product:	Red Hat Enterprise Linux 7	Reporter:	Petr Vobornik <pvoborni>
Component:	ipa	Assignee:	IPA Maintainers <ipa-maint>
Status:	CLOSED ERRATA	QA Contact:	Michal Reznik <mreznik>
Severity:	urgent	Docs Contact:
Priority:	urgent
Version:	7.4	CC:	cheimes, frenaud, mreznik, myusuf, ndehadra, pasik, pvoborni, rcritten, tscherf
Target Milestone:	rc	Keywords:	ZStream
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	ipa-4.6.4-7.el7	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:
Clones:	1623676 (view as bug list)		Environment:
Last Closed:	2018-10-30 10:58:44 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1623676

Description Petr Vobornik 2018-06-22 08:49:04 UTC

Cloned from upstream: https://pagure.io/freeipa/issue/7593

``DogtagInstance.setup_admin`` and related methods have multiple LDAP replication race conditions. The bugs can cause parallel ``ipa-replica-install`` to fail. The issue typically manifests itself as:

``com.netscape.certsrv.base.PKIException: Failed to obtain installation token from security domain: com.netscape.certsrv.base.UnauthorizedException: User admin-replica1.ipa.example is not a member of Enterprise CA Administrators group.``

## _add_admin_to_group

The ``_add_admin_to_group`` method https://pagure.io/freeipa/blob/84ae625fe2c3786f7c5430f23a55c171ff54e110/f/ipaserver/install/dogtaginstance.py#_397-407 uses an LDAP search + ``MOD_REPLACE`` to add the host admin to the admin group. This is subject to race conditions. In case an other thread, process, or replica modifies the group between read and mod, the previous modification is lost. The method must use ``MOD_ADD`` to add the user to the group.

## setup_admin

The ``setup_admin`` waits until the new admin user has been replicated to the replication source. But that is not sufficient. Since the admin is first created and then appended to the groups, the method must wait until the group membership addition have been replicated, too.

https://pagure.io/freeipa/blob/84ae625fe2c3786f7c5430f23a55c171ff54e110/f/ipaserver/install/dogtaginstance.py#_439-444

Comment 2 Petr Vobornik 2018-06-22 08:49:16 UTC

Upstream ticket:
https://pagure.io/freeipa/issue/7593

Comment 3 Petr Vobornik 2018-06-22 08:49:51 UTC

https://github.com/freeipa/freeipa/pull/2051

Comment 4 Christian Heimes 2018-06-22 11:02:44 UTC

Fixed upstream
master:
https://pagure.io/freeipa/c/14c869b347e488e40544ee1e6c4b35341124c76c
https://pagure.io/freeipa/c/1b966f708aa33c07f68fc30daaf6e4800a6b4a53
https://pagure.io/freeipa/c/ad838c37a9ca2d1c5a2e0becf73ddacb004b3ab6

Comment 5 Christian Heimes 2018-06-22 16:14:32 UTC

ipa-4-5:

* 7e9648736185734fc39752df875321558f4464e4 Improve and fix timeout bug in wait_for_entry()
* d129cb2ab70f658e4b07e1fbb7f16905c66895e9 Use common replication wait timeout of 5min
* b763b62069040f3efafc65197063a84411bd4d10 Fix replication races in Dogtag admin code

Comment 15 Michal Reznik 2018-08-31 10:17:15 UTC

Sanity Verified on ipa-server-4.6.4-7.el7.x86_64.
Installed 3x replica simultaneously in 15 runs and all installations were successful.

Comment 18 errata-xmlrpc 2018-10-30 10:58:44 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:3187