Bug 784378

Summary: Run CLEANRUV task when completely deleting a replica
Product: Red Hat Enterprise Linux 6 Reporter: Dmitri Pal <dpal>
Component: ipaAssignee: Rob Crittenden <rcritten>
Status: CLOSED ERRATA QA Contact: Namita Soman <nsoman>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 6.3CC: clasohm, jgalipea, mkosek, spoore
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: ipa-3.0.0-1.el6 Doc Type: Bug Fix
Doc Text:
Cause: When a master is removed from a replicated environment via "ipa-replica-manage del" command the meta-data for that master is still contained in the other servers. Consequence: Directory Server replication plugin produces warnings about the old meta-data as this may cause replication issues under certain circumstances. Fix: When replica is removed via "ipa-replica-manage del" command, Directory Server CLEANALLRUV task is triggered to take care of the old meta-data in whole replicated Directory Server environment. ipa-replica-manage also provides additional commands for listing and cleaning replication meta-data to manually handle already present old meta-data. Result: Deleting an Identity Management replica should no longer cause old meta-data present in Identity Management Directory Servers.
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-02-21 09:09:30 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Dmitri Pal 2012-01-24 17:50:11 UTC
This bug is created as a clone of upstream ticket:
https://fedorahosted.org/freeipa/ticket/2303

When a replica is disconnected or removed we need to run the CLEANRUV task in DS to avoid having hanging pointers to the old replica.

Comment 2 Dmitri Pal 2012-01-24 17:54:01 UTC
*** Bug 784371 has been marked as a duplicate of this bug. ***

Comment 3 Dmitri Pal 2012-01-24 17:54:41 UTC
*** Bug 784373 has been marked as a duplicate of this bug. ***

Comment 4 Dmitri Pal 2012-01-24 17:54:59 UTC
*** Bug 784374 has been marked as a duplicate of this bug. ***

Comment 5 Dmitri Pal 2012-01-24 17:55:38 UTC
*** Bug 784376 has been marked as a duplicate of this bug. ***

Comment 6 Dmitri Pal 2012-01-24 17:56:45 UTC
Upstream ticket:
https://fedorahosted.org/freeipa/ticket/2303

Comment 7 Jenny Severance 2012-02-27 14:57:24 UTC
Please add steps to verify/reproduce this issue?

What does CLEANRUV do?  What should it be run?  How does an admin do this and when?

Comment 8 Rob Crittenden 2012-02-27 18:06:53 UTC
When a master is removed from a replicated environment the meta-data for that master is still contained in the other servers. There is a special task you can use to remove this meta-data - the CLEANRUV task. 

There is lots more information at http://directory.fedoraproject.org/wiki/Howto:CLEANRUV

We are going to attempt to make this invisible for IPA users, running this automatically when a master is removed, so no explicit action from an admin will be necessary.

Comment 10 Martin Kosek 2012-09-17 15:56:14 UTC
Fixed upstream:
master: https://fedorahosted.org/freeipa/changeset/c9c55a2845fd8471bc609a23f5a32d252f7df04c
ipa-3-0: https://fedorahosted.org/freeipa/changeset/40582a1f1e40607e7a1d1950dd07f638b156251e

Adds 3 new commands for ipa-replica-manage:
* list-ruv: list all replica update vectors - useful to find out replica ID of RUV belonging to a replica that is no longer connected and should be cleaned
* clean-ruv: cleans a RUV with given ID. This step should not be usually needed as RUV cleanup procedure is automatically run during "ipa-replica-manage del". The command executes a CLEANALLRUV task which should safely clean the RUV of replica being removed by waiting until all configured replicas are online.

2 "maintenance" commands for the CLEANALLRUV task:
* list-clean-ruv: lists all running CLEANALLRUV tasks
* abort-clean-ruv: abort a running CLEANALLRUV task

Comment 12 Scott Poore 2012-11-30 19:45:35 UTC
Verified.

Note, more detailed verification for new ipa-replica-manage commands can be found here:

https://bugzilla.redhat.com/show_bug.cgi?id=837365

Version ::

ipa-server-3.0.0-8.el6.x86_64

Manual Test Results ::

We're verifying that CLEANRUV is run when ipa-replica-manage del is executed.

[root@rhel6-1 log]# ipa-replica-manage list
rhel6-1.testrelm.com: master
rhel6-3.testrelm.com: master
rhel6-4.testrelm.com: master

[root@rhel6-1 log]# ipa-replica-manage list-ruv
rhel6-1.testrelm.com:389: 4
rhel6-3.testrelm.com:389: 5
rhel6-4.testrelm.com:389: 6

[root@rhel6-4 ~]# date
Fri Nov 30 14:40:34 EST 2012
[root@rhel6-4 ~]# ipa-replica-manage del rhel6-3.testrelm.com
Deleting a master is irreversible.
To reconnect to the remote master you will need to prepare a new replica file
and re-install.
Continue to delete? [no]: yes
Deleting replication agreements between rhel6-3.testrelm.com and rhel6-4.testrelm.com
ipa: INFO: Setting agreement cn=meTorhel6-4.testrelm.com,cn=replica,cn=dc\=testrelm\,dc\=com,cn=mapping tree,cn=config schedule to 2358-2359 0 to force synch
ipa: INFO: Deleting schedule 2358-2359 0 from agreement cn=meTorhel6-4.testrelm.com,cn=replica,cn=dc\=testrelm\,dc\=com,cn=mapping tree,cn=config
ipa: INFO: Replication Update in progress: FALSE: status: 0 Replica acquired successfully: Incremental update succeeded: start: 0: end: 0
Deleted replication agreement from 'rhel6-4.testrelm.com' to 'rhel6-3.testrelm.com'
Background task created to clean replication data. This may take a while.
This may be safely interrupted with Ctrl+C

Then we can see the CLEANRUV in the log:

[30/Nov/2012:14:40:49 -0500] NSMMReplicationPlugin - agmt_delete: begin
[30/Nov/2012:14:40:49 -0500] NSMMReplicationPlugin - CleanAllRUV Task: Cleaning rid (5)...
[30/Nov/2012:14:40:49 -0500] NSMMReplicationPlugin - CleanAllRUV Task: Waiting to process all the updates from the deleted replica...
[30/Nov/2012:14:40:49 -0500] NSMMReplicationPlugin - CleanAllRUV Task: Waiting for all the replicas to be online...
[30/Nov/2012:14:40:49 -0500] NSMMReplicationPlugin - CleanAllRUV Task: Waiting for all the replicas to receive all the deleted replica updates...
[30/Nov/2012:14:40:49 -0500] NSMMReplicationPlugin - CleanAllRUV Task: Sending cleanAllRUV task to all the replicas...
[30/Nov/2012:14:40:49 -0500] NSMMReplicationPlugin - CleanAllRUV Task: Cleaning local ruv's...
[30/Nov/2012:14:40:49 -0500] NSMMReplicationPlugin - CleanAllRUV Task: Waiting for all the replicas to be cleaned...
[30/Nov/2012:14:40:49 -0500] NSMMReplicationPlugin - CleanAllRUV Task: Replicas have not been cleaned yet, retrying in 10 seconds
[30/Nov/2012:14:41:01 -0500] NSMMReplicationPlugin - CleanAllRUV Task: Successfully cleaned rid(5).
[30/Nov/2012:14:41:02 -0500] referint-plugin - _update_all_per_mod: entry krbprincipalname=DNS/rhel6-3.testrelm.com,cn=services,cn=accounts,dc=testrelm,dc=com: deleting "managedBy: fqdn=rhel6-3.testrelm.com,cn=computers,cn=accounts,dc=testrelm,dc=com" failed (32)

Comment 14 errata-xmlrpc 2013-02-21 09:09:30 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHSA-2013-0528.html