Bug 863505

Summary: Backport CLEANALLRUV to RHEL6.3
Product: Red Hat Enterprise Linux 6 Reporter: Jr Aquino <jr.aquino>
Component: 389-ds-baseAssignee: Rich Megginson <rmeggins>
Status: CLOSED WONTFIX QA Contact: Sankar Ramalingam <sramling>
Severity: urgent Docs Contact:
Priority: medium    
Version: 6.3CC: jgalipea, msauton, nkinder
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-10-12 17:36:36 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
all 1.2.11 cleanallruv patches compressed into 1 patch file none

Description Jr Aquino 2012-10-05 16:15:05 UTC
Description of problem:
Trying to migrate a massive amount of IPA servers from Fedora to RHEL6.3 and getting hung up with the removal of servers which leave a GHOST RUV.

This is extremely problematic since it is necessary for me to delete the old servers and bring them back up with the same names and this has proven to cause serious issues with replication.

Version-Release number of selected component (if applicable):


How reproducible:
100%

Steps to Reproduce:
1. Install > 10 idM servers in RHEL6.3
2. ipa-replica-manage del a server
3. See that cleanruv task is not sufficient to clean it from all servers.
  
Actual results:
Persistent ghost RUVs causing issues with replication.

Expected results:
CLEANALLRUV to successfully delete all references of deleted replica servers

Additional info:

Comment 3 Marc Sauton 2012-10-05 17:48:44 UTC
associated upstream ticket
https://fedorahosted.org/389/ticket/483
The cliean RUV did not go into 6.3/2.2, but I believe it is set for 6.4

the ghost case may be same as
salesforce 00693337 and 00654496 with bz 826677 / upstream 2797 - IPA cannot remove disconnected replica data to reconnect

If this is the same issue, you currently have to delete the "ghost" entry directly from the database.

assuming all replication agreements are already removed, as well as RUV records:
* delete ghost with ldapdelete command
* remove host record using ipa commands
* prepare replica using ipa-replica-prepare
* install replica
* test replication

ldapsearch -xLLL -D "cn=Directory Manager" -W -b cn=ipaserver1.example.com,cn=masters,cn=ipa,cn=etc,dc=example,dc=com

ldapdelete -r -x -D "cn=Directory Manager" -W cn=ipaserver1.example.com,cn=masters,cn=ipa,cn=etc,dc=domain,dc=com

Comment 4 Jr Aquino 2012-10-05 17:55:16 UTC
Can't actually successfully do that since there are > 20 IPA servers who reinput the ghost back into one replica or another.

I'm told that the only way to do this is to have the cleanruv take place on all replicas simultaneously, which is why cleanallruv was created.

Spent the last 7 hours trying to clean this.

Comment 5 Rich Megginson 2012-10-05 19:36:35 UTC
Created attachment 622397 [details]
all 1.2.11 cleanallruv patches compressed into 1 patch file

These are all of the cleanallruv related patches from the 1.2.11 branch, rebased to 1.2.10, rebased to the internal rhel-6.3 branch, and compressed into 1 big patch file.

Comment 6 Rich Megginson 2012-10-08 15:12:38 UTC
Upstream ticket:
https://fedorahosted.org/389/ticket/483

Comment 7 Nathan Kinder 2012-10-09 14:56:38 UTC
A hotfix adding cleanallruv support has been provided in the support ticket.  Has this hotfix been tried yet?

Comment 8 Rich Megginson 2012-10-12 16:25:43 UTC
To ssh://pkgs.devel.redhat.com/rpms/389-ds-base
 * [new branch]      private-bug863505-rhel-6.3-branch -> private-bug863505-rhel-6.3-branch

Build with patch passes 389-ds-base RHEL 6.3 acceptance tests and RHEL 6.4 cleanallruv tests.

Can we mark this bug as CLOSED?

Comment 10 Nathan Kinder 2012-10-12 17:36:36 UTC
(In reply to comment #8)
> To ssh://pkgs.devel.redhat.com/rpms/389-ds-base
>  * [new branch]      private-bug863505-rhel-6.3-branch ->
> private-bug863505-rhel-6.3-branch
> 
> Build with patch passes 389-ds-base RHEL 6.3 acceptance tests and RHEL 6.4
> cleanallruv tests.
> 
> Can we mark this bug as CLOSED?

Yes, marking as WONTFIX.  We have provided a hotfix, and the official implementation of CLEANALLRUV is slated for RHEL 6.4.

Comment 11 Jr Aquino 2012-10-12 22:07:00 UTC
Just tested the supplied hotfix RPMS in my Stage Environment.  I cleaned 3 different IPA Replicas, and about 5 different Ghosted CA Replicas.

This hotfix works as advertised!

Comment 12 Marc Sauton 2012-10-25 18:38:52 UTC
updated the salesforce case 00717250 with newer rpm set:
0023-Ticket-337-RFE-Improve-CLEANRUV-functionality.patch
0024-CLEANALLRUV-add-force-cleaning.patch
389-ds-base-1.2.10.2-23.el6_3.src.rpm
389-ds-base-1.2.10.2-23.el6_3.x86_64.rpm
389-ds-base-debuginfo-1.2.10.2-23.el6_3.x86_64.rpm
389-ds-base-devel-1.2.10.2-23.el6_3.x86_64.rpm
389-ds-base-libs-1.2.10.2-23.el6_3.x86_64.rpm

Comment 13 Marc Sauton 2012-11-16 18:09:46 UTC
updated the salesforce case 00717250 with newer rpm set:
389-ds-base-1.2.10.2-28.el6_3.src.rpm
389-ds-base-1.2.10.2-28.el6_3.x86_64.rpm
389-ds-base-debuginfo-1.2.10.2-28.el6_3.x86_64.rpm
389-ds-base-devel-1.2.10.2-28.el6_3.x86_64.rpm
389-ds-base-libs-1.2.10.2-28.el6_3.x86_64.rpm