Bug 1369570

Summary: cleanallruv changelog cleaning incorrectly impacts all backends
Product: Red Hat Enterprise Linux 7 Reporter: Noriko Hosoi <nhosoi>
Component: 389-ds-baseAssignee: Noriko Hosoi <nhosoi>
Status: CLOSED ERRATA QA Contact: Viktor Ashirov <vashirov>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 7.3CC: lmiksik, mreynolds, nkinder, rmeggins, sramling
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: 389-ds-base-1.3.5.10-9.el7 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-11-03 20:45:15 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Noriko Hosoi 2016-08-23 19:41:00 UTC
This bug is created as a clone of upstream ticket:
https://fedorahosted.org/389/ticket/48964

At the end of the cleanAllRUV task the changelog is purged of entries that contain the invalid rid, but it "cleans" all the backend changelogs.  It should only clean the specific backend changelog specified in the clean task.

Comment 2 Sankar Ramalingam 2016-09-13 17:56:09 UTC
Hi Mark, request you to add steps to verify this bugzilla. Thanks!

Comment 3 mreynolds 2016-09-13 18:47:29 UTC
Steps to reproduce

[1] Setup 3 was MMR on userroot (dc=example,dc=com)
   Replica A - use Replica ID 1
   Replica B - use Replica ID 2
   Replica C - use Replica ID 3

[2] On Replica A create a new backend/suffix: o=test.com
[3] Setup o=test.com as a Replication Master
   - use Replica ID 2

[4] Make any kind of update to o=test.com - so something gets written to its changelog.
[5] Make updates to dc=example,dc=com on Replica A, B, and C
[6] Stop Replica B
[7] Move the replication agreements to Replica B from Replica A & C
[8] Run the cleanallruv task on "dc=example,dc=com" and clean rid 2
[9] Once task is complete, stop replica A
[10] Check the changelog of o=test.com to see if it still has the update from step [4] in it:

Example, fill in the the DN from the update that was used in step [4]:

/usr/bin/cl-dump -D "cn=directory manager" -w Secret123 -r "o=test.com" | grep "DN from step [4]"

[11]  If the update is still present, then the fix has been verified

Comment 4 Sankar Ramalingam 2016-09-14 19:16:01 UTC
Thanks Mark! for the detailed steps.

1). Four master & 2 consumer setup is in place for suffix "ou=passsync,dc=com"

2). On M2
[root@ratangad ~]# ldapsearch -D "cn=directory manager" -w Secret123 -h localhost -p 1289 -x -o ldif-wrap=no -b  "cn=replica,cn=dc\3Dpasssync\2Cdc\3Dcom,cn=mapping tree,cn=config"
dn: cn=replica,cn=dc\3Dpasssync\2Cdc\3Dcom,cn=mapping tree,cn=config
nsDS5ReplicaRoot: dc=passsync,dc=com
nsDS5ReplicaType: 3
nsDS5ReplicaId: 2212

3). Add suffix o=test.com and configure replica master with same RID as M2.
[root@ratangad MMR_WINSYNC]# ldapsearch -LLL -D "cn=directory manager" -w Secret123 -h localhost -p 1189 -x -o ldif-wrap=no -b "cn=replica,cn=\"o=test.com\",cn=mapping tree,cn=config"
dn: cn=replica,cn=o\3Dtest.com,cn=mapping tree,cn=config
nsDS5ReplicaRoot: o=test.com
nsDS5ReplicaType: 3
nsDS5ReplicaId: 2212
nsds5ReplicaChangeCount: 12

4). New entries added to o=test.com. Changelog updated

5). New entries added to dc=passsync,dc=com. Changelog updated.

6). Stop M2. /usr/lib64/dirsrv/slapd-M2/stop-slapd

7). Remove replication agreements from all masters which points to M2
[root@ratangad MMR_WINSYNC]# ldapdelete -D "cn=directory manager" -w Secret123 -h localhost -p 1189 -x "cn=1189_to_1626_on_ratangad.eng.blr.redhat.com,cn=replica,cn=dc\3Dpasssync\2Cdc\3Dcom,cn=mapping tree,cn=config"
[root@ratangad MMR_WINSYNC]# ldapdelete -D "cn=directory manager" -w Secret123 -h localhost -p 2189 -x "cn=2189_to_1626_on_ratangad.eng.blr.redhat.com,cn=replica,cn=dc\3Dpasssync\2Cdc\3Dcom,cn=mapping tree,cn=config"
[root@ratangad MMR_WINSYNC]# ldapdelete -D "cn=directory manager" -w Secret123 -h localhost -p 2289 -x "cn=2289_to_1626_on_ratangad.eng.blr.redhat.com,cn=replica,cn=dc\3Dpasssync\2Cdc\3Dcom,cn=mapping tree,cn=config"

8). [root@ratangad MMR_WINSYNC]# cat /export/clean_m2.ldif
dn: cn=M2clean,cn=cleanallruv,cn=tasks,cn=config
cn: M2clean
objectclass: extensibleObject
replica-base-dn: dc=passsync,dc=com
replica-id: 2212
[root@ratangad MMR_WINSYNC]# ldapmodify -x -p 1189 -h localhost -D "cn=Directory Manager" -w Secret123 -avf /export/clean_m2.ldif 
ldap_initialize( ldap://localhost:1189 )
add cn:
	M2clean
add objectclass:
	extensibleObject
add replica-base-dn:
	dc=passsync,dc=com
add replica-id:
	2212
adding new entry "cn=M2clean,cn=cleanallruv,cn=tasks,cn=config"
modify complete

==> /var/log/dirsrv/slapd-M3/errors <==
[15/Sep/2016:00:32:11.451215825 +051800] NSMMReplicationPlugin - CleanAllRUV Task (rid 2212): Cleaning rid (2212)... 
[15/Sep/2016:00:32:11.481326122 +051800] NSMMReplicationPlugin - CleanAllRUV Task (rid 2212): Waiting to process all the updates from the deleted replica... 
[15/Sep/2016:00:32:11.523065237 +051800] NSMMReplicationPlugin - CleanAllRUV Task (rid 2212): Waiting for all the replicas to be online... 
[15/Sep/2016:00:32:11.590802258 +051800] NSMMReplicationPlugin - CleanAllRUV Task (rid 2212): Waiting for all the replicas to receive all the deleted replica updates... 
[15/Sep/2016:00:32:11.650533230 +051800] NSMMReplicationPlugin - CleanAllRUV Task (rid 2212): Sending cleanAllRUV task to all the replicas... 
[15/Sep/2016:00:32:11.700180731 +051800] NSMMReplicationPlugin - CleanAllRUV Task (rid 2212): Cleaning local ruv's... 
[15/Sep/2016:00:32:12.736788757 +051800] NSMMReplicationPlugin - CleanAllRUV Task (rid 2212): Waiting for all the replicas to be cleaned... 
[15/Sep/2016:00:32:12.779116577 +051800] NSMMReplicationPlugin - CleanAllRUV Task (rid 2212): Waiting for all the replicas to finish cleaning... 
[15/Sep/2016:00:32:12.817946287 +051800] NSMMReplicationPlugin - CleanAllRUV Task (rid 2212): Not all replicas finished cleaning, retrying in 10 seconds 
[15/Sep/2016:00:32:22.862236331 +051800] NSMMReplicationPlugin - CleanAllRUV Task (rid 2212): Successfully cleaned rid(2212). 

9). Stop M1. /usr/lib64/dirsrv/slapd-M1/stop-slapd

10). Start slapd on M1 and check if cl-dump shows DN of entries added to o=test.com on M1

[root@ratangad MMR_WINSYNC]# /usr/lib64/dirsrv/slapd-M1/stop-slapd
[root@ratangad MMR_WINSYNC]# /usr/bin/cl-dump -D "cn=directory manager" -w Secret123 -p 1189 -h localhost -r "o=test.com" | grep -i "uid=earrooo3,ou=People,o=test.com"
dn: uid=earrooo3,ou=People,o=test.com
entrydn: uid=earrooo3,ou=people,o=test.com
[root@ratangad MMR_WINSYNC]# /usr/bin/cl-dump -D "cn=directory manager" -w Secret123 -p 1189 -h localhost -r "o=test.com" | grep -i "uid=earrrrre3,ou=People,o=test.com"
dn: uid=earrrrre3,ou=People,o=test.com
entrydn: uid=earrrrre3,ou=people,o=test.com

Running cleanallruv task is not impacting other backends as tested above. Hence, marking the bug as Verified.

[root@ratangad MMR_WINSYNC]# rpm -qa |grep -i 389-ds-base
389-ds-base-1.3.5.10-10.el7.x86_64
389-ds-base-debuginfo-1.3.5.10-6.el7.x86_64
389-ds-base-libs-1.3.5.10-10.el7.x86_64
389-ds-base-devel-1.3.5.10-10.el7.x86_64

Comment 6 errata-xmlrpc 2016-11-03 20:45:15 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2016-2594.html