1109339 – Nested tombstones become orphaned after purge

RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1109339 - Nested tombstones become orphaned after purge

Summary: Nested tombstones become orphaned after purge

Keywords:
Status:	CLOSED ERRATA
Alias:	None
Product:	Red Hat Enterprise Linux 7
Classification:	Red Hat
Component:	389-ds-base
Sub Component:
Version:	7.1
Hardware:	Unspecified
OS:	Unspecified
Priority:	medium
Severity:	unspecified
Target Milestone:	rc
Target Release:	---
Assignee:	Noriko Hosoi
QA Contact:	Viktor Ashirov
Docs Contact:
URL:
Whiteboard:
Depends On:	1109337
Blocks:
TreeView+	depends on / blocked

Reported:	2014-06-13 17:20 UTC by Noriko Hosoi
Modified:	2020-09-13 21:09 UTC (History)
CC List:	4 users (show)
Fixed In Version:	389-ds-base-1.3.3.1-1.el7
Doc Type:	Bug Fix
Doc Text:
Clone Of:	1109337
Environment:
Last Closed:	2015-03-05 09:35:21 UTC
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)

Links
System	ID	Priority	Status	Summary	Last Updated
Github	389ds 389-ds-base issues 1099	None	None	None	2020-09-13 21:02:17 UTC
Github	389ds 389-ds-base issues 1189	None	None	None	2020-09-13 21:09:26 UTC
Red Hat Product Errata	RHSA-2015:0416	normal	SHIPPED_LIVE	Important: 389-ds-base security, bug fix, and enhancement update	2015-03-05 14:26:33 UTC

Description Noriko Hosoi 2014-06-13 17:20:52 UTC

+++ This bug was initially created as a clone of Bug #1109337 +++

This bug is created as a clone of upstream ticket:
https://fedorahosted.org/389/ticket/47767

When tombstone purging occurs it will remove a parent tombstone, which leaves any child tombstone entries orphaned.

Comment 2 Viktor Ashirov 2015-01-26 18:05:52 UTC

Using verification steps from https://bugzilla.redhat.com/show_bug.cgi?id=1109337#c0

[0] Set up MMR
[1] Add a tree to the master, e.g.,
  ou=A,<suffix>
  ou=subA,ou=A,<suffix>
  ou=subsubA,ou=subA,ou=A,<suffix>
  uid=user1,ou=subsubA,ou=subA,ou=A,<suffix>

$ ldapmodify -D "cn=Directory Manager" -w Secret123  -H ldap://localhost:1189 -a << EOF
dn: ou=A,dc=example,dc=com
ou: People
ou: A
objectClass: top
objectClass: organizationalunit

dn: ou=subA,ou=A,dc=example,dc=com
ou: People
ou: subA
objectClass: top
objectClass: organizationalunit

dn: ou=subsubA,ou=subA,ou=A,dc=example,dc=com
ou: People
ou: subsubA
objectClass: top
objectClass: organizationalunit

dn: uid=user,ou=subsubA,ou=subA,ou=A,dc=example,dc=com
uid: user
givenName: user
objectClass: top
objectClass: person
objectClass: organizationalPerson
objectClass: inetorgperson
sn: user
cn: user
EOF
adding new entry "ou=A,dc=example,dc=com"

adding new entry "ou=subA,ou=A,dc=example,dc=com"

adding new entry "ou=subsubA,ou=subA,ou=A,dc=example,dc=com"

adding new entry "uid=user,ou=subsubA,ou=subA,ou=A,dc=example,dc=com"

[2] Set small number (e.g., 30 sec) to these config params on all the servers.
nsds5ReplicaPurgeDelay
nsds5ReplicaTombstonePurgeInterval

$ ldapmodify -D "cn=Directory Manager" -w Secret123  -H ldap://localhost:1189 -a << EOF
dn: cn=replica,cn=dc\3Dexample\2Cdc\3Dcom,cn=mapping tree,cn=config
changetype: modify
replace: nsds5ReplicaPurgeDelay
nsds5ReplicaPurgeDelay: 10
-
replace: nsds5ReplicaTombstonePurgeInterval
nsds5ReplicaTombstonePurgeInterval: 10
-
EOF
modifying entry "cn=replica,cn=dc\3Dexample\2Cdc\3Dcom,cn=mapping tree,cn=config"

$ ldapmodify -D "cn=Directory Manager" -w Secret123  -H ldap://localhost:1289 -a << EOF
dn: cn=replica,cn=dc\3Dexample\2Cdc\3Dcom,cn=mapping tree,cn=config
changetype: modify
replace: nsds5ReplicaPurgeDelay
nsds5ReplicaPurgeDelay: 10
-
replace: nsds5ReplicaTombstonePurgeInterval
nsds5ReplicaTombstonePurgeInterval: 10
-
EOF
modifying entry "cn=replica,cn=dc\3Dexample\2Cdc\3Dcom,cn=mapping tree,cn=config"

[3] Restart the servers.
$ sudo systemctl restart dirsrv.target

[4] Delete the tree.
$ ldapdelete -D "cn=Directory Manager"  -w Secret123  -H ldap://localhost:1189 -r "ou=A,dc=example,dc=com"

[5] Check all the entries in the tree are turned to be a tombstone entry.
$ ldapsearch -LLL -o ldif-wrap=no -D 'cn=Directory Manager' -w Secret123 -H ldap://localhost:1189 -b dc=example,dc=com '(objectclass=nsTombstone)'
dn: cn=replica,cn=dc\3Dexample\2Cdc\3Dcom,cn=mapping tree,cn=config
objectClass: nsDS5Replica
objectClass: top
nsDS5ReplicaRoot: dc=example,dc=com
nsDS5ReplicaType: 3
nsDS5Flags: 1
nsds5ReplicaPurgeDelay: 10
nsDS5ReplicaId: 1231
nsDS5ReplicaBindDN: cn=SyncManager,cn=config
cn: replica
nsState:: zwQAAAAAAAAjd8ZUAAAAAAAAAAAAAAAAAAAAAAAAAAACAAAAAAAAAA==
nsDS5ReplicaName: 5f85ef39-a57f11e4-ba92a3c2-663fcbbb
nsds5ReplicaTombstonePurgeInterval: 10
nsds50ruv: {replicageneration} 54c676f5000004cf0000
nsds50ruv: {replica 1231 ldap://rhel7.brq.redhat.com:1189} 54c67740000004cf0000 54c67740000804cf0000
nsds50ruv: {replica 1232 ldap://rhel7.brq.redhat.com:1289}
nsds5agmtmaxcsn: dc=example,dc=com;1189_to_1626_on_rhel7.brq.redhat.com;rhel7.brq.redhat.com;1626;1232;54c67740000804cf0000
nsruvReplicaLastModified: {replica 1231 ldap://rhel7.brq.redhat.com:1189} 54c67740
nsruvReplicaLastModified: {replica 1232 ldap://rhel7.brq.redhat.com:1289} 00000000
nsds5ReplicaChangeCount: 8
nsds5replicareapactive: 0

dn: nsuniqueid=7d535426-a57f11e4-ba92a3c2-663fcbbb,ou=A,dc=example,dc=com
ou: People
ou: A
objectClass: top
objectClass: organizationalunit
objectClass: nsTombstone
nsParentUniqueId: b2f73d00-a57d11e4-b70d837b-f5154b0c
nstombstonecsn: 54c67740000804cf0000

dn: nsuniqueid=7d535427-a57f11e4-ba92a3c2-663fcbbb,ou=subA,ou=A,dc=example,dc=com
ou: People
ou: subA
objectClass: top
objectClass: organizationalunit
objectClass: nsTombstone
nsParentUniqueId: 7d535426-a57f11e4-ba92a3c2-663fcbbb
nstombstonecsn: 54c67740000704cf0000

dn: nsuniqueid=7d535428-a57f11e4-ba92a3c2-663fcbbb,ou=subsubA,ou=subA,ou=A,dc=example,dc=com
ou: People
ou: subsubA
objectClass: top
objectClass: organizationalunit
objectClass: nsTombstone
nsParentUniqueId: 7d535427-a57f11e4-ba92a3c2-663fcbbb
nstombstonecsn: 54c67740000604cf0000

dn: nsuniqueid=7d535429-a57f11e4-ba92a3c2-663fcbbb,uid=user,ou=subsubA,ou=subA,ou=A,dc=example,dc=com
uid: user
givenName: user
objectClass: top
objectClass: person
objectClass: organizationalPerson
objectClass: inetorgperson
objectClass: nsTombstone
sn: user
cn: user
nsParentUniqueId: 7d535428-a57f11e4-ba92a3c2-663fcbbb
nstombstonecsn: 54c67740000504cf0000

[6] Wait for the second set to the purge interval.
[7] Do some modify operation such as add another dummy entry.
$ ldapmodify -D "cn=Directory Manager" -w Secret123  -H ldap://localhost:1189 -a << EOF
dn: ou=People,dc=example,dc=com
changetype: modify
replace: description
description: dummy change
EOF
modifying entry "ou=People,dc=example,dc=com"

[8] Check the tombstone entries which are supposed to have disappeared.
If you see no tombstone entries (especially any orphaned tombstone entry), the fix was verified.

Tombstone entries are still in place after purge delay, unless I perform *two* modify operations in [7]. Off-by-one error somewhere?

After second modify all tombstone entries were purged after configured delay:

$ ldapsearch -LLL -o ldif-wrap=no -D 'cn=Directory Manager' -w Secret123 -H ldap://localhost:1189 -b dc=example,dc=com '(objectclass=nsTombstone)'
dn: cn=replica,cn=dc\3Dexample\2Cdc\3Dcom,cn=mapping tree,cn=config
objectClass: nsDS5Replica
objectClass: top
nsDS5ReplicaRoot: dc=example,dc=com
nsDS5ReplicaType: 3
nsDS5Flags: 1
nsds5ReplicaPurgeDelay: 10
nsDS5ReplicaId: 1231
nsDS5ReplicaBindDN: cn=SyncManager,cn=config
cn: replica
nsState:: zwQAAAAAAABDd8ZUAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA==
nsDS5ReplicaName: 5f85ef39-a57f11e4-ba92a3c2-663fcbbb
nsds5ReplicaTombstonePurgeInterval: 10
nsds50ruv: {replicageneration} 54c676f5000004cf0000
nsds50ruv: {replica 1231 ldap://rhel7.brq.redhat.com:1189} 54c67740000004cf0000 54c67755000004cf0000
nsds50ruv: {replica 1232 ldap://rhel7.brq.redhat.com:1289}
nsds5agmtmaxcsn: dc=example,dc=com;1189_to_1626_on_rhel7.brq.redhat.com;rhel7.brq.redhat.com;1626;1232;54c67755000004cf0000
nsruvReplicaLastModified: {replica 1231 ldap://rhel7.brq.redhat.com:1189} 54c67755
nsruvReplicaLastModified: {replica 1232 ldap://rhel7.brq.redhat.com:1289} 00000000
nsds5ReplicaChangeCount: 10
nsds5replicareapactive: 0

Comment 3 Viktor Ashirov 2015-01-26 18:06:32 UTC

Build tested: 
$ rpm -qa | grep 389
389-ds-base-1.3.3.1-12.el7.x86_64
389-ds-base-libs-1.3.3.1-12.el7.x86_64
389-ds-base-debuginfo-1.3.3.1-12.el7.x86_64

Comment 4 Viktor Ashirov 2015-01-27 11:07:57 UTC

Hello Mark, 

tombstone entries are still in place after purge delay, unless I perform *two* modify operations in [7]. Is this OK?

Comment 5 mreynolds 2015-01-27 14:12:36 UTC

(In reply to Viktor Ashirov from comment #4)
> Hello Mark, 
> 
> tombstone entries are still in place after purge delay, unless I perform
> *two* modify operations in [7]. Is this OK?

This is fine, it was probably a timing issue.  The original bug would keep those tombstones around indefinitely - they would never be removed.

Comment 6 Viktor Ashirov 2015-01-27 14:18:39 UTC

Thank you, Mark!

Marking this bug as VERIFIED then.

Comment 8 errata-xmlrpc 2015-03-05 09:35:21 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2015-0416.html

Note You need to log in before you can comment on or make changes to this bug.