Bug 688730

Summary: Exported tombstone cannot be imported correctly.
Product: Red Hat Enterprise Linux 6 Reporter: Noriko Hosoi <nhosoi>
Component: 389-ds-baseAssignee: Rich Megginson <rmeggins>
Status: CLOSED ERRATA QA Contact: Chandrasekar Kannan <ckannan>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 6.1CC: amsharma, benl, dpal, jgalipea, nkinder, shaines
Target Milestone: rcKeywords: screened
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: 389-ds-base-1.2.8-0.7.rc2.el6 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 684996 Environment:
Last Closed: 2011-05-19 08:42:42 EDT Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
Bug Depends On: 684996    
Bug Blocks:    

Description Noriko Hosoi 2011-03-17 16:44:07 EDT
+++ This bug was initially created as a clone of Bug #684996 +++

Description of problem:

This is a sample tombstone entry in id2entry.db4
id 11
    rdn: nsuniqueid=042d8081-4e6111e0-ac8585ed-ca8fe9f7,uid=tuser0
    uid;vucsn-4d7e5156000000010000;mdcsn-4d7e5156000000010000: tuser0
    givenName;vucsn-4d7e5156000000010000: test
    objectClass;vucsn-4d7e5156000000010000: top
    objectClass;vucsn-4d7e5156000000010000: person
    objectClass;vucsn-4d7e5156000000010000: organizationalPerson
    objectClass;vucsn-4d7e5156000000010000: inetorgperson
    objectClass;vucsn-4d7e5163000000010000: nsTombstone
    sn;vucsn-4d7e5156000000010000: user0
    cn;vucsn-4d7e5156000000010000: test user0
    userPassword;vucsn-4d7e5156000000010000: {SSHA}QscFOIbdsvg6+qdW8Dy+MKwuvPdlIEu
     w0tChBg==
    creatorsName;vucsn-4d7e5156000000010000: uid=admin,ou=administrators,ou=topolo
     gymanagement,o=netscaperoot
    modifiersName;vucsn-4d7e5156000000010000: uid=admin,ou=administrators,ou=topol
     ogymanagement,o=netscaperoot
    createTimestamp;vucsn-4d7e5156000000010000: 20110314173310Z
    modifyTimestamp;vucsn-4d7e5156000000010000: 20110314173310Z
    nsUniqueId: 042d8081-4e6111e0-ac8585ed-ca8fe9f7
    parentid: 4
    entryid: 11
    nsParentUniqueId: 89ad3283-4e5911e0-98dfa788-5bfd3127
    nscpEntryDN: uid=tuser0,ou=people,dc=example,dc=com

This is the same entry in the exported ldif file with db2ldif -r:
# entry-id: 11
dn: nsuniqueid=042d8081-4e6111e0-ac8585ed-ca8fe9f7,uid=tuser0,ou=People,dc=exa
 mple,dc=com
nsUniqueId: 042d8081-4e6111e0-ac8585ed-ca8fe9f7
uid;vucsn-4d7e5156000000010000;mdcsn-4d7e5156000000010000: tuser0
givenName;vucsn-4d7e5156000000010000: test
objectClass;vucsn-4d7e5156000000010000: top
objectClass;vucsn-4d7e5156000000010000: person
objectClass;vucsn-4d7e5156000000010000: organizationalPerson
objectClass;vucsn-4d7e5156000000010000: inetorgperson
objectClass;vucsn-4d7e5163000000010000: nsTombstone
sn;vucsn-4d7e5156000000010000: user0
cn;vucsn-4d7e5156000000010000: test user0
userPassword;vucsn-4d7e5156000000010000: {SSHA}QscFOIbdsvg6+qdW8Dy+MKwuvPdlIEu
 w0tChBg==
creatorsName;vucsn-4d7e5156000000010000: uid=admin,ou=administrators,ou=topolo
 gymanagement,o=netscaperoot
modifiersName;vucsn-4d7e5156000000010000: uid=admin,ou=administrators,ou=topol
 ogymanagement,o=netscaperoot
createTimestamp;vucsn-4d7e5156000000010000: 20110314173310Z
modifyTimestamp;vucsn-4d7e5156000000010000: 20110314173310Z
nsParentUniqueId: 89ad3283-4e5911e0-98dfa788-5bfd3127
nscpEntryDN: uid=tuser0,ou=people,dc=example,dc=com

Once this tombstone entry is imported, it loses the original rdn info (uid=tuser0) as well as the attribute parentid:
id 11
	rdn: nsuniqueid=042d8081-4e6111e0-ac8585ed-ca8fe9f7
	nsUniqueId: 042d8081-4e6111e0-ac8585ed-ca8fe9f7
	uid;vucsn-4d7e5156000000010000;mdcsn-4d7e5156000000010000: tuser0
	givenName;vucsn-4d7e5156000000010000: test
	objectClass;vucsn-4d7e5156000000010000: top
	objectClass;vucsn-4d7e5156000000010000: person
	objectClass;vucsn-4d7e5156000000010000: organizationalPerson
	objectClass;vucsn-4d7e5156000000010000: inetorgperson
	objectClass;vucsn-4d7e5163000000010000: nsTombstone
	sn;vucsn-4d7e5156000000010000: user0
	cn;vucsn-4d7e5156000000010000: test user0
	userPassword;vucsn-4d7e5156000000010000: {SSHA}QscFOIbdsvg6+qdW8Dy+MKwuvPdlIEu
	 w0tChBg==
	creatorsName;vucsn-4d7e5156000000010000: uid=admin,ou=administrators,ou=topolo
	 gymanagement,o=netscaperoot
	modifiersName;vucsn-4d7e5156000000010000: uid=admin,ou=administrators,ou=topol
	 ogymanagement,o=netscaperoot
	createTimestamp;vucsn-4d7e5156000000010000: 20110314173310Z
	modifyTimestamp;vucsn-4d7e5156000000010000: 20110314173310Z
	nsParentUniqueId: 89ad3283-4e5911e0-98dfa788-5bfd3127
	nscpEntryDN: uid=tuser0,ou=people,dc=example,dc=com
	entryid: 10

This makes the tombstone search fail.
$ ldapsearch ... -D 'cn=directory manager' -w password -b "dc=example,dc=com" "(&(objectclass=nstombstone)(uid=*))" dn
$

--- Additional comment from nhosoi@redhat.com on 2011-03-17 14:38:48 EDT ---

Created attachment 486083 [details]
git patch file (master)

Description: When nsslapd-subtree-rename-switch is on,
a tombstone entry has a special RDN which looks like this:
  nsuniqueid=042d8081-...-ca8fe9f7,<original_leaf_rdn>
This special format was not treated properly.

This patch adds the code to handle the special tombstone
RDN, where an internal entry has the above RDN and points
the correct parent entry.
Comment 2 Amita Sharma 2011-04-29 05:51:32 EDT
Steps to verify
0. set up a replication topology (I used one master - one consumer)
1. add some entries and delete them.
<Amita>
[amsharma@rhel61-ds90-amita scripts]$ ldapmodify -x -h localhost -p 20100 -D "cn=Directory Manager" -w Secret123 << EOF
> dn: uid=amita,ou=people,dc=replsuffix,dc=com
> changetype: add
> objectClass: top
> objectClass: person
> objectClass: inetorgperson
> sn: testkrbuser
> cn: kkk testkrbuser
> userPassword: redhat
> EOF
adding new entry "uid=amita,ou=people,dc=replsuffix,dc=com"

[amsharma@rhel61-ds90-amita scripts]$ ldapdelete -x -h localhost -p 20100 -D "cn=Directory Manager" -w Secret123 "uid=amita,ou=people,dc=replsuffix,dc=com"
</Amita>
search them with ldapsearch:
   $ ldapsearch ... -D 'cn=directory manager' -w <pw> -b "<suffix>"
"(objectclass=nstombstone)" dn
   You should get the search results something like this:
   dn: nsuniqueid=9a63a581-...-ca8fe9f7,uid=tuser1,ou=People,dc=example,dc=com
   dn: nsuniqueid=ffffffff-ffffffff-ffffffff-ffffffff,dc=example,dc=com

<Amita>
ldapsearch -h localhost -p 20100 -D "cn=Directory Manager" -w Secret123 -b "dc=replsuffix,dc=com" "(objectclass=nstombstone)" dn
# extended LDIF
#
# LDAPv3
# base <dc=replsuffix,dc=com> with scope subtree
# filter: (objectclass=nstombstone)
# requesting: dn 
#

# ffffffff-ffffffff-ffffffff-ffffffff, replsuffix.com
dn: nsuniqueid=ffffffff-ffffffff-ffffffff-ffffffff,dc=replsuffix,dc=com

# 48b78481-723311e0-bb1bfc39-318906b0, amita, People, replsuffix.com
dn: nsuniqueid=48b78481-723311e0-bb1bfc39-318906b0,uid=amita,ou=People,dc=repl
 suffix,dc=com

# f5decc81-723411e0-bb1bfc39-318906b0, amita, People, replsuffix.com
dn: nsuniqueid=f5decc81-723411e0-bb1bfc39-318906b0,uid=amita,ou=People,dc=repl
 suffix,dc=com

# 86b8a681-723b11e0-bb1bfc39-318906b0, amita, People, replsuffix.com
dn: nsuniqueid=86b8a681-723b11e0-bb1bfc39-318906b0,uid=amita,ou=People,dc=repl
 suffix,dc=com

# search result
search: 2
result: 0 Success

# numResponses: 5
# numEntries: 4
</Amita>

2. stop the master
   export the database with '-r' option
   /usr/lib[64]/dirsrv/slapd-ID/db2ldif -n <backend> -r -a /tmp/test.ldif
<Amita>
/usr/lib64/dirsrv/slapd-M1/stop-slapd 

[amsharma@rhel61-ds90-amita slapd-M1]$ ./db2ldif -n repman20100 -r -a /tmp/test.ldif
Exported ldif file: /tmp/test.ldif
[29/Apr/2011:14:13:12 +051800] - I'm resizing my cache now...cache was 20000000 and is now 8000000
ldiffile: /tmp/test.ldif
[29/Apr/2011:14:13:13 +051800] - export repman20100: Processed 2 entries (100%).
[29/Apr/2011:14:13:13 +051800] - Waiting for 4 database threads to stop
[29/Apr/2011:14:13:14 +051800] - All database threads now stopped
</Amita>

3. import back the ldif file to the master
   /usr/lib[64]/dirsrv/slapd-ID/ldif2db -n <backend> -i /tmp/test.ldif

<Amita>
./ldif2db -n repman20100 -i /tmp/test.ldif
importing data ...
[29/Apr/2011:14:16:51 +051800] - WARNING: Import is running with nsslapd-db-private-import-mem on; No other process is allowed to access the database
[29/Apr/2011:14:16:51 +051800] - check_and_set_import_cache: pagesize: 4096, pages: 255398, procpages: 50007
[29/Apr/2011:14:16:51 +051800] - WARNING: After allocating import cache 408636KB, the available memory is 612956KB, which is less than the soft limit 1048576KB. You may want to decrease the import cache size and rerun import.
[29/Apr/2011:14:16:51 +051800] - Import allocates 408636KB import cache.
[29/Apr/2011:14:16:51 +051800] - import repman20100: Beginning import job...
[29/Apr/2011:14:16:51 +051800] - import repman20100: Index buffering enabled with bucket size 100
[29/Apr/2011:14:16:51 +051800] - import repman20100: Processing file "/tmp/test.ldif"
[29/Apr/2011:14:16:51 +051800] - import repman20100: Finished scanning file "/tmp/test.ldif" (2 entries)
[29/Apr/2011:14:16:52 +051800] - import repman20100: Workers finished; cleaning up...
[29/Apr/2011:14:16:52 +051800] - import repman20100: Workers cleaned up.
[29/Apr/2011:14:16:52 +051800] - import repman20100: Cleaning up producer thread...
[29/Apr/2011:14:16:52 +051800] - import repman20100: Indexing complete.  Post-processing...
[29/Apr/2011:14:16:52 +051800] - import repman20100: Flushing caches...
[29/Apr/2011:14:16:52 +051800] - import repman20100: Closing files...
[29/Apr/2011:14:16:52 +051800] - All database threads now stopped
[29/Apr/2011:14:16:52 +051800] - import repman20100: Import complete.  Processed 2 entries in 1 seconds. (2.00 entries/sec)

</Amita>

4. start the master 
   run the above ldapsearch to check if it returns the same result.

<Amita>
/usr/lib64/dirsrv/slapd-M1/start-slapd
ldapsearch -h localhost -p 20100 -D "cn=Directory Manager" -w Secret123 -b "dc=replsuffix,dc=com" "(objectclass=nstombstone)" dn
# extended LDIF
#
# LDAPv3
# base <dc=replsuffix,dc=com> with scope subtree
# filter: (objectclass=nstombstone)
# requesting: dn 
#

# ffffffff-ffffffff-ffffffff-ffffffff, replsuffix.com
dn: nsuniqueid=ffffffff-ffffffff-ffffffff-ffffffff,dc=replsuffix,dc=com

# 48b78481-723311e0-bb1bfc39-318906b0, amita, People, replsuffix.com
dn: nsuniqueid=48b78481-723311e0-bb1bfc39-318906b0,uid=amita,ou=People,dc=repl
 suffix,dc=com

# f5decc81-723411e0-bb1bfc39-318906b0, amita, People, replsuffix.com
dn: nsuniqueid=f5decc81-723411e0-bb1bfc39-318906b0,uid=amita,ou=People,dc=repl
 suffix,dc=com

# 86b8a681-723b11e0-bb1bfc39-318906b0, amita, People, replsuffix.com
dn: nsuniqueid=86b8a681-723b11e0-bb1bfc39-318906b0,uid=amita,ou=People,dc=repl
 suffix,dc=com

# search result
search: 2
result: 0 Success

# numResponses: 5
# numEntries: 4
</Amita>

5. initialize the consumer
   run the above ldapsearch against the consumer to check if it returns the
same result.
<Amita>
ldapsearch -h localhost -p 20102 -D "cn=Directory Manager" -w Secret123 -b "dc=replsuffix,dc=com" "(objectclass=nstombstone)" dn
# extended LDIF
#
# LDAPv3
# base <dc=replsuffix,dc=com> with scope subtree
# filter: (objectclass=nstombstone)
# requesting: dn 
#

# ffffffff-ffffffff-ffffffff-ffffffff, replsuffix.com
dn: nsuniqueid=ffffffff-ffffffff-ffffffff-ffffffff,dc=replsuffix,dc=com

# 48b78481-723311e0-bb1bfc39-318906b0, amita, People, replsuffix.com
dn: nsuniqueid=48b78481-723311e0-bb1bfc39-318906b0,uid=amita,ou=People,dc=repl
 suffix,dc=com

# f5decc81-723411e0-bb1bfc39-318906b0, amita, People, replsuffix.com
dn: nsuniqueid=f5decc81-723411e0-bb1bfc39-318906b0,uid=amita,ou=People,dc=repl
 suffix,dc=com

# 86b8a681-723b11e0-bb1bfc39-318906b0, amita, People, replsuffix.com
dn: nsuniqueid=86b8a681-723b11e0-bb1bfc39-318906b0,uid=amita,ou=People,dc=repl
 suffix,dc=com

# search result
search: 2
result: 0 Success

# numResponses: 5
# numEntries: 4
</Amita>
6. stop the master
   Open /etc/dirsrv/slapd-ID/dse.ldif and modify nsds5ReplicaPurgeDelay in
   the replica entry, e.g. (note: suffix is "dc=example,dc=com"):
   dn: cn=replica,cn=dc\3Dexample\2Cdc\3Dcom,cn=mapping tree,cn=config
   so that:
   nsds5ReplicaPurgeDelay: 60
   and add
   nsds5ReplicaTombstonePurgeInterval: 60
   Restart the master
   Wait for a minute and add any entry to the master
   Run the above ldapsearch and check that the search result only shows an RUV:
   dn: nsuniqueid=ffffffff-ffffffff-ffffffff-ffffffff,dc=example,dc=com

<Amita>
/usr/lib64/dirsrv/slapd-M1/stop-slapd
vim /etc/dirsrv/slapd-M1/dse.ldif

added
nsds5ReplicaPurgeDelay: 60
nsds5ReplicaTombstonePurgeInterval: 60

/usr/lib64/dirsrv/slapd-M1/start-slapd

ldapmodify -x -h localhost -p 20100 -D "cn=Directory Manager" -w Secret123 << EOF
> dn: uid=amita,ou=people,dc=replsuffix,dc=com
> changetype: add
> objectClass: top
> objectClass: person
> objectClass: inetorgperson
> sn: testkrbuser
> cn: kkk testkrbuser
> userPassword: redhat
> EOF

????? Why I am not able to see the message from slapd "adding new entry "uid=amita,ou=people,dc=replsuffix,dc=com", It was showing before making changes in dse.ldif?????

ldapsearch -h localhost -p 20100 -D "cn=Directory Manager" -w Secret123 -b "dc=replsuffix,dc=com" "(objectclass=nstombstone)" dn
# extended LDIF
#
# LDAPv3
# base <dc=replsuffix,dc=com> with scope subtree
# filter: (objectclass=nstombstone)
# requesting: dn 
#

# ffffffff-ffffffff-ffffffff-ffffffff, replsuffix.com
dn: nsuniqueid=ffffffff-ffffffff-ffffffff-ffffffff,dc=replsuffix,dc=com

# 86b8a681-723b11e0-bb1bfc39-318906b0, amita, People, replsuffix.com
dn: nsuniqueid=86b8a681-723b11e0-bb1bfc39-318906b0,uid=amita,ou=People,dc=repl
 suffix,dc=com

# search result
search: 2
result: 0 Success

# numResponses: 3
# numEntries: 2

?????????? Check above, Is it what is expected??????????

ldapsearch -h localhost -p 20102 -D "cn=Directory Manager" -w Secret123 -b "dc=replsuffix,dc=com" "(objectclass=nstombstone)" dn
# extended LDIF
#
# LDAPv3
# base <dc=replsuffix,dc=com> with scope subtree
# filter: (objectclass=nstombstone)
# requesting: dn 
#

# ffffffff-ffffffff-ffffffff-ffffffff, replsuffix.com
dn: nsuniqueid=ffffffff-ffffffff-ffffffff-ffffffff,dc=replsuffix,dc=com

# 48b78481-723311e0-bb1bfc39-318906b0, amita, People, replsuffix.com
dn: nsuniqueid=48b78481-723311e0-bb1bfc39-318906b0,uid=amita,ou=People,dc=repl
 suffix,dc=com

# f5decc81-723411e0-bb1bfc39-318906b0, amita, People, replsuffix.com
dn: nsuniqueid=f5decc81-723411e0-bb1bfc39-318906b0,uid=amita,ou=People,dc=repl
 suffix,dc=com

# 86b8a681-723b11e0-bb1bfc39-318906b0, amita, People, replsuffix.com
dn: nsuniqueid=86b8a681-723b11e0-bb1bfc39-318906b0,uid=amita,ou=People,dc=repl
 suffix,dc=com

# search result
search: 2
result: 0 Success

# numResponses: 5
# numEntries: 4
</Amita>

?????????? Why Master and Relica are showing different results for this ldapserach now, which was not the case before making the changes in dse.ldif??????????
Comment 3 Rich Megginson 2011-04-29 10:30:27 EDT
(In reply to comment #2)
<snip>
> <Amita>
> /usr/lib64/dirsrv/slapd-M1/stop-slapd
> vim /etc/dirsrv/slapd-M1/dse.ldif
> 
> added
> nsds5ReplicaPurgeDelay: 60
> nsds5ReplicaTombstonePurgeInterval: 60
> 
> /usr/lib64/dirsrv/slapd-M1/start-slapd
> 
> ldapmodify -x -h localhost -p 20100 -D "cn=Directory Manager" -w Secret123 <<
> EOF
> > dn: uid=amita,ou=people,dc=replsuffix,dc=com
> > changetype: add
> > objectClass: top
> > objectClass: person
> > objectClass: inetorgperson
> > sn: testkrbuser
> > cn: kkk testkrbuser
> > userPassword: redhat
> > EOF
> 
> ????? Why I am not able to see the message from slapd "adding new entry
> "uid=amita,ou=people,dc=replsuffix,dc=com", It was showing before making
> changes in dse.ldif?????

I don't know, but this has to do with the ldapmodify command and not the directory server.  You can use ldapmodify -d 1 if you want to see the trace messages from the command to verify the operation was sent to the directory server.

> 
> ldapsearch -h localhost -p 20100 -D "cn=Directory Manager" -w Secret123 -b
> "dc=replsuffix,dc=com" "(objectclass=nstombstone)" dn
> # extended LDIF
> #
> # LDAPv3
> # base <dc=replsuffix,dc=com> with scope subtree
> # filter: (objectclass=nstombstone)
> # requesting: dn 
> #
> 
> # ffffffff-ffffffff-ffffffff-ffffffff, replsuffix.com
> dn: nsuniqueid=ffffffff-ffffffff-ffffffff-ffffffff,dc=replsuffix,dc=com
> 
> # 86b8a681-723b11e0-bb1bfc39-318906b0, amita, People, replsuffix.com
> dn: nsuniqueid=86b8a681-723b11e0-bb1bfc39-318906b0,uid=amita,ou=People,dc=repl
>  suffix,dc=com
> 
> # search result
> search: 2
> result: 0 Success
> 
> # numResponses: 3
> # numEntries: 2
> 
> ?????????? Check above, Is it what is expected??????????

Yes.  The old tombstones for the deleted entries have been deleted except for the most recent one.

> 
> ldapsearch -h localhost -p 20102 -D "cn=Directory Manager" -w Secret123 -b
> "dc=replsuffix,dc=com" "(objectclass=nstombstone)" dn
> # extended LDIF
> #
> # LDAPv3
> # base <dc=replsuffix,dc=com> with scope subtree
> # filter: (objectclass=nstombstone)
> # requesting: dn 
> #
> 
> # ffffffff-ffffffff-ffffffff-ffffffff, replsuffix.com
> dn: nsuniqueid=ffffffff-ffffffff-ffffffff-ffffffff,dc=replsuffix,dc=com
> 
> # 48b78481-723311e0-bb1bfc39-318906b0, amita, People, replsuffix.com
> dn: nsuniqueid=48b78481-723311e0-bb1bfc39-318906b0,uid=amita,ou=People,dc=repl
>  suffix,dc=com
> 
> # f5decc81-723411e0-bb1bfc39-318906b0, amita, People, replsuffix.com
> dn: nsuniqueid=f5decc81-723411e0-bb1bfc39-318906b0,uid=amita,ou=People,dc=repl
>  suffix,dc=com
> 
> # 86b8a681-723b11e0-bb1bfc39-318906b0, amita, People, replsuffix.com
> dn: nsuniqueid=86b8a681-723b11e0-bb1bfc39-318906b0,uid=amita,ou=People,dc=repl
>  suffix,dc=com
> 
> # search result
> search: 2
> result: 0 Success
> 
> # numResponses: 5
> # numEntries: 4
> </Amita>
> 
> ?????????? Why Master and Relica are showing different results for this
> ldapserach now, which was not the case before making the changes in
> dse.ldif??????????

Because on the master, you told it to clean up all of the old tombstones, except for the last one.  On the replica, you did not tell it to do this.  Tombstone cleanup operations are not replicated - they are local to each server.
Comment 4 Amita Sharma 2011-05-02 01:47:44 EDT
ok, thanks for the information Rich.
Marking the bug as VERIFIED.
Comment 5 errata-xmlrpc 2011-05-19 08:42:42 EDT
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHEA-2011-0533.html