Bug 693962

Summary: Full replica push loses some entries with multi-valued RDNs
Product: [Retired] 389 Reporter: Karsten Sperling <mail>
Component: Database - Indexes/SearchesAssignee: Rich Megginson <rmeggins>
Status: CLOSED CURRENTRELEASE QA Contact: Viktor Ashirov <vashirov>
Severity: high Docs Contact:
Priority: unspecified    
Version: 1.2.8CC: amsharma
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 694927 (view as bug list) Environment:
Last Closed: 2015-12-07 16:41:57 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 639035, 656390, 694927    
Attachments:
Description Flags
0001-Bug-693962-Full-replica-push-loses-some-entries-with.patch
none
0001-Bug-693962-Full-replica-push-loses-some-entries-with.patch nhosoi: review+

Description Karsten Sperling 2011-04-06 04:58:14 UTC 
Description of problem:

In 1.2.8 the components of multi-valued RDNs seem to be sorted, whereas in 1.2.5 the server kept them in the order they were in when the entry was created. When doing a full replica push from my 1.2.5 instance to 1.2.8 only those entries with multi-valued RDNs make it into 1.2.8 where the RDN components happen to be in the correct order.

Version-Release number of selected component (if applicable):

1.2.8.rc2, replicating from 1.2.5

How reproducible:

I have these two entries (amongst others) in the 1.2.5 instance:

dn: cn=Karsten+sn=Sperling,ou=People,dc=sky,dc=co,dc=nz
objectClass: inetOrgPerson
objectClass: top
objectClass: organizationalPerson
objectClass: person
cn: Karsten
sn: Sperling

dn: sn=Doe+cn=John,ou=People,dc=sky,dc=co,dc=nz
objectClass: inetOrgPerson
objectClass: top
objectClass: organizationalPerson
objectClass: person
cn: John
sn: Doe

After doing a full replica push to the 1.2.8 instance, only the first one of those is present in the 1.2.8 instance, the John Doe entry is missing.

If the John Doe entry is imported directly into the 1.2.8 instance, the DN is normalized to cn=John+sn=Doe,ou=People,dc=sky,dc=co,dc=nz. It seems the absence of this normalization step during the full replica initialization is what's causing this issue.
Comment 1 Rich Megginson 2011-04-08 18:02:01 UTC 
Created attachment 490848 [details]
0001-Bug-693962-Full-replica-push-loses-some-entries-with.patch
Comment 2 Rich Megginson 2011-04-08 18:05:37 UTC 
Created attachment 490849 [details]
0001-Bug-693962-Full-replica-push-loses-some-entries-with.patch
Comment 3 Rich Megginson 2011-04-08 22:56:42 UTC 
To ssh://git.fedorahosted.org/git/389/ds.git
   f9237d6..a82b8f0  389-ds-base-1.2.8 -> 389-ds-base-1.2.8
commit 54b95201cfc274c72e2364f49fc21b8a8e8fa688
Author: Rich Megginson <rmeggins>
Date:   Fri Apr 8 11:55:27 2011 -0600
   47e725d..2472169  master -> master
commit 2472169b3e077ff379812f9e91f439ce1e4edffb
Author: Rich Megginson <rmeggins>
Date:   Fri Apr 8 11:55:27 2011 -0600
    Reviewed by: nhosoi (Thanks!)
    Branch: master
    Fix Description: The code in _entryrdn_insert_key was assuming the srdn
    passed in was already normalized.  This is not true in some cases where
    the data is coming from a source of old data such as replication with an
    older server.  The solution is to make sure the rdn code always
    normalizes the code using slapi_dn_normalize_case_ext() instead of
    slapi_dn_normalize_case() which now doesn't do any normalization, it just
    converts the given string to lower case.  I added a function
    normalize_case_helper() to return a normalized dn as a copy or in place
    depending on the arguments.
    Tested with gdb - stepped through and verified the char arrays are correctly
    replaced, and copy values are correctly assigned.
    Used valgrind with online import to verify no leaks or errors.
    Exported a netscaperoot ldif from an older 1.2.8 server and imported with
    the new code.  Verified that the dbscan -f entryrdn.db4 output was
    identical between the two.
    Platforms tested: RHEL6 x86_64
    Flag Day: no
    Doc impact: no
Comment 4 Amita Sharma 2011-05-02 17:31:56 UTC 
Master - 20100
Slave - 20102

[root@rhel61-ds90-amita ~]# ldapmodify -x -h localhost -p 20100 -D
"cn=Directory Manager" -w Secret123 << EOF
> dn: sn=Doe+cn=John,ou=people,dc=replsuffix,dc=com
> changetype: add
> objectClass: inetOrgPerson
> objectClass: top
> objectClass: organizationalPerson
> objectClass: person
> cn: John
> sn: Doe
> EOF
adding new entry "sn=Doe+cn=John,ou=people,dc=replsuffix,dc=com"

[root@rhel61-ds90-amita ~]# ldapsearch -h localhost -p 20100 -D "cn=Directory
Manager" -w Secret123 -b "ou=people,dc=replsuffix,dc=com"
# extended LDIF
#
# LDAPv3
# base <ou=people,dc=replsuffix,dc=com> with scope subtree
# filter: (objectclass=*)
# requesting: ALL
#

# People, replsuffix.com
dn: ou=People,dc=replsuffix,dc=com
objectClass: top
objectClass: organizationalunit
ou: People

# John + Doe, People, replsuffix.com
dn: cn=John+sn=Doe,ou=People,dc=replsuffix,dc=com
objectClass: inetOrgPerson
objectClass: top
objectClass: organizationalPerson
objectClass: person
cn: John
sn: Doe

# search result
search: 2
result: 0 Success

# numResponses: 3
# numEntries: 2

ldapsearch -h localhost -p 20102 -D "cn=Directory Manager" -w Secret123 -b
"ou=people,dc=replsuffix,dc=com"
# extended LDIF
#
# LDAPv3
# base <ou=people,dc=replsuffix,dc=com> with scope subtree
# filter: (objectclass=*)
# requesting: ALL
#

# People, replsuffix.com
dn: ou=People,dc=replsuffix,dc=com
ou: People
objectClass: top
objectClass: organizationalunit

# John + Doe, People, replsuffix.com
dn: cn=John+sn=Doe,ou=People,dc=replsuffix,dc=com
objectClass: inetOrgPerson
objectClass: top
objectClass: organizationalPerson
objectClass: person
cn: John
sn: Doe

# search result
search: 2
result: 0 Success

# numResponses: 3
# numEntries: 2

Entry got replicated without crash. Bug is VERIFIED.