Bug 730396
| Summary: | DEADLOCK in entryrdn.db may add "nsuniqueid=...+" to the DN | ||||||
|---|---|---|---|---|---|---|---|
| Product: | [Retired] 389 | Reporter: | Noriko Hosoi <nhosoi> | ||||
| Component: | Database - Indexes/Searches | Assignee: | Noriko Hosoi <nhosoi> | ||||
| Status: | CLOSED NOTABUG | QA Contact: | Chandrasekar Kannan <ckannan> | ||||
| Severity: | unspecified | Docs Contact: | |||||
| Priority: | unspecified | ||||||
| Version: | 1.2.9 | CC: | benl, rmeggins | ||||
| Target Milestone: | --- | ||||||
| Target Release: | --- | ||||||
| Hardware: | Unspecified | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | Doc Type: | Bug Fix | |||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2011-08-12 20:15:57 UTC | Type: | --- | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Attachments: |
|
||||||
Created attachment 518095 [details]
test program to reproduce the problem
How to reproduce the problem.
Build the test program:
$ gcc -Wall replperf.c -o replperf -lpthread -lldap -lm
Set up 2-way MMR + one read only replica.
Assume 3 servers are all on hostX.
M1 (port 389) <--> M2 (port 390)
\ /
v v
read only replica (port 391)
Launch the test program from 2 terminals.
terminal1> ./replperf -h hostX -p 389 -i hostX -q 391 -D 'cn=directory manager' -w password -d 'cn=directory manager' -W password -n 25000 -I 50 -t 4 -a -e user389
terminal2> ./replperf -h hostX -p 390 -i hostX -q 391 -D 'cn=directory manager' -w password -d 'cn=directory manager' -W password -n 25000 -I 50 -t 4 -a -e user390
When it's done, run a command line against the replica (port 391) to get the DNs to which nsuniqueid is added. If none, this bug is fixed.
$ ldapsearch -LLLx -h hostX -p 391 -b "dc=example,dc=com" -D 'cn=directory manager' -w password "nsuniqueid=*" dn | egrep nsuniqueid
dn: nsuniqueid=958d3759-c48111e0-9386a9ab-cfa2440d+uid=user389.3.23,dc=example
dn: nsuniqueid=958d375f-c48111e0-9386a9ab-cfa2440d+uid=user389.0.23,dc=example
Also check the error log of the replica. DEADLOCKs would still have occurred.
[11/Aug/2011:15:07:34 -0700] entryrdn-index - _entryrdn_put_data: Adding the par
ent link (P9918:uid=user389.1.497) failed: DB_LOCK_DEADLOCK: Locker killed to re
solve a deadlock (-30994)
It's NOT a bug. It's "just" a conflict. I am closing this bug.
# entry-id: 727
dn: nsuniqueid=ae95e8a1-c48111e0-b5efb617-5d664145+uid=user390.1.40,dc=example
,dc=com
nsUniqueId: ae95e8a1-c48111e0-b5efb617-5d664145
objectClass;vucsn-4e4480a9000500020000: top
objectClass;vucsn-4e4480a9000500020000: person
objectClass;vucsn-4e4480a9000500020000: organizationalperson
objectClass;vucsn-4e4480a9000500020000: inetorgperson
cn;vucsn-4e4480a9000500020000: cn value
sn;vucsn-4e4480a9000500020000: sn value
givenName;vucsn-4e4480a9000500020000: givenname value
mail;vucsn-4e4480a9000500020000: mail value
uid;vucsn-4e4480a9000500020000;mdcsn-4e4480a9000500020000: user390.1.40
userPassword;vucsn-4e4480a9000500020000: {SSHA}4/vLsqRnfa8be0I95sEtZbkPYzghbmX
/opS1aw==
creatorsName;vucsn-4e4480a9000500020000: cn=directory manager
modifiersName;vucsn-4e4480a9000500020000: cn=directory manager
createTimestamp;vucsn-4e4480a9000500020000: 20110812012352Z
modifyTimestamp;vucsn-4e4480a9000500020000: 20110812012352Z
nsds5ReplConflict;vucsn-4e4480a9000500020000: namingConflict uid=user390.1.40,dc=example,dc=com
|
Description of problem: The test program (being attached later) adds entries with dn: uid=user<master_port>.<thread_num>.<seq_num>,dc=example,dc=com where <master_port> is 389 or 390, <thread_num> is 0, 1, 2, or 3, <seq_num> is 0 .. 24999. Since 4 threads stresses the same location of the entryrdn index, it causes quite many DEADLOCKs. I see 1131 DEADLOCKS out of 100,000 adds. [11/Aug/2011:18:24:52 -0700] entryrdn-index - _entryrdn_put_data: Adding the parent link (P727:nsuniqueid=ae95e8a1-c48111e0-b5efb617-5d664145+uid=user390.1.40) failed: DB_LOCK_DEADLOCK: Locker killed to resolve a deadlock (-30994) The DEADLOCK should be roll-backed and the operation should be retried. But it seems there is a bug in the retry code and 282 entries out of 1131 DEADLOCK errors end up having "nsuniqueid=...+" in front of the dn as follows: $ ldapsearch -LLLx -h nereid -p 391 -b "dc=example,dc=com" -D 'cn=directory manager' -w Secret123 "uid=user390.1.40" dn: nsuniqueid=ae95e8a1-c48111e0-b5efb617-5d664145+uid=user390.1.40,dc=example,dc=com objectClass: top objectClass: person objectClass: organizationalperson objectClass: inetorgperson cn: cn value sn: sn value givenName: givenname value mail: mail value uid: user390.1.40 userPassword:: e1NTSEF9NC92THNxUm5mYThiZTBJOTVzRXRaYmtQWXpnaGJtWC9vcFMxYXc9PQ== It does not have a nsTombstone objectclass value. And the entry is still indexed (e.g., in the uid.db4). So, as ldapsearch returns the entry, it's not "deleted". But the DN is not correct.