Red Hat Bugzilla – Bug 1353629
DS shuts down automatically if dnaThreshold is set to 0 in a MMR setup
Last modified: 2016-11-03 16:43:51 EDT
Description of problem: DS shuts down automatically if dnaThreshold is set to 0 in a MMR setup Version-Release number of selected component (if applicable): [root@ds ~]# rpm -qa | grep 389 389-ds-base-libs-1.3.5.10-1.el7.x86_64 389-ds-base-1.3.5.10-1.el7.x86_64 How reproducible: Always Steps to Reproduce: It's a MMR setup with 2 masters 1. Enabled the dna plugin on both instances 2. Added the required container entries for the dna plugin on both masters 3. Now on master1 which will basically transfer the next range to master2, I added the dna plugin configuration entry like this: dn: cn=Account UIDs,cn=Distributed Numeric Assignment Plugin,cn=plugins,cn=config objectClass: top objectClass: dnaPluginConfig cn: Account UIDs dnatype: uidNumber dnatype: gidNumber dnafilter: (objectclass=posixAccount) dnascope: ou=People,dc=example,dc=com dnaNextValue: 1 dnaMaxValue: 50 dnasharedcfgdn: cn=Account UIDs,ou=Ranges,dc=example,dc=com dnaThreshold: 0 dnaRangeRequestTimeout: 60 dnaMagicRegen: magic dnaRemoteBindDN: uid=dnaAdmin,ou=People,dc=example,dc=com dnaRemoteBindCred: secret123 dnaNextRange: 80-90 As can be seen in the above entry, I've set dnaThreshold to '0' and the dnaNextRange to 80-90 4. Then on master2, I added dna plugin configuration entry like this, dn: cn=Account UIDs,cn=Distributed Numeric Assignment Plugin,cn=plugins,cn=config objectClass: top objectClass: dnaPluginConfig cn: Account UIDs dnatype: uidNumber dnatype: gidNumber dnafilter: (objectclass=posixAccount) dnascope: ou=People,dc=example,dc=com dnanextvalue: 61 dnaMaxValue: 70 dnasharedcfgdn: cn=Account UIDs,ou=Ranges,dc=example,dc=com dnaThreshold: 2 dnaRangeRequestTimeout: 60 dnaMagicRegen: magic dnaRemoteBindDN: uid=dnaAdmin,ou=People,dc=example,dc=com dnaRemoteBindCred: secret123 master2 only has 10 numbers which it can allocate automatically for uidNumber and gidNumber attributes 5. Then I added added the required replication configuration entries on both masters to configure replication 6. Added 10 entries on master2 to exhaust its available range 7. Did an ldapsearch on master2, all the 10 entries were added and the attributes uidNumber and gidNumber were set accordingly by dna plugin, so far so good 8. Now I tried adding 11th entry on master2, It basically failed with this error ldap_add: Operations error (1) additional info: Allocation of a new value for range cn=account uids,cn=distributed numeric assignment plugin,cn=plugins,cn=config failed! Unable to proceed. 9. Checked the error logs on master2, this is what they show [01/Jul/2016:18:11:00.370146064 +051800] slapi_ldap_bind - Error: could not send bind request for id [uid=dnaAdmin,ou=People,dc=example,dc=com] authentication mechanism [SIMPLE]: error -1 (Can't contact LDAP server), system error -5987 (Invalid function argument.), network error 107 (Transport endpoint is not connected, host "ds.example.com:3389") [01/Jul/2016:18:12:49.411378922 +051800] dna-plugin - dna_request_range: Error sending range extension extended operation request to server ds.example.com:3389 [error -1] [01/Jul/2016:18:12:49.536799341 +051800] dna-plugin - dna_pre_op: no more values available!! [01/Jul/2016:18:14:12.466096468 +051800] slapi_ldap_bind - Error: could not send bind request for id [uid=dnaAdmin,ou=People,dc=example,dc=com] authentication mechanism [SIMPLE]: error -1 (Can't contact LDAP server), system error -5987 (Invalid function argument.), network error 107 (Transport endpoint is not connected, host "ds.example.com:3389") 10. Upon investing with status-dirsrv, I found that master1 was killed [root@ds ~]# status-dirsrv mast1 ● dirsrv@mast1.service - 389 Directory Server mast1. Loaded: loaded (/usr/lib/systemd/system/dirsrv@.service; enabled; vendor preset: disabled) Active: failed (Result: signal) since Thu 2016-07-07 20:35:23 IST; 9min ago Main PID: 5852 (code=killed, signal=FPE) Status: "slapd started: Ready to process requests" The status however here still shows Ready to process requests 11. Restarted master1 with start-dirsrv master1 12. Checked error logs on master1, this is what they show [01/Jul/2016:18:18:04.629383441 +051800] Detected Disorderly Shutdown last time Directory Server was running, recovering database. [01/Jul/2016:18:18:04.816326545 +051800] NSMMReplicationPlugin - changelog program - _cl5NewDBFile: PR_DeleteSemaphore: /var/lib/dirsrv/slapd-mast1/changelogdb/62f98283-3f8811e6-8e9ac15d-c133e87f.sema; NSPR error - -5943 [01/Jul/2016:18:18:04.974329732 +051800] NSMMReplicationPlugin - replica_check_for_data_reload: Warning: disordely shutdown for replica dc=example,dc=com. Check if DB RUV needs to be updated [01/Jul/2016:18:18:05.011287516 +051800] slapd started. Listening on All Interfaces port 3389 for LDAP requests 13. Did an ldapsearch on master1 and found that only some of the 10 entries that were added on master2 were replicated to master1 and others were missing
Upstream ticket: https://fedorahosted.org/389/ticket/48916
RHEL: RHEL 7.3 x86_64 Server DS builds: [root@org47 ~]# rpm -qa | grep 389-ds-base 389-ds-base-1.3.5.10-4.el7.x86_64 389-ds-base-snmp-1.3.5.10-4.el7.x86_64 389-ds-base-libs-1.3.5.10-4.el7.x86_64 Steps Performed: 1. Created two standalone instances as master1 and master2 2. Configured DNA plugin on both instances, Below is the DNA plugin configuration entry on master1 dn: cn=Account UIDs,cn=Distributed Numeric Assignment Plugin,cn=plugins,cn=config objectClass: top objectClass: dnaPluginConfig cn: Account UIDs dnatype: uidNumber dnatype: gidNumber dnafilter: (objectclass=posixAccount) dnascope: ou=People,dc=example,dc=com dnaNextValue: 1 dnaMaxValue: 20 dnasharedcfgdn: cn=Account UIDs,ou=Ranges,dc=example,dc=com dnaThreshold: 0 dnaRangeRequestTimeout: 60 dnaMagicRegen: magic dnaRemoteBindDN: uid=dnaAdmin,ou=People,dc=example,dc=com dnaRemoteBindCred: secret123 dnaNextRange: 41-50 dnaThreshold is set to 0 on master1 Below is the DNA plugin configuration entry on master2 dn: cn=Account UIDs,cn=Distributed Numeric Assignment Plugin,cn=plugins,cn=config objectClass: top objectClass: dnaPluginConfig cn: Account UIDs dnatype: uidNumber dnatype: gidNumber dnafilter: (objectclass=posixAccount) dnascope: ou=People,dc=example,dc=com dnanextvalue: 21 dnaMaxValue: 30 dnasharedcfgdn: cn=Account UIDs,ou=Ranges,dc=example,dc=com dnaThreshold: 2 dnaRangeRequestTimeout: 60 dnaMagicRegen: magic dnaRemoteBindDN: uid=dnaAdmin,ou=People,dc=example,dc=com dnaRemoteBindCred: secret123 master2 only has 10 numbers available for allocation 3. Configured a 2x MMR setup by added required replication configuration entries on both masters 4. Added 10 users on master2 to exhaust its available range [root@org47 dna_setup]# ldapmodify -x -D 'cn=Directory Manager' -w secret123 -h localhost -p 1389 -a -f users.ldif adding new entry "uid=testuser21,ou=People,dc=example,dc=com" adding new entry "uid=testuser22,ou=People,dc=example,dc=com" adding new entry "uid=testuser23,ou=People,dc=example,dc=com" adding new entry "uid=testuser24,ou=People,dc=example,dc=com" adding new entry "uid=testuser25,ou=People,dc=example,dc=com" adding new entry "uid=testuser26,ou=People,dc=example,dc=com" adding new entry "uid=testuser27,ou=People,dc=example,dc=com" adding new entry "uid=testuser28,ou=People,dc=example,dc=com" adding new entry "uid=testuser29,ou=People,dc=example,dc=com" adding new entry "uid=testuser30,ou=People,dc=example,dc=com" 5. Checked the status of master1 [root@org47 ~]# status-dirsrv master1 ● dirsrv@master1.service - 389 Directory Server master1. Loaded: loaded (/usr/lib/systemd/system/dirsrv@.service; enabled; vendor preset: disabled) Active: active (running) since Sun 2016-09-18 17:31:58 IST; 9min ago Main PID: 19494 (ns-slapd) Status: "slapd started: Ready to process requests" master1 is still running, no crash here 6. Tried adding another entry on master2 as below [root@org47 dna_setup]# ldapmodify -x -D 'cn=Directory Manager' -w secret123 -h localhost -p 1389 -a -f user.ldif adding new entry "uid=testuser32,ou=People,dc=example,dc=com" ldap_add: Operations error (1) additional info: Allocation of a new value for range cn=account uids,cn=distributed numeric assignment plugin,cn=plugins,cn=config failed! Unable to proceed. 7. Modified the value of dnaThreshold on master1 [root@org47 dna_setup]# ldapmodify -x -D 'cn=Directory Manager' -w secret123 -h localhost -p 389 dn: cn=Account UIDs,cn=Distributed Numeric Assignment Plugin,cn=plugins,cn=config changetype: modify replace: dnaThreshold dnaThreshold: 5 modifying entry "cn=Account UIDs,cn=Distributed Numeric Assignment Plugin,cn=plugins,cn=config" 9. Once again tried to add an entry on master2 [root@org47 dna_setup]# ldapmodify -x -D 'cn=Directory Manager' -w secret123 -h localhost -p 1389 -a -f user.ldif adding new entry "uid=testuser31,ou=People,dc=example,dc=com" entry was added successfully 10. Verified that range was transferred properly [root@org47 dna_setup]# ldapsearch -xLLL -b 'uid=testuser31,ou=People,dc=example,dc=com' -h localhost -p 1389 dn: uid=testuser31,ou=People,dc=example,dc=com cn: test user homeDirectory: /home/testuser objectClass: top objectClass: person objectClass: inetOrgPerson objectClass: posixAccount objectClass: organizationalPerson sn: user uid: testuser uid: testuser31 uidNumber: 46 gidNumber: 46
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHSA-2016-2594.html