Bug 1402012
Summary: | Importing big ldif file with duplicate DNs throwing "unable to flush" error | ||
---|---|---|---|
Product: | Red Hat Enterprise Linux 6 | Reporter: | Sankar Ramalingam <sramling> |
Component: | 389-ds-base | Assignee: | mreynolds |
Status: | CLOSED ERRATA | QA Contact: | Viktor Ashirov <vashirov> |
Severity: | high | Docs Contact: | Marc Muehlfeld <mmuehlfe> |
Priority: | high | ||
Version: | 6.9 | CC: | mreynolds, nhosoi, nkinder, rmeggins, sramling, tlavigne |
Target Milestone: | rc | ||
Target Release: | --- | ||
Hardware: | x86_64 | ||
OS: | Linux | ||
Whiteboard: | |||
Fixed In Version: | 389-ds-base-1.2.11.15-86.el6 | Doc Type: | Bug Fix |
Doc Text: |
Directory Server no longer displays multiple error messages when importing fails
Previously, if importing data failed, multiple "Unable to flush" error message were be displayed, because the connection to the database was not closed. This update applies a patch and as a result, Directory Server no longer displays multiple errors in the mentioned situation.
|
Story Points: | --- |
Clone Of: | Environment: | ||
Last Closed: | 2017-03-21 10:24:12 UTC | Type: | Bug |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: |
Comment 6
Sankar Ramalingam
2016-12-15 16:26:10 UTC
I was able to reproduce this with a 3 entry ldif file: dn: dc=example,dc=com objectclass: top objectclass: domain dc: example dn: ou=myDups00001,dc=example,dc=com objectclass: top objectclass: organizationalUnit ou: myDups00001 dn: ou=myDups00001,dc=example,dc=com objectclass: top objectclass: organizationalUnit ou: myDups00001 On master branch I see: [19/Dec/2016:12:05:06.356574675 -0500] - INFO - import_producer - import userroot: Processing file "/tmp/out.ldif" [19/Dec/2016:12:05:06.562994763 -0500] - ERR - _entryrdn_insert_key - Same DN (dn: ou=myDups00001,dc=example,dc=com) is already in the entryrdn file with different ID 2. Expected ID is 3. [19/Dec/2016:12:05:06.565045265 -0500] - ERR - foreman_do_entryrdn - import userroot: Duplicated DN detected: "ou=myDups00001,dc=example,dc=com": Entry ID: (3) [19/Dec/2016:12:05:06.658642236 -0500] - ERR - import_run_pass - import userroot: Thread monitoring returned: -23 [19/Dec/2016:12:05:06.660725660 -0500] - ERR - import_main_offline - import userroot: Aborting all Import threads... [19/Dec/2016:12:05:12.168287348 -0500] - ERR - import_main_offline - import userroot: Import threads aborted. [19/Dec/2016:12:05:12.170243072 -0500] - INFO - import_main_offline - import userroot: Closing files... [19/Dec/2016:12:05:12.184798674 -0500] - ERR - libdb - BDB3028 userroot/entryrdn.db: unable to flush: No such file or directory [19/Dec/2016:12:05:12.186952872 -0500] - ERR - libdb - BDB3028 userroot/id2entry.db: unable to flush: No such file or directory [19/Dec/2016:12:05:12.188989348 -0500] - ERR - libdb - BDB3028 userroot/parentid.db: unable to flush: No such file or directory /var/lib/dirsrv/slapd-localhost/db/userroot: No such file or directory [19/Dec/2016:12:05:12.203029854 -0500] - INFO - dblayer_pre_close - All database threads now stopped [19/Dec/2016:12:05:12.205085596 -0500] - ERR - import_main_offline - import userroot: Import failed. Upstream ticket: https://fedorahosted.org/389/ticket/49071 There was a bug in DS where we were deleting the backend files before we closed the database down - this is what caused those unexpected errors. Fixed upstream. Unable to flush error is gone now with the latest build of 389-ds-base-1.2.11.15-86. However, the suffix/database cannot be reached for further ldap operations. I did also notice that the import task is still running and cannot be deleted. I believe the import task should exit as soon as it says "Aborting all import threads". [root@cypher 389ds-replica]# /usr/lib64/dirsrv/slapd-Inst1/ldif2db.pl -D "cn=Directory Manager" -w Secret123 -n importest1121 -s "dc=importest,dc=com" -i /var/lib/dirsrv/slapd-Inst1/ldif/MyNew02_01.ldif adding new entry "cn=import_2017_1_11_4_2_35, cn=import, cn=tasks, cn=config" [root@cypher export]# tail -f /var/log/dirsrv/slapd-Inst1/errors [11/Jan/2017:04:02:35 -0500] - import importest1121: Beginning import job... [11/Jan/2017:04:02:35 -0500] - import importest1121: Index buffering enabled with bucket size 19 [11/Jan/2017:04:02:35 -0500] - import importest1121: Processing file "/var/lib/dirsrv/slapd-Inst1/ldif/MyNew02_01.ldif" [11/Jan/2017:04:02:55 -0500] - import importest1121: Processed 12319 entries -- average rate 616.0/sec, recent rate 615.9/sec, hit ratio 0% [11/Jan/2017:04:03:15 -0500] - import importest1121: Processed 22870 entries -- average rate 571.8/sec, recent rate 571.7/sec, hit ratio 100% [11/Jan/2017:04:03:36 -0500] - import importest1121: Processed 33687 entries -- average rate 561.4/sec, recent rate 534.2/sec, hit ratio 100% [11/Jan/2017:04:03:56 -0500] - import importest1121: Processed 44062 entries -- average rate 550.7/sec, recent rate 529.7/sec, hit ratio 100% [11/Jan/2017:04:04:16 -0500] - import importest1121: Processed 54003 entries -- average rate 534.7/sec, recent rate 495.5/sec, hit ratio 100% [11/Jan/2017:04:04:42 -0500] - import importest1121: Processed 67035 entries -- average rate 527.8/sec, recent rate 488.9/sec, hit ratio 100% [11/Jan/2017:04:05:02 -0500] - import importest1121: Processed 76844 entries -- average rate 522.7/sec, recent rate 496.5/sec, hit ratio 99% [11/Jan/2017:04:05:22 -0500] - import importest1121: Processed 86641 entries -- average rate 518.8/sec, recent rate 490.1/sec, hit ratio 99% [11/Jan/2017:04:05:38 -0500] - ldbm: 'importest1121' is already in the middle of another task and cannot be disturbed. [11/Jan/2017:04:05:48 -0500] - import importest1121: Processed 98924 entries -- average rate 512.6/sec, recent rate 480.0/sec, hit ratio 99% [11/Jan/2017:04:05:51 -0500] entryrdn-index - _entryrdn_insert_key: Same DN (dn: ou=Netscape Servers,dc=importest,dc=com) is already in the entryrdn file with different ID 160. Expected ID is 100315. [11/Jan/2017:04:05:58 -0500] - import importest1121: Duplicated DN detected: "ou=Netscape Servers,dc=importest,dc=com": Entry ID: (100315) [11/Jan/2017:04:05:59 -0500] - import importest1121: Aborting all Import threads... [root@cypher 389ds-replica]# date Wed Jan 11 05:52:52 EST 2017 [root@cypher 389ds-replica]# ldapsearch -LLL -x -p 1121 -h localhost -D "cn=Directory Manager" -w Secret123 -b "dc=importest,dc=com" Operations error (1) [root@cypher 389ds-replica]# ldapdelete -x -p 1121 -h localhost -D "cn=Directory Manager" -w Secret123 "cn=import_2017_1_11_4_2_35,cn=import,cn=tasks,cn=config" ldap_delete: Server is unwilling to perform (53) [root@cypher 389ds-replica]# rpm -qa |grep -i 389-ds-base 389-ds-base-libs-1.2.11.15-86.el6.x86_64 389-ds-base-debuginfo-1.2.11.15-86.el6.x86_64 389-ds-base-1.2.11.15-86.el6.x86_64 389-ds-base-devel-1.2.11.15-86.el6.x86_64 [root@cypher 389ds-replica]# ldapsearch -x -p 1121 -h localhost -D "cn=Directory Manager" -w Secret123 -b "cn=import, cn=tasks, cn=config" |grep -i "dn: cn=import_2017_1_11_4_2_35" dn: cn=import_2017_1_11_4_2_35,cn=import,cn=tasks,cn=config Hi Sankar, This is all expected behavior. Since the import fails the database is now hosed. So the operations error is correct. Also, you can not delete tasks - the server typically handles that at shutdown. Mark Based on comment #11 and comment #12, marking the bug as Verified. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHBA-2017-0667.html |