Bug 474237 - db2ldif -s "suffix" issues confusing warnings when sub suffix exists
db2ldif -s "suffix" issues confusing warnings when sub suffix exists
Status: CLOSED CURRENTRELEASE
Product: 389
Classification: Community
Component: Command Line Utilities (Show other bugs)
1.1.3
All Linux
low Severity medium
: ---
: ---
Assigned To: Noriko Hosoi
Chandrasekar Kannan
:
Depends On:
Blocks: 249650 FDS1.2.0
  Show dependency treegraph
 
Reported: 2008-12-02 17:06 EST by Noriko Hosoi
Modified: 2015-01-04 18:35 EST (History)
3 users (show)

See Also:
Fixed In Version: 8.1
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2009-04-29 19:08:19 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
cvs diff ldap/servers/slapd/main.c ldap/servers/slapd/back-ldbm/ldif2ldbm.c (11.17 KB, patch)
2008-12-03 16:14 EST, Noriko Hosoi
no flags Details | Diff
cvs commit message (1.49 KB, text/plain)
2008-12-03 19:51 EST, Noriko Hosoi
no flags Details

  None (edit)
Description Noriko Hosoi 2008-12-02 17:06:01 EST
Steps to reproduce the problem
0. Create a DS instance which suffix is "dc=example,dc=com"
1. Create a sub suffix "ou=sub,dc=example,dc=com"
2. Add or import some entries under dc=example,dc=com as well as under ou=sub,dc=example,dc=com
3. Export the contents
   db2ldif -s "dc=example,dc=com"

Here's the output from my test DS:
Exported ldif file: /var/lib/dirsrv/slapd-m0/ldif/m0-example-2008_12_02_135318.ldif
[02/Dec/2008:13:53:18 -0800] - Backend Instance: userRoot
ldiffile: /var/lib/dirsrv/slapd-m0/ldif/m0-example-2008_12_02_135318.ldif
[02/Dec/2008:13:53:18 -0800] - export userRoot: Processed 1000 entries (19%).
[02/Dec/2008:13:53:18 -0800] - export userRoot: Processed 2000 entries (39%).
[02/Dec/2008:13:53:18 -0800] - export userRoot: Processed 3000 entries (59%).
[02/Dec/2008:13:53:18 -0800] - export userRoot: Processed 4000 entries (79%).
[02/Dec/2008:13:53:19 -0800] - export userRoot: Processed 5000 entries (99%).
[02/Dec/2008:13:53:19 -0800] - export userRoot: Processed 5007 entries (100%).
[02/Dec/2008:13:53:19 -0800] - All database threads now stopped
ldiffile: /var/lib/dirsrv/slapd-m0/ldif/m0-example-2008_12_02_135318.ldif
[02/Dec/2008:13:53:19 -0800] - WARNING: ldbm instance userRoot already exists
[02/Dec/2008:13:53:19 -0800] - WARNING: ldbm instance subRoot already exists
[02/Dec/2008:13:53:19 -0800] - warning: entrydn not indexed on 'dc=example,dc=com'; entry dc=example,dc=com may not be added to the database yet.
[02/Dec/2008:13:53:19 -0800] - Failed to fetch subtree lists (error 0) Successful return: 0
[02/Dec/2008:13:53:19 -0800] - Possibly the entrydn or ancestorid index is corrupted or does not exist.
[02/Dec/2008:13:53:19 -0800] - Attempting direct unindexed export instead.
[02/Dec/2008:13:53:19 -0800] - export subRoot: Processed 17 entries (100%).
[02/Dec/2008:13:53:19 -0800] - All database threads now stopped

From the line: WARNING: ldbm instance userRoot already exists
to: Attempting direct unindexed export instead.
are unnecessary since dc=example,dc=com is a parent suffix which is not part of the backend that sub suffix "ou=sub,dc=example,dc=com" belongs to.
Comment 1 Noriko Hosoi 2008-12-03 16:14:30 EST
Created attachment 325600 [details]
cvs diff ldap/servers/slapd/main.c ldap/servers/slapd/back-ldbm/ldif2ldbm.c
Comment 2 Noriko Hosoi 2008-12-03 16:48:39 EST
Fix description:
[main.c]
* if -s <dn> is passed to db2ldif, the <dn> is used to look up the instance name the <dn> belongs to with the base dn "cn=mapping tree,cn=config" and the filter "(&(objectclass=nsmappingtree)(|(cn=*<dn>\")(cn=*<dn>)))".  If the <dn> is not the suffix, but the sub node, it fails to find out the instance which contains the <dn>.  To solve the problem, going upward the DIT until the instance is found.
With this fix, -s takes non suffix dn:
Before>
./db2ldif -s "ou=payroll,dc=example,dc=com"
Exported ldif file: /var/lib/dirsrv/slapd-m0/ldif/m0-payroll-2008_12_03_132555.ldif
[..] - ERROR 2: There is no backend instance to export from.
After>
./db2ldif -s "ou=payroll,dc=example,dc=com"
Exported ldif file: /var/lib/dirsrv/slapd-m0/ldif/m0-payroll-2008_12_03_132358.ldif
[..] - Backend Instance(s): 
[..] - 	userRoot
ldiffile: /var/lib/dirsrv/slapd-m0/ldif/m0-payroll-2008_12_03_132358.ldif
[..] - export userRoot: Processed 981 entries (100%).
[..] - All database threads now stopped

* If multiple backends are specified to export, all the names are printed.
./db2ldif -s "dc=example,dc=com" -s "dc=test,dc=com"
Exported ldif file: /var/lib/dirsrv/slapd-m0/ldif/m0-example-test-2008_12_03_133016.ldif
[..] - Backend Instance(s): 
[..] - 	userRoot
[..] - 	subRoot
[..] - 	testRoot
ldiffile: /var/lib/dirsrv/slapd-m0/ldif/m0-example-test-2008_12_03_133016.ldif
[..] - export userRoot: Processed 1000 entries (19%).
       [..]

[ldif2ldbm.c]
* ldbm_fetch_subtrees: when -s <dn> is passsed to db2ldif, added a logic to avoid the further process if the <dn> does not belong to the backend.  It eliminates this unnecessary warning:
    warning: entrydn not indexed on 'dc=example,dc=com'; entry 
    dc=example,dc=com may not be added to the database yet.
Also, changed the "warning:" to "info:" since this is not a problem.

* When multiple backends are exported, dse was loaded each time.  Changed not to do so.  It eliminates these warnings:
    [..] - WARNING: ldbm instance userRoot already exists
    [..] - WARNING: ldbm instance subRoot already exists

* The following messages were always printed if ldbm_fetch_subtrees return NULL.  But the messages are true only when the error code is set.
    [..] - Failed to fetch subtree lists (error 0) Successful return: 0
    [..] - Possibly the entrydn or ancestorid index is corrupted or does not exist.
    [..] - Attempting direct unindexed export instead.

* Export counter was not decremented when the entry was not to be exported.
Comment 3 Noriko Hosoi 2008-12-03 16:54:05 EST
The same command in the original comment does not print out the warnings and errors any more.

./db2ldif -s "dc=example,dc=com"
Exported ldif file: /var/lib/dirsrv/slapd-m0/ldif/m0-example-2008_12_03_134645.ldif
[03/Dec/2008:13:46:46 -0800] - Backend Instance(s): 
[03/Dec/2008:13:46:46 -0800] - 	userRoot
[03/Dec/2008:13:46:46 -0800] - 	subRoot
ldiffile: /var/lib/dirsrv/slapd-m0/ldif/m0-example-2008_12_03_134645.ldif
[03/Dec/2008:13:46:46 -0800] - export userRoot: Processed 1000 entries (19%).
[03/Dec/2008:13:46:46 -0800] - export userRoot: Processed 2000 entries (39%).
[03/Dec/2008:13:46:46 -0800] - export userRoot: Processed 3000 entries (59%).
[03/Dec/2008:13:46:46 -0800] - export userRoot: Processed 4000 entries (79%).
[03/Dec/2008:13:46:46 -0800] - export userRoot: Processed 5000 entries (99%).
[03/Dec/2008:13:46:46 -0800] - export userRoot: Processed 5007 entries (100%).
[03/Dec/2008:13:46:46 -0800] - All database threads now stopped
ldiffile: /var/lib/dirsrv/slapd-m0/ldif/m0-example-2008_12_03_134645.ldif
[03/Dec/2008:13:46:46 -0800] - export subRoot: Processed 17 entries (100%).
[03/Dec/2008:13:46:46 -0800] - All database threads now stopped

Ran tet export test: 100% pass
Export startup 	 100% (1/1) 	  	 
Export run 	100% (17/17) 	  	 
Export cleanup 	100% (1/1) 	  	 

Ran db2ldif with valgrind.  It showed no memory leaks.
Comment 4 Noriko Hosoi 2008-12-03 19:51:25 EST
Created attachment 325617 [details]
cvs commit message

Reviewed by Nathan (Thank you!!)

Checked in into CVS HEAD.
Comment 5 Jenny Galipeau 2009-03-31 09:07:55 EDT
fix verified DS 8.1 RHEL 5 - four sub suffixes

[root@jennyv2 slapd-jennyv2]# ldapsearch -x -h `hostname` -p 389 -D "cn=Directory Manager" -w Secret123 -b "dc=example, dc=com" "(objectclass=organizationalunit)"
# extended LDIF
#
# LDAPv3
# base <dc=example, dc=com> with scope subtree
# filter: (objectclass=organizationalunit)
# requesting: ALL
#

# Accounting, example.com
dn: ou=Accounting, dc=example,dc=com
objectClass: top
objectClass: organizationalUnit
ou: Accounting

# Product Development, example.com
dn: ou=Product Development, dc=example,dc=com
objectClass: top
objectClass: organizationalUnit
ou: Product Development

# Product Testing, example.com
dn: ou=Product Testing, dc=example,dc=com
objectClass: top
objectClass: organizationalUnit
ou: Product Testing

# Human Resources, example.com
dn: ou=Human Resources, dc=example,dc=com
objectClass: top
objectClass: organizationalUnit
ou: Human Resources

# Payroll, example.com
dn: ou=Payroll, dc=example,dc=com
objectClass: top
objectClass: organizationalUnit
ou: Payroll

# search result
search: 2
result: 0 Success

# numResponses: 6
# numEntries: 5

[root@jennyv2 slapd-jennyv2]# ./db2ldif -s "dc=example,dc=com"
Exported ldif file: /var/lib/dirsrv/slapd-jennyv2/ldif/jennyv2-example-2009_03_31_090029.ldif
[31/Mar/2009:09:00:30 -0400] - Backend Instance(s): 
[31/Mar/2009:09:00:30 -0400] -  testRoot
[31/Mar/2009:09:00:30 -0400] -  userRoot
ldiffile: /var/lib/dirsrv/slapd-jennyv2/ldif/jennyv2-example-2009_03_31_090029.ldif
[31/Mar/2009:09:00:30 -0400] - All database threads now stopped
ldiffile: /var/lib/dirsrv/slapd-jennyv2/ldif/jennyv2-example-2009_03_31_090029.ldif
[31/Mar/2009:09:00:30 -0400] - export userRoot: Processed 1000 entries (9%).
[31/Mar/2009:09:00:30 -0400] - export userRoot: Processed 2000 entries (19%).
[31/Mar/2009:09:00:30 -0400] - export userRoot: Processed 3000 entries (29%).
[31/Mar/2009:09:00:30 -0400] - export userRoot: Processed 4000 entries (39%).
[31/Mar/2009:09:00:30 -0400] - export userRoot: Processed 5000 entries (49%).
[31/Mar/2009:09:00:31 -0400] - export userRoot: Processed 6000 entries (59%).
[31/Mar/2009:09:00:31 -0400] - export userRoot: Processed 7000 entries (69%).
[31/Mar/2009:09:00:31 -0400] - export userRoot: Processed 8000 entries (79%).
[31/Mar/2009:09:00:31 -0400] - export userRoot: Processed 9000 entries (89%).
[31/Mar/2009:09:00:31 -0400] - export userRoot: Processed 10000 entries (99%).
[31/Mar/2009:09:00:31 -0400] - export userRoot: Processed 10006 entries (100%).
[31/Mar/2009:09:00:31 -0400] - All database threads now stopped
Comment 6 Chandrasekar Kannan 2009-04-29 19:08:19 EDT
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHEA-2009-0455.html

Note You need to log in before you can comment on or make changes to this bug.