Bug 462922 - Import of data does not record timestamps...
Summary: Import of data does not record timestamps...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: 389
Classification: Retired
Component: Database - Import/Export
Version: 1.1.2
Hardware: All
OS: Linux
medium
medium
Target Milestone: ---
Assignee: Noriko Hosoi
QA Contact: Chandrasekar Kannan
URL:
Whiteboard:
Depends On:
Blocks: 249650 FDS1.2.0
TreeView+ depends on / blocked
 
Reported: 2008-09-19 18:26 UTC by reinhard nappert
Modified: 2015-01-04 23:34 UTC (History)
5 users (show)

Fixed In Version: 8.1
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2009-04-29 23:06:34 UTC
Embargoed:


Attachments (Terms of Use)
cvs diff ldap/servers/slapd/back-ldbm/import-threads.c (1.80 KB, patch)
2008-11-04 07:28 UTC, Noriko Hosoi
no flags Details | Diff
cvs diff ldap/servers/slapd/back-ldbm/import-threads.c (1.85 KB, patch)
2008-11-04 16:05 UTC, Noriko Hosoi
no flags Details | Diff
cvs commit message (793 bytes, text/plain)
2008-11-04 23:14 UTC, Noriko Hosoi
no flags Details

Description reinhard nappert 2008-09-19 18:26:16 UTC
During the setup, I import a ldif file, which contains just the suffix object (with the InstallLdifFile option during silent setup). 

The createTimestamp and modifyTimestamp operational attributes were not generated during this import. This eventually leads into a WARNING message, when I create vlv indexes for the timestamp attributes.


See the following thread from the mailing list.
https://www.redhat.com/archives/fedora-directory-users/2008-September/msg00084.html

Comment 1 Noriko Hosoi 2008-11-04 02:29:43 UTC
Comment by Rich in the mailing list: createTimestamp should be present, unless you have nsslapd-lastmod turned off.

Comment 2 Noriko Hosoi 2008-11-04 07:28:29 UTC
Created attachment 322395 [details]
cvs diff ldap/servers/slapd/back-ldbm/import-threads.c

File: ldapserver/ldap/servers/slapd/back-ldbm/import-threads.c

Description: add a static function import_add_created_attrs to add CreatorsName, ModifiersName, CreateTimestamp, ModifyTimestamp to each entry.

Sample entry:
 dn: dc=example,dc=com
 objectClass: top
 [...]
 nsUniqueId: ce7e5ea2-aa4011dd-b970c0aa-ead44d23
 creatorsName:
 modifiersName:
 createTimestamp: 20081104071841Z
 modifyTimestamp: 20081104071841Z

Comment 3 Noriko Hosoi 2008-11-04 16:05:37 UTC
Created attachment 322441 [details]
cvs diff ldap/servers/slapd/back-ldbm/import-threads.c

In addition to the previous comment, added a check if nsslapd-lastmod is on or off.  It adds the created info only if the lastmod is on.

Comment 4 Noriko Hosoi 2008-11-04 23:14:52 UTC
Created attachment 322498 [details]
cvs commit message

Reviewed by Rich and Nathan (Thank you!!)

Checked in into CVS HEAD.

Comment 5 Jenny Severance 2009-04-01 17:37:12 UTC
Is there anyway you can attach the ldif file used and silent install ini file for fix verication?

Comment 6 Rich Megginson 2009-04-01 17:42:35 UTC
(In reply to comment #5)
> Is there anyway you can attach the ldif file used and silent install ini file
> for fix verication?  

You should just be able to do an import (ldif2db) , then try ldapsearch with an attribute list of createTimestamp creatorsName modifyTimestamp modifiersName - you should see these attributes
then try db2ldif to export the database - you should see these attributes in the ldif file too

Comment 7 Noriko Hosoi 2009-04-01 17:51:11 UTC
(In reply to comment #5)
> Is there anyway you can attach the ldif file used and silent install ini file
> for fix verication?  

I don't think the install/configuration is needed to verify this bug. 

1. without nsslapd-lastmod
run ldif2db and check each entry contains the attribute value pairs of createTimestamp and modifyTimestamp.

2. set "nsslapd-lastmod: off" in cn=config
run ldif2db and check each entry does not contain the attribute value pairs of createTimestamp and modifyTimestamp.

Comment 8 Yi Zhang 2009-04-01 21:49:53 UTC
verified on rhel 5 i386 host, test result : pass
------------------------------------

Test script and running result
==============================
[yi@mv64j-vm sbin]$ cat ds.addsuffix 
#!/bin/sh

LDAPMODIFY=/usr/lib*/mozldap*/ldapmodify
CN="CN=directory manager"
PW="redhat123"

HOST=$1
if [ -z $HOST ]; then
 echo "usage: ds.addsuffix < ds host>"
 echo "       example: ds.addsuffix mv32a-vm.idm.lab.bos.redhat.com"
 exit
fi

echo "AddSuffix for $HOST starts..."
echo "   [1/2] add the suffix (1) : add into ldbm database,cn=plugin,cn=config"
$LDAPMODIFY -v -h $HOST -D "$CN" -w "$PW" -c <<-EOF
dn: cn=test.com,cn=ldbm database,cn=plugins,cn=config
changetype: add
objectclass: top
objectclass: extensibleObject
objectclass: nsBackendInstance
cn: test.com
nsslapd-suffix: o=test.com
nsslapd-cachesize: -1
nsslapd-cachememsize: 10485760
EOF

echo "    [2/2] add the suffix (2) : add into cn=mapping
tree,cn=plugin,cn=config"

$LDAPMODIFY -v -h $HOST -D "$CN" -w "$PW" -c <<-EOF
dn: cn="o=test.com",cn=mapping tree,cn=config
changetype: add
objectclass: top
objectclass: extensibleObject
objectclass: nsMappingTree
cn: "o=test.com"
nsslapd-state: backend
nsslapd-backend: test.com
EOF

echo "ADDSuffix Done"
echo ""

-------------------running result-----------
[yi@mv64j-vm sbin]$ ./ds.addsuffix mv32a-vm
AddSuffix for mv32a-vm starts...
   [1/2] add the suffix (1) : add into ldbm database,cn=plugin,cn=config
ldapmodify: started Wed Apr  1 08:27:37 2009

ldap_init( mv32a-vm, 389 )
add objectclass:
 top
 extensibleObject
 nsBackendInstance
add cn:
 test.com
add nsslapd-suffix:
 o=test.com
add nsslapd-cachesize:
 -1
add nsslapd-cachememsize:
 10485760
adding new entry cn=test.com,cn=ldbm database,cn=plugins,cn=config
modify complete

    [2/2] add the suffix (2) : add into cn=mapping tree,cn=plugin,cn=config
ldapmodify: started Wed Apr  1 08:27:37 2009

ldap_init( mv32a-vm, 389 )
add objectclass:
 top
 extensibleObject
 nsMappingTree
add cn:
 "o=test.com"
add nsslapd-state:
 backend
add nsslapd-backend:
 test.com
adding new entry cn="o=test.com",cn=mapping tree,cn=config
modify complete

ADDSuffix Done

==============================

Test verification :
check the dse.ldif file of target ds server /etc/dirsrv/slapd-mv32a-vm
-------------------------
dn: cn="o=test.com",cn=mapping tree,cn=config
objectClass: top
objectClass: extensibleObject
objectClass: nsMappingTree
cn: "o=test.com"
nsslapd-state: backend
nsslapd-backend: test.com
creatorsName: cn=directory manager
modifiersName: cn=directory manager
createTimestamp: 20090401213736Z
modifyTimestamp: 20090401213736Z

-----------------------------
dn: cn=test.com,cn=ldbm database,cn=plugins,cn=config
objectClass: top
objectClass: extensibleObject
objectClass: nsBackendInstance
cn: test.com
creatorsName: cn=directory manager
modifiersName: cn=directory manager
createTimestamp: 20090401213736Z
modifyTimestamp: 20090401213736Z
numSubordinates: 4
nsslapd-suffix: o=test.com
nsslapd-cachesize: -1
nsslapd-cachememsize: 10485760
nsslapd-readonly: off
nsslapd-require-index: off
nsslapd-directory: /var/lib/dirsrv/slapd-mv32a-vm/db/test.com


==>  I don't know Jenny is also testing on this bug. Rich/Noriko please review my test above and reopen this bug if my test is not valid. For now, i will close this bug as "verified"

Comment 9 Yi Zhang 2009-04-01 22:08:05 UTC
well, the test above is not valid. I am doing it again now.

Comment 10 Yi Zhang 2009-04-01 22:22:16 UTC
New test performed, test result: pass

Test steps:
1. stop slapd
2. create 1000 entry with dbgen.pl (a tool shipped with our produce)
    dbgen.pl -s "dc=idm,dc=lab,dc=bos,dc=redhat,dc=com" -o 1k.ldif -n 1000
3. import the data file (1k.ldif)
    ldif2db -n userRoot -i /root/1k.ldif
4. start slapd
5. stop slapd again
6. export the userRoot data 
    db2ldif -n userRoot -a /tmp/out.ldif
7. verify out.ldif has timestamp

==> result : pass.

sample test data and final out.ldif can be find below:



the 1k.ldif has something like this:
-------------------------------------------------------
dn: dc=idm,dc=lab,dc=bos,dc=redhat,dc=com
objectClass: top
objectClass: domain
dc: idm
aci: (target=ldap:///dc=idm,dc=lab,dc=bos,dc=redhat,dc=com)(targetattr=*)(version 3.0; acl "acl1"; allow(write) userdn = "ldap:///self";)
aci: (target=ldap:///dc=idm,dc=lab,dc=bos,dc=redhat,dc=com)(targetattr=*)(version 3.0; acl "acl2"; allow(write) groupdn = "ldap:///cn=Directory Administrators, dc=idm,dc=lab,dc=bos,dc=redhat,dc=com";)
aci: (target=ldap:///dc=idm,dc=lab,dc=bos,dc=redhat,dc=com)(targetattr=*)(version 3.0; acl "acl3"; allow(read, search, compare) userdn = "ldap:///anyone";)

dn: ou=Accounting, dc=idm,dc=lab,dc=bos,dc=redhat,dc=com
objectClass: top
objectClass: organizationalUnit
ou: Accounting

dn: ou=Product Development, dc=idm,dc=lab,dc=bos,dc=redhat,dc=com
objectClass: top
objectClass: organizationalUnit
ou: Product Development

dn: ou=Product Testing, dc=idm,dc=lab,dc=bos,dc=redhat,dc=com
objectClass: top
objectClass: organizationalUnit
ou: Product Testing

...
...
...
--------------------------
which does not contain the timestamp
the db2ldif result, which is out.ldif has timestamp

=========== out.ldif =============
version: 1

# entry-id: 1
dn: dc=idm,dc=lab,dc=bos,dc=redhat,dc=com
objectClass: top
objectClass: domain
dc: idm
aci:: KHRhcmdldD1sZGFwOi8vL2RjPWlkbSxkYz1sYWIsZGM9Ym9zLGRjPXJlZGhhdCxkYz1jb20
 pKHRhcmdldGF0dHI9KikodmVyc2lvbiAzLjA7IGFjbCAiYWNsMSI7IGFsbG93KHdyaXRlKSB1c2V
 yZG4gPSAibGRhcDovLy9zZWxmIjspIA==
aci: (target=ldap:///dc=idm,dc=lab,dc=bos,dc=redhat,dc=com)(targetattr=*)(vers
 ion 3.0; acl "acl2"; allow(write) groupdn = "ldap:///cn=Directory Administra
 tors, dc=idm,dc=lab,dc=bos,dc=redhat,dc=com";)
aci: (target=ldap:///dc=idm,dc=lab,dc=bos,dc=redhat,dc=com)(targetattr=*)(vers
 ion 3.0; acl "acl3"; allow(read, search, compare) userdn = "ldap:///anyone";
 )
nsUniqueId: d5cf68e4-1dd111b2-967ba382-040b0000
creatorsName:
modifiersName:
createTimestamp: 20090401220957Z
modifyTimestamp: 20090401220957Z

# entry-id: 2
dn: ou=Accounting, dc=idm,dc=lab,dc=bos,dc=redhat,dc=com
objectClass: top
objectClass: organizationalUnit
ou: Accounting
nsUniqueId: d5cf68e5-1dd111b2-967ba382-040b0000
creatorsName:
modifiersName:
createTimestamp: 20090401220957Z
modifyTimestamp: 20090401220957Z

# entry-id: 3
dn: ou=Product Development, dc=idm,dc=lab,dc=bos,dc=redhat,dc=com
objectClass: top
objectClass: organizationalUnit
ou: Product Development
nsUniqueId: d5cf68e6-1dd111b2-967ba382-040b0000
creatorsName:
modifiersName:
createTimestamp: 20090401220957Z
modifyTimestamp: 20090401220957Z

# entry-id: 4

Comment 11 Yi Zhang 2009-04-01 22:25:10 UTC
Noriko proved it. bug closed. :-)

thanks!

Comment 12 Chandrasekar Kannan 2009-04-29 23:06:34 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHEA-2009-0455.html


Note You need to log in before you can comment on or make changes to this bug.