Bug 705942

Summary: db2bak shell script creates ldif export instead of database backup
Product: [Retired] 389 Reporter: David Baird <dbaird>
Component: Command Line UtilitiesAssignee: Nathan Kinder <nkinder>
Status: CLOSED WORKSFORME QA Contact: Chandrasekar Kannan <ckannan>
Severity: medium Docs Contact:
Priority: medium    
Version: 1.2.8CC: benl, nhosoi, rmeggins
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2011-06-23 16:24:28 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On:    
Bug Blocks: 434915    

Description David Baird 2011-05-18 22:59:24 UTC
Description of problem: db2bak shell script produces an ldif file rather than creating a backup of the databases.  Additionally, if you provide an output directory as a parameter, it produces no output at all.


Version-Release number of selected component (if applicable):389 1.2.8.1


How reproducible:Every time


Steps to Reproduce:
1. /usr/lib64/dirsrv/slapd-xxx/db2bak
or
1. /usr/lib64/dirsrv/slapd-xxx/db2bak /var/lib/dirsrv/slapd-xxx/bak/daily 

  
Actual results:
In the first case (letting the script generate the directory path/name) it creates an ldif dump.  In the second case (providing a directory path/name) it produces no output.


Expected results:
Create a backup of the database files.

Additional info:
[root@server ~]/var/lib64/dirsrv/slapd-xxx/db2bak
Back up directory: /var/lib/dirsrv/slapd-xxx/bak/xxx-2011_05_10_09_46_43
[10/May/2011:09:46:43 +1200] 389-Directory/1.2.8.1 - debug level: backend (524288)
[root@server ~]# ls -al /var/lib/dirsrv/slapd-xxx/bak
total 152
drwxrwx--- 2 ldap ldap  4096 May 10 09:46 .
drwxrwx--- 6 ldap ldap  4096 Apr 21 08:30 ..
-rw------- 1 ldap ldap 67292 May 10 09:46 xxx-2011_05_10_094641.ldif 

[root@server ~]# /usr/lib64/dirsrv/slapd-xxx/db2bak /var/lib/dirsrv/slapd-xxx/bak/daily
Back up directory: /var/lib/dirsrv/slapd-xxx/bak/daily
[10/May/2011:10:06:52 +1200] 389-Directory/1.2.8.1 - debug level: backend (524288)
[root@server ~]# ls /var/lib/dirsrv/slapd-xxx/bak
xxx-2011_05_10_094641.ldif 

note the ldif file in the second example is the output from the first command

Comment 1 Noriko Hosoi 2011-05-23 22:58:54 UTC
I could not reproduce the problem using db2bak from 389-ds-base 1.2.8.3 

# /usr/lib64/dirsrv/slapd-ID/db2bak /var/lib/dirsrv/slapd-ID/daily
Back up directory: /var/lib/dirsrv/slapd-ID/daily
[..] 389-Directory/1.2.8.3 - debug level: backend (524288)
[..] - db2archive: /var/lib/dirsrv/slapd-ID/daily exists. Renaming to /var/lib/dirsrv/slapd-ID/daily.bak
[..] - Backing up file 1 (/var/lib/dirsrv/slapd-ID/daily/userRoot/cn.db4)
[..] - Copying /var/lib/dirsrv/slapd-ID/db/userRoot/cn.db4 to /var/lib/dirsrv/slapd-ID/daily/userRoot/cn.db4
[..] - Backing up file 2 (/var/lib/dirsrv/slapd-ID/daily/userRoot/parentid.db4)
[..] - Copying /var/lib/dirsrv/slapd-ID/db/userRoot/parentid.db4 to /var/lib/dirsrv/slapd-ID/daily/userRoot/parentid.db4
...
[..] - Backing up file 26 (/var/lib/dirsrv/slapd-ID/daily/log.0000000001)
[..] - Copying /var/lib/dirsrv/slapd-ID/db/log.0000000001 to /var/lib/dirsrv/slapd-ID/daily/log.0000000001
[..] - Backing up file 27 (/var/lib/dirsrv/slapd-ID/daily/DBVERSION)
[..] - Copying /var/lib/dirsrv/slapd-ID/db/DBVERSION to /var/lib/dirsrv/slapd-ID/daily/DBVERSION
[..] - All database threads now stopped

# ls /var/lib/dirsrv/slapd-ID/daily
DBVERSION       dse_instance.ldif  NetscapeRoot
dse_index.ldif  log.0000000001     userRoot

Could it be possible to upgrade your 389-ds-base to 1.2.8.3 and rerun db2bak?

Comment 2 Rich Megginson 2011-05-23 23:23:15 UTC
Could it be a problem with upgrade?  When you run setup-ds[-admin].pl -u, it attempts to regenerate the scripts such as db2bak.  Perhaps setup -u somehow corrupts db2bak?

Comment 3 Noriko Hosoi 2011-05-24 00:15:24 UTC
I'm also interested in how the script looks like.  Could you show us the last 2 lines of db2bak?  Here is mine.

# tail -2 /usr/lib64/dirsrv/slapd-ID/db2bak
echo "Back up directory: $bak_dir"
./ns-slapd db2archive -D /etc/dirsrv/slapd-ID -a $bak_dir -d $dlevel

Comment 4 David Baird 2011-05-24 00:38:38 UTC
# tail -n 2 /usr/lib64/dirsrv/slapd-ID/db2bak
echo "Back up directory: $bak_dir"
./ns-slapd db2archive -D /etc/dirsrv/slapd-ID -a $bak_dir -d $dlevel

Nothing different there.  

The only difference between the 1.2.8.2 script (not working) and the 1.2.7.5 script (which does work) is the following.

1.2.7.5
LD_LIBRARY_PATH=$prefix//usr/lib64/dirsrv:$prefix:$prefix/usr/lib64:$prefix
if [ -n "$prefix" ] ; then
    LD_LIBRARY_PATH="${LD_LIBRARY_PATH}:"
fi


1.2.8.2
libpath_add() {
    [ -z "$1" ] && return
    LD_LIBRARY_PATH=${LD_LIBRARY_PATH:+$LD_LIBRARY_PATH:}$1
}

libpath_add "$prefix/usr/lib64/dirsrv"
libpath_add "$prefix"
libpath_add "$prefix/usr/lib64"
libpath_add "$prefix"


Both our development and production servers were running 1.2.7.5.  There were problems, so the development server was re-installed with 1.2.8 Release Candidate 2 then upgraded to 1.2.8.2.  The 1.2.8.2 is now our production server, and the 1.2.7.5 is now our development server.  I would need management approval to upgrade our production server.  Should I just upgrade the development server to 1.2.8.3 and see if it works, or would it be advantageous to follow the same upgrade path and track the changes to the script?

Comment 5 Rich Megginson 2011-05-24 13:15:39 UTC
Does db2bak.pl also produce an ldif file, or just db2bak?

I don't think it is a problem with the script.  If the only difference between the 1.2.7.5 script that works and the 1.2.8.2 script that doesn't is the libpath_add stuff, then it must be something else.

On the one that is broken - you upgraded from 1.2.7.5 to 1.2.8.2?  Can you do 
rpm -qi 389-ds-base
and
rpm -qi 389-ds-base-libs
and
ls -al /usr/lib64/dirsrv
and
rpm -q --whatprovides /usr/lib64/dirsrv/libslapd.so
and
rpm -q --whatprovides /usr/lib64/dirsrv/libslapd.so.0.0.0

Comment 6 Rich Megginson 2011-05-24 14:55:36 UTC
also
rpm -q --whatprovides /usr/lib64/dirsrv/plugins/libback-ldbm.so
finally
rpm --verify 389-ds-base
and
rpm --verify 389-ds-base-libs

389-ds-base 1.2.8 split off the 389-ds-base-libs package containing libslapd from the 389-ds-base package. I just want to confirm that the upgrade worked correctly, and that you're not running a new 389-ds-base with old libraries.

Comment 7 David Baird 2011-05-24 21:33:33 UTC
Sorry Rich, my last comment should have stated 1.2.8.1 (not 1.2.8.2)

Was installed as 1.2.7.5, upgraded to 1.2.8rc2, then upgraded to 1.2.8.1

# rpm -qi 389-ds-base
Name        : 389-ds-base                  Relocations: (not relocatable)
Version     : 1.2.8.1                           Vendor: Fedora Project
Release     : 1.el5                         Build Date: Sat 09 Apr 2011 11:25:16 AM NZST
Install Date: Wed 13 Apr 2011 01:40:21 PM NZST      Build Host: x86-09.phx2.fedoraproject.org
Group       : System Environment/Daemons    Source RPM: 389-ds-base-1.2.8.1-1.el5.src.rpm
Size        : 4952802                          License: GPLv2 with exceptions
Signature   : DSA/SHA1, Sun 10 Apr 2011 05:13:34 AM NZST, Key ID 119cc036217521f6
Packager    : Fedora Project
URL         : http://port389.org/
Summary     : 389 Directory Server (base)
Description :
389 Directory Server is an LDAPv3 compliant server.  The base package includes
the LDAP server and command line utilities for server administration.

# rpm -qi 389-ds-base-libs
Name        : 389-ds-base-libs             Relocations: (not relocatable)
Version     : 1.2.8.1                           Vendor: Fedora Project
Release     : 1.el5                         Build Date: Sat 09 Apr 2011 11:25:16 AM NZST
Install Date: Wed 13 Apr 2011 01:40:21 PM NZST      Build Host: x86-09.phx2.fedoraproject.org
Group       : System Environment/Daemons    Source RPM: 389-ds-base-1.2.8.1-1.el5.src.rpm
Size        : 1016191                          License: GPLv2 with exceptions
Signature   : DSA/SHA1, Sun 10 Apr 2011 05:13:35 AM NZST, Key ID 119cc036217521f6
Packager    : Fedora Project
URL         : http://port389.org/
Summary     : Core libraries for 389 Directory Server
Description :
Core libraries for the 389 Directory Server base package.  These libraries
are used by the main package and the -devel package.  This allows the -devel
package to be installed with just the -libs package and without the main package.

# ls -al /usr/lib64/dirsrv/
total 1292
drwxr-xr-x  8 root root   4096 Mar 30 14:27 .
drwxr-xr-x 45 root root  20480 Apr 13 13:40 ..
drwxr-xr-x  2 root root   4096 Apr 14 02:02 cgi-bin
drwxr-xr-x  2 root root   4096 Jan 11 16:20 dsgw-cgi-bin
lrwxrwxrwx  1 root root     22 Apr 13 13:40 libns-dshttpd.so -> libns-dshttpd.so.0.0.0
lrwxrwxrwx  1 root root     22 Apr 13 13:40 libns-dshttpd.so.0 -> libns-dshttpd.so.0.0.0
-rwxr-xr-x  1 root root 266200 Apr  9 11:25 libns-dshttpd.so.0.0.0
lrwxrwxrwx  1 root root     17 Apr 13 13:40 libslapd.so.0 -> libslapd.so.0.0.0
-rwxr-xr-x  1 root root 995216 Apr  9 11:25 libslapd.so.0.0.0
drwxr-xr-x  2 root root   4096 Apr 13 13:40 modules
drwxr-xr-x  2 root root   4096 Apr 13 13:40 perl
drwxr-xr-x  2 root root   4096 Apr 13 13:40 plugins
drwxrwx---  2 ldap ldap   4096 Jan 11 16:41 slapd-cms

There is no libslapd.so  Did you mean libslapd.so.0? (which is just a symlink to libslapd.so.0.0.0)
# rpm -q --whatprovides /usr/lib64/dirsrv/libslapd.so.0.0.0 
389-ds-base-libs-1.2.8.1-1.el5.x86_64

# rpm -q --whatprovides /usr/lib64/dirsrv/plugins/libback-ldbm.so
389-ds-base-1.2.8.1-1.el5.x86_64

Both 389-ds-base and 389-ds-base-libs verify.

db2bak.pl works as expected.

In case it is relevant, LD_LIBRARY_PATH is not set on either server.

Comment 8 Rich Megginson 2011-05-26 21:56:18 UTC
(In reply to comment #7)
> db2bak.pl works as expected.

So db2bak.pl produces a binary backup, not an LDIF file?

Comment 9 David Baird 2011-05-29 23:22:51 UTC
Correct.

# ls -l /var/lib/dirsrv/slapd-ID/bak
total 532
-rw------- 1 ldap ldap 67292 May 24 00:10 ID-2011_05_24_001001.ldif
-rw------- 1 ldap ldap 67292 May 25 00:10 ID-2011_05_25_001001.ldif
-rw------- 1 ldap ldap 67292 May 26 00:10 ID-2011_05_26_001001.ldif
-rw------- 1 ldap ldap 67292 May 27 00:10 ID-2011_05_27_001001.ldif
-rw------- 1 ldap ldap 67292 May 28 00:10 ID-2011_05_28_001001.ldif
-rw------- 1 ldap ldap 67292 May 29 00:10 ID-2011_05_29_001001.ldif
-rw------- 1 ldap ldap 67292 May 30 00:10 ID-2011_05_30_001001.ldif
drwx------ 5 ldap ldap  4096 May 24 00:10 ID-2011_5_24_0_10_4
drwx------ 5 ldap ldap  4096 May 25 00:10 ID-2011_5_25_0_10_4
drwx------ 5 ldap ldap  4096 May 26 00:10 ID-2011_5_26_0_10_4
drwx------ 5 ldap ldap  4096 May 27 00:10 ID-2011_5_27_0_10_4
drwx------ 5 ldap ldap  4096 May 28 00:10 ID-2011_5_28_0_10_4
drwx------ 5 ldap ldap  4096 May 29 00:10 ID-2011_5_29_0_10_3
drwx------ 5 ldap ldap  4096 May 30 00:10 ID-2011_5_30_0_10_4

The ldifs in the above output are the output from saveconfig, which runs before db2bak.pl in our backup script.

Comment 12 Rich Megginson 2011-06-01 21:11:29 UTC
I have not yet been able to reproduce.  Steps
1) update a CentOS 5 64-bit system
2) install 389-ds-base-1.2.7.5 from koji - setup-ds.pl with the defaults - db2bak and db2bak.pl both work
3) yum update to 389-ds-base 1.2.8.3
4) db2bak and db2bak.pl both work - no LDIF file produced
5) db2ldif and db2ldif.pl both work - both produce an LDIF file

So I'm not really sure what to try next.  There is a remote possibility this was a bug fixed in 1.2.8.3, but I don't recall any bugs like this that would have been fixed between 1.2.8.1 and 1.2.8.3.

Comment 13 Rich Megginson 2011-06-23 16:24:28 UTC
can't reproduce