Bug 527848 - make sure db upgrade to 4.7 and later works correctly
Summary: make sure db upgrade to 4.7 and later works correctly
Alias: None
Product: 389
Classification: Retired
Component: Database - General
Version: 1.3.0
Hardware: All
OS: Linux
Target Milestone: ---
Assignee: Noriko Hosoi
QA Contact: Viktor Ashirov
: 528480 (view as bug list)
Depends On:
Blocks: 389_1.3.0
TreeView+ depends on / blocked
Reported: 2009-10-07 22:32 UTC by Rich Megginson
Modified: 2015-12-07 17:16 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Last Closed: 2015-12-07 17:16:52 UTC

Attachments (Terms of Use)
git patch file (10.48 KB, patch)
2010-02-18 22:47 UTC, Noriko Hosoi
nhosoi: review?
rmeggins: review+
Details | Diff

Description Rich Megginson 2009-10-07 22:32:12 UTC
berkeley db 4.7 is being used on several platforms now.  There have been problems reported upgrading a system.

[05/Oct/2009:01:46:37 -0400] NSMMReplicationPlugin - changelog program - Upgrading from bdb/4 to bdb/4.7/libreplication-plugin is successfully done (/var/lib/dirsrv/slapd-REALM-COM/cldb)
[05/Oct/2009:01:46:37 -0400] NSMMReplicationPlugin - changelog program - _cl5NewDBFile: db_open failed; db error - 22 Invalid argument
[05/Oct/2009:01:46:37 -0400] NSMMReplicationPlugin - changelog program - cl5Open: failed to open changelog
[05/Oct/2009:01:46:37 -0400] NSMMReplicationPlugin - changelog program - changelog5_init: failed to start changelog at /var/lib/dirsrv/slapd-REALM-COM/cldb


[05/Oct/2009:01:46:34 -0400] - Clean up db environment and start from archive.
[05/Oct/2009:01:46:34 -0400] - libdb: Program version 4.7 doesn't match environment version 4.6
[05/Oct/2009:01:46:34 -0400] - libdb: Program version 4.7 doesn't match environment version 4.6

From bdb 4.3 -> 4.6 or 4.7, the transaction log format is changed, but not the btree/recno database formats are.  So, the procedure we should follow is this:

    If the application has a Berkeley DB transactional environment, and the log files need upgrading but the databases do not, the application may be installed in the field using the following steps:

         1. Shut down the old version of the application.
         2. Still using the old version of Berkeley DB, run recovery on the database environment using the DB_ENV->open() utility.
         3. If you used the DB_ENV->open() method to run recovery, make sure that the Berkeley DB environment is removed using the DB_ENV->remove() method or an appropriate system utility.
         4. Archive the database environment for catastrophic recovery. See Database and log file archival for more information.
         5. Recompile and install the new version of the application.
         6. Force a checkpoint using the DB_ENV->txn_checkpoint() method or the db_checkpoint utility. If you use the db_checkpoint utility, make sure to use the new version of the utility; that is, the version that came with the release of Berkeley DB to which you are upgrading.
         7. Remove unnecessary log files from the environment using the -d option on the db_archive utility, or from an application which calls the DB_ENV->log_archive() method with the DB_ARCH_REMOVE flag.

            Note that if you are upgrading a replicated application, then you should not perform this step until all of the replication sites have been upgraded to the current release level. If you run this site before all your sites are upgraded, then errors can occur in your replication activities because important version information might be lost.
         8. Restart the application.

Looking at the error log, replication changelog upgrade failed.  Most likely, the upgrade function _cl5Upgrade4_4 is incomplete... :(

I think we can do all of the above steps programatically, in the dblayer code and in the cl5_api code.

Comment 1 Rich Megginson 2009-10-12 14:36:58 UTC
*** Bug 528480 has been marked as a duplicate of this bug. ***

Comment 2 Noriko Hosoi 2010-02-18 22:47:31 UTC
Created attachment 395003 [details]
git patch file

Change Description:
1. Replication Changelog
1-1. In the clean recover mode, transaction logs should not be removed.
1-2. When nsslapd-db-circular-logging is on (by default, it's on),
     call log_archive function with DB_ARCH_REMOVE, which removes
     log files that are no longer needed.
1-3. Call transaction checkpoint just before shutting down the server.
1-4. "From string" in the upbrade message had a flaw.
2. Backend dblayer
2-1. In checkpoint_threadmain, call log_archive with DB_ARCH_ABS,
     which returns the absolute path of the transaction log files.
     It eliminates the code which generates the absolute paths.


Comment 3 Noriko Hosoi 2010-02-18 23:03:27 UTC
Steps to verify:
1. build {redhat,389}-ds-base with an older version of db4.
   (I built 389-ds-base with db4-4.3.29 on RHEL5.)
2. install it.
   set up replication.
   make some changes to create entries in the changelog.
3. build corresponding ds-base with a newer version of db4.
   (I built 389-ds-base with db4-4.7.25 on RHEL5.)
4. install it.
   restart the servers.
5. check errors log of the master(s).  
   The following message is supposed to be logged:
   NSMMReplicationPlugin - changelog program - Upgrading from bdb/4.3/libreplication-plugin to bdb/4.7/libreplication-plugin is successfully done (/path/to/changelogdb)
   And any other db errors should not be observed.

Comment 4 Noriko Hosoi 2010-02-18 23:55:47 UTC
Reviewed by Rich (Thank you!!)

Pushed to master:
$ git merge work
Updating ecf93e6..79e9309
Fast forward
 ldap/servers/plugins/replication/cl5_api.c |   38 +++++++-----
 ldap/servers/slapd/back-ldbm/dblayer.c     |   91 ++++++++++++----------------
 2 files changed, 61 insertions(+), 68 deletions(-)
$ git push
Counting objects: 19, done.
Delta compression using 4 threads.
Compressing objects: 100% (10/10), done.
Writing objects: 100% (10/10), 2.32 KiB, done.
Total 10 (delta 8), reused 0 (delta 0)
To ssh://git.fedorahosted.org/git/389/ds.git
   ecf93e6..79e9309  master -> master

Pushed to Directory_Server_8_2_Branch:
$ git cherry-pick 79e93093e031bdd5a0bc047baf767644832eb562
$ git push origin ds82-local:Directory_Server_8_2_Branch
Counting objects: 19, done.
Delta compression using 4 threads.
Compressing objects: 100% (10/10), done.
Writing objects: 100% (10/10), 46.51 KiB, done.
Total 10 (delta 7), reused 1 (delta 0)
To ssh://git.fedorahosted.org/git/389/ds.git
   67ba596..be07459  ds82-local -> Directory_Server_8_2_Branch

Comment 13 Rich Megginson 2011-05-19 15:03:20 UTC
What about upgrade from RHEL5 to RHEL6?
Can you upgrade a machine from RHEL5 to RHEL6?
Even if not, the customer could just copy their files and data from the old RHEL5 machine to the new RHEL6 machine and run setup-ds-admin.pl -u.  We should test this scenario.

Comment 14 Amita Sharma 2011-05-27 12:56:45 UTC
yes, I have tested setup-ds-admin.pl -u for upgrading the ds builds.

Note You need to log in before you can comment on or make changes to this bug.