Bug 527848
| Summary: | make sure db upgrade to 4.7 and later works correctly | ||||||
|---|---|---|---|---|---|---|---|
| Product: | [Retired] 389 | Reporter: | Rich Megginson <rmeggins> | ||||
| Component: | Database - General | Assignee: | Noriko Hosoi <nhosoi> | ||||
| Status: | CLOSED CURRENTRELEASE | QA Contact: | Viktor Ashirov <vashirov> | ||||
| Severity: | high | Docs Contact: | |||||
| Priority: | high | ||||||
| Version: | 1.3.0 | CC: | amsharma, james_roman, jgalipea, Terry.Arends | ||||
| Target Milestone: | --- | ||||||
| Target Release: | --- | ||||||
| Hardware: | All | ||||||
| OS: | Linux | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | Doc Type: | Bug Fix | |||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2015-12-07 17:16:52 UTC | Type: | --- | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Bug Depends On: | |||||||
| Bug Blocks: | 495079 | ||||||
| Attachments: |
|
||||||
*** Bug 528480 has been marked as a duplicate of this bug. *** Created attachment 395003 [details]
git patch file
Change Description:
1. Replication Changelog
1-1. In the clean recover mode, transaction logs should not be removed.
1-2. When nsslapd-db-circular-logging is on (by default, it's on),
call log_archive function with DB_ARCH_REMOVE, which removes
log files that are no longer needed.
1-3. Call transaction checkpoint just before shutting down the server.
1-4. "From string" in the upbrade message had a flaw.
2. Backend dblayer
2-1. In checkpoint_threadmain, call log_archive with DB_ARCH_ABS,
which returns the absolute path of the transaction log files.
It eliminates the code which generates the absolute paths.
Files:
ldap/servers/plugins/replication/cl5_api.c
ldap/servers/slapd/back-ldbm/dblayer.c
Steps to verify:
1. build {redhat,389}-ds-base with an older version of db4.
(I built 389-ds-base with db4-4.3.29 on RHEL5.)
2. install it.
set up replication.
make some changes to create entries in the changelog.
3. build corresponding ds-base with a newer version of db4.
(I built 389-ds-base with db4-4.7.25 on RHEL5.)
4. install it.
restart the servers.
5. check errors log of the master(s).
The following message is supposed to be logged:
NSMMReplicationPlugin - changelog program - Upgrading from bdb/4.3/libreplication-plugin to bdb/4.7/libreplication-plugin is successfully done (/path/to/changelogdb)
And any other db errors should not be observed.
Reviewed by Rich (Thank you!!) Pushed to master: $ git merge work Updating ecf93e6..79e9309 Fast forward ldap/servers/plugins/replication/cl5_api.c | 38 +++++++----- ldap/servers/slapd/back-ldbm/dblayer.c | 91 ++++++++++++---------------- 2 files changed, 61 insertions(+), 68 deletions(-) $ git push Counting objects: 19, done. Delta compression using 4 threads. Compressing objects: 100% (10/10), done. Writing objects: 100% (10/10), 2.32 KiB, done. Total 10 (delta 8), reused 0 (delta 0) To ssh://git.fedorahosted.org/git/389/ds.git ecf93e6..79e9309 master -> master Pushed to Directory_Server_8_2_Branch: $ git cherry-pick 79e93093e031bdd5a0bc047baf767644832eb562 $ git push origin ds82-local:Directory_Server_8_2_Branch Counting objects: 19, done. Delta compression using 4 threads. Compressing objects: 100% (10/10), done. Writing objects: 100% (10/10), 46.51 KiB, done. Total 10 (delta 7), reused 1 (delta 0) To ssh://git.fedorahosted.org/git/389/ds.git 67ba596..be07459 ds82-local -> Directory_Server_8_2_Branch What about upgrade from RHEL5 to RHEL6? Can you upgrade a machine from RHEL5 to RHEL6? Even if not, the customer could just copy their files and data from the old RHEL5 machine to the new RHEL6 machine and run setup-ds-admin.pl -u. We should test this scenario. yes, I have tested setup-ds-admin.pl -u for upgrading the ds builds. |
berkeley db 4.7 is being used on several platforms now. There have been problems reported upgrading a system. [05/Oct/2009:01:46:37 -0400] NSMMReplicationPlugin - changelog program - Upgrading from bdb/4 to bdb/4.7/libreplication-plugin is successfully done (/var/lib/dirsrv/slapd-REALM-COM/cldb) [05/Oct/2009:01:46:37 -0400] NSMMReplicationPlugin - changelog program - _cl5NewDBFile: db_open failed; db error - 22 Invalid argument [05/Oct/2009:01:46:37 -0400] NSMMReplicationPlugin - changelog program - cl5Open: failed to open changelog [05/Oct/2009:01:46:37 -0400] NSMMReplicationPlugin - changelog program - changelog5_init: failed to start changelog at /var/lib/dirsrv/slapd-REALM-COM/cldb and [05/Oct/2009:01:46:34 -0400] - Clean up db environment and start from archive. [05/Oct/2009:01:46:34 -0400] - libdb: Program version 4.7 doesn't match environment version 4.6 [05/Oct/2009:01:46:34 -0400] - libdb: Program version 4.7 doesn't match environment version 4.6 From bdb 4.3 -> 4.6 or 4.7, the transaction log format is changed, but not the btree/recno database formats are. So, the procedure we should follow is this: If the application has a Berkeley DB transactional environment, and the log files need upgrading but the databases do not, the application may be installed in the field using the following steps: 1. Shut down the old version of the application. 2. Still using the old version of Berkeley DB, run recovery on the database environment using the DB_ENV->open() utility. 3. If you used the DB_ENV->open() method to run recovery, make sure that the Berkeley DB environment is removed using the DB_ENV->remove() method or an appropriate system utility. 4. Archive the database environment for catastrophic recovery. See Database and log file archival for more information. 5. Recompile and install the new version of the application. 6. Force a checkpoint using the DB_ENV->txn_checkpoint() method or the db_checkpoint utility. If you use the db_checkpoint utility, make sure to use the new version of the utility; that is, the version that came with the release of Berkeley DB to which you are upgrading. 7. Remove unnecessary log files from the environment using the -d option on the db_archive utility, or from an application which calls the DB_ENV->log_archive() method with the DB_ARCH_REMOVE flag. Note that if you are upgrading a replicated application, then you should not perform this step until all of the replication sites have been upgraded to the current release level. If you run this site before all your sites are upgraded, then errors can occur in your replication activities because important version information might be lost. 8. Restart the application. Looking at the error log, replication changelog upgrade failed. Most likely, the upgrade function _cl5Upgrade4_4 is incomplete... :( I think we can do all of the above steps programatically, in the dblayer code and in the cl5_api code.