Bug 194531

Summary:

db2bak is too noisy

Product:

[Retired] 389

Reporter:

Chris St. Pierre <cstpierr>

Component:

Database - Import/Export

Assignee:

Noriko Hosoi <nhosoi>

Status:

CLOSED CURRENTRELEASE

QA Contact:

Viktor Ashirov <vashirov>

Severity:

medium

Docs Contact:

Priority:

medium

Version:

1.0

CC:

amsharma, rmeggins

Target Milestone:

---

Target Release:

---

Hardware:

All

OS:

Linux

Whiteboard:

Fixed In Version:

Doc Type:

Bug Fix

Doc Text:

Story Points:

---

Clone Of:

Environment:

Last Closed:

2015-12-07 16:37:15 UTC

Type:

---

Regression:

---

Mount Type:

---

Documentation:

---

CRM:

Verified Versions:

Category:

---

oVirt Team:

---

RHEL 7.3 requirements from Atomic Host:

Cloudforms Team:

---

Target Upstream Version:

Embargoed:

Bug Depends On:

Bug Blocks:

576869, 639035

Attachments:

Description	Flags
git patch file (master)	none
git patch file (master)	nhosoi: review?, rmeggins: review+

Description Chris St. Pierre 2006-06-08 19:12:28 UTC

db2bak dumps all of its information to stderr, and it dumps a lot of it.  This 
isn't very nice for cron'ing a snapshot.  It would be awesome if it could dump 
its information to stdout, and have either a -q flag to silence it (except for 
errors), or a -v flag and be silent by default.

Comment 1 Chris St. Pierre 2006-06-08 20:04:23 UTC

*** Bug 194534 has been marked as a duplicate of this bug. ***

Comment 2 Noriko Hosoi 2010-08-05 21:03:11 UTC

Created attachment 436980 [details]
git patch file (master)

Description:
Added -v option to db2bak and bak2db and moved the Backing up/
Restoring logs to the verbose mode output.  To implement the
backend verbose mode, log level SLAPI_LOG_BACKLDBM has been
introduced.
  Usage: db2bak [archivedir] [-v]
  Usage: bak2db archivedir [-n backendname] [-v]

Files:
 ldap/admin/src/scripts/template-bak2db.in
 ldap/admin/src/scripts/template-db2bak.in
 ldap/include/ldaplog.h
 ldap/servers/slapd/back-ldbm/archive.c
 ldap/servers/slapd/back-ldbm/dblayer.c
 ldap/servers/slapd/log.c
 ldap/servers/slapd/main.c
 ldap/servers/slapd/slapi-plugin.h

Comment 3 Noriko Hosoi 2010-08-05 21:10:57 UTC

Test Usage:
# ./db2bak
Back up directory: /var/lib/dirsrv/slapd-ID/bak/ID-DATE
[..] - All database threads now stopped

# ./db2bak -v
Back up directory: /var/lib/dirsrv/slapd-ID/bak/ID-DATE
[..] 389-Directory/1.2.7.a1.git2e188fa - debug level: backend (524288)
[..] - Backing up file 1 (/var/lib/dirsrv/slapd-ID/bak/ID-DATE/userRoot/givenName.db4)
[..] - Backing up file 2 (/var/lib/dirsrv/slapd-ID/bak/ID-DATE/userRoot/uid.db4)
       ...
[..] - All database threads now stopped

# ./db2bak /tmp/bak
Back up directory: /tmp/bak
[..] - All database threads now stopped

# ./db2bak /tmp/bak -v
Back up directory: /tmp/bak
[..] 389-Directory/1.2.7.a1.git2e188fa - debug level: backend (524288)
[..] - db2archive: /tmp/bak exists. Renaming to /tmp/bak.bak
[..] - Backing up file 1 (/tmp/bak/userRoot/givenName.db4)
[..] - Backing up file 2 (/tmp/bak/userRoot/uid.db4)
       ...
[..] - All database threads now stopped

# ./bak2db /tmp/bak
[..] - All database threads now stopped

# ./bak2db /tmp/bak -v
[..] 389-Directory/1.2.7.a1.git2e188fa - debug level: backend (524288)
[..] - Deleting log file: (/var/lib/dirsrv/slapd-ID/db/log.0000000003)
[..] - Restoring file 1 (/var/lib/dirsrv/slapd-ID/db/NetscapeRoot/givenName.db4)
       ...
[..] - All database threads now stopped

Comment 4 Noriko Hosoi 2010-08-09 23:31:31 UTC

Created attachment 437730 [details]
git patch file (master)

Thanks to Nathan for his comments to the previous proposal.  Based upon the discussion with him, changed the spec as follows:
1) leave the db2bak/bak2db's behaviour intact (verbose by default)
2) introduced a suppress option "-q" not to send the verbose messages to the standard error, but just to the errors log.

Description: Introduced "-q" option to suppress the backing up/
restoring message for each db file.
  Usage: db2bak [archivedir] [-q] [-h]
  Usage: bak2db archivedir [-n backendname] [-q] | [-h]
If "-q" is specified, the verbose messages are not prited to
the standard error, but just to the errors log.

In addition, new log level SLAPI_LOG_BACKLDBM/LDAP_DEBUG_BACKLDBM
has been introduced.

Comment 5 Noriko Hosoi 2010-08-09 23:43:51 UTC

Usages:
# ./db2bak -h
Usage: db2bak [archivedir] [-q] [-h]

# ./bak2db -h
Usage: bak2db archivedir [-n backendname] [-q] | [-h]

# ./db2bak
Back up directory: /var/lib/dirsrv/slapd-ID/bak/ID-DATE
[..] 389-Directory/1.2.7.a1.git8fa94a3 - debug level: backend (524288)
[..] - Backing up file 1 (/var/lib/dirsrv/slapd-ID/bak/ID-DATE/userRoot/givenName.db4)
[..] ...
[..] - All database threads now stopped

# ./db2bak -q
Back up directory: /var/lib/dirsrv/slapd-ID/bak/ID-DATE
# tail /var/log/dirsrv/slapd-ID/errors
[..] - Backing up file 1 (/var/lib/dirsrv/slapd-ID/bak/ID-DATE/userRoot/givenName.db4)
[..] - Backing up file 2 (/var/lib/dirsrv/slapd-ID/bak/ID-DATE/userRoot/uid.db4)
[..] ...
[..] - All database threads now stopped

# ./bak2db /var/lib/dirsrv/slapd-ID/bak/ID-DATE
[..] 389-Directory/1.2.7.a1.git8fa94a3 - debug level: backend (524288)
[..] - Deleting log file: (/var/lib/dirsrv/slapd-ID/db/log.0000000001)
[..] - Restoring file 1 (/var/lib/dirsrv/slapd-ID/db/log.0000000001)
[..] ...
[..] - All database threads now stopped

# ./bak2db /var/lib/dirsrv/slapd-ID/bak/ID-DATE -q
# tail /var/log/dirsrv/slapd-ID/errors
[..] - Deleting log file: (/var/lib/dirsrv/slapd-ID/db/log.0000000001)
[..] - Restoring file 1 (/var/lib/dirsrv/slapd-ID/db/log.0000000001)
[..] - Restoring file 2 (/var/lib/dirsrv/slapd-ID/db/userRoot/givenName.db4)
[..] ...
[..] - All database threads now stopped

Comment 6 Noriko Hosoi 2010-08-10 16:35:28 UTC

Reviewed by Rich (Thank you!!).

Pushed to master

$ git merge 194531
Updating 8fa94a3..9f07f9d
Fast-forward
 ldap/admin/src/scripts/template-bak2db.in    |   34 +++++++++++++++-------
 ldap/admin/src/scripts/template-db2bak.in    |   38 ++++++++++++++++++++++---
 ldap/include/ldaplog.h                       |    1 +
 ldap/servers/slapd/back-ldbm/ldbm_entryrdn.c |   34 +++++++++++-----------
 ldap/servers/slapd/log.c                     |    1 +
 ldap/servers/slapd/main.c                    |    8 +++++
 ldap/servers/slapd/slapi-plugin.h            |    3 +-
 7 files changed, 85 insertions(+), 34 deletions(-)

$ git push
Counting objects: 33, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (16/16), done.
Writing objects: 100% (17/17), 2.65 KiB, done.
Total 17 (delta 13), reused 0 (delta 0)
To ssh://git.fedorahosted.org/git/389/ds.git
   8fa94a3..9f07f9d  master -> master

Comment 8 Noriko Hosoi 2011-06-16 16:37:57 UTC

It works just fine for me...  Again, you'd better give me your test env.

# ./db2bak -h
Usage: db2bak [archivedir] [-q] [-h]
# ./db2bak -q
Back up directory: /var/lib/dirsrv/slapd-test/bak/test-2011_06_16_09_29_45
# ./db2bak
Back up directory: /var/lib/dirsrv/slapd-test/bak/test-2011_06_16_09_29_38
[16/Jun/2011:09:29:38 -0700] 389-Directory/1.2.9.a2.git9cd0752 - debug level: backend (524288)
[16/Jun/2011:09:29:38 -0700] - Backing up file 1 (/var/lib/dirsrv/slapd-test/bak/test-2011_06_16_09_29_38/userRoot/objectclass.db4)
[16/Jun/2011:09:29:38 -0700] - Copying /var/lib/dirsrv/slapd-test/db/userRoot/objectclass.db4 to /var/lib/dirsrv/slapd-test/bak/test-2011_06_16_09_29_38/userRoot/objectclass.db4
[16/Jun/2011:09:29:38 -0700] - Backing up file 2 (/var/lib/dirsrv/slapd-test/bak/test-2011_06_16_09_29_38/userRoot/id2entry.db4)
[16/Jun/2011:09:29:38 -0700] - Copying /var/lib/dirsrv/slapd-test/db/userRoot/id2entry.db4 to /var/lib/dirsrv/slapd-test/bak/test-2011_06_16_09_29_38/userRoot/id2entry.db4
[16/Jun/2011:09:29:38 -0700] - Backing up file 3 (/var/lib/dirsrv/slapd-test/bak/test-2011_06_16_09_29_38/userRoot/uniquemember.db4)
[16/Jun/2011:09:29:38 -0700] - Copying /var/lib/dirsrv/slapd-test/db/userRoot/uniquemember.db4 to /var/lib/dirsrv/slapd-test/bak/test-2011_06_16_09_29_38/userRoot/uniquemember.db4
[16/Jun/2011:09:29:38 -0700] - Backing up file 4 (/var/lib/dirsrv/slapd-test/bak/test-2011_06_16_09_29_38/userRoot/ancestorid.db4)
[16/Jun/2011:09:29:38 -0700] - Copying /var/lib/dirsrv/slapd-test/db/userRoot/ancestorid.db4 to /var/lib/dirsrv/slapd-test/bak/test-2011_06_16_09_29_38/userRoot/ancestorid.db4
[16/Jun/2011:09:29:38 -0700] - Backing up file 5 (/var/lib/dirsrv/slapd-test/bak/test-2011_06_16_09_29_38/userRoot/cn.db4)
[16/Jun/2011:09:29:38 -0700] - Copying /var/lib/dirsrv/slapd-test/db/userRoot/cn.db4 to /var/lib/dirsrv/slapd-test/bak/test-2011_06_16_09_29_38/userRoot/cn.db4
[16/Jun/2011:09:29:38 -0700] - Backing up file 6 (/var/lib/dirsrv/slapd-test/bak/test-2011_06_16_09_29_38/userRoot/numsubordinates.db4)
[16/Jun/2011:09:29:39 -0700] - Copying /var/lib/dirsrv/slapd-test/db/userRoot/numsubordinates.db4 to /var/lib/dirsrv/slapd-test/bak/test-2011_06_16_09_29_38/userRoot/numsubordinates.db4
[16/Jun/2011:09:29:39 -0700] - Backing up file 7 (/var/lib/dirsrv/slapd-test/bak/test-2011_06_16_09_29_38/userRoot/DBVERSION)
[16/Jun/2011:09:29:39 -0700] - Copying /var/lib/dirsrv/slapd-test/db/userRoot/DBVERSION to /var/lib/dirsrv/slapd-test/bak/test-2011_06_16_09_29_38/userRoot/DBVERSION
[16/Jun/2011:09:29:39 -0700] - Backing up file 8 (/var/lib/dirsrv/slapd-test/bak/test-2011_06_16_09_29_38/userRoot/entryrdn.db4)
[16/Jun/2011:09:29:39 -0700] - Copying /var/lib/dirsrv/slapd-test/db/userRoot/entryrdn.db4 to /var/lib/dirsrv/slapd-test/bak/test-2011_06_16_09_29_38/userRoot/entryrdn.db4
[16/Jun/2011:09:29:39 -0700] - Backing up file 9 (/var/lib/dirsrv/slapd-test/bak/test-2011_06_16_09_29_38/userRoot/parentid.db4)
[16/Jun/2011:09:29:39 -0700] - Copying /var/lib/dirsrv/slapd-test/db/userRoot/parentid.db4 to /var/lib/dirsrv/slapd-test/bak/test-2011_06_16_09_29_38/userRoot/parentid.db4
[16/Jun/2011:09:29:39 -0700] - Backing up file 10 (/var/lib/dirsrv/slapd-test/bak/test-2011_06_16_09_29_38/userRoot/aci.db4)
[16/Jun/2011:09:29:39 -0700] - Copying /var/lib/dirsrv/slapd-test/db/userRoot/aci.db4 to /var/lib/dirsrv/slapd-test/bak/test-2011_06_16_09_29_38/userRoot/aci.db4
[16/Jun/2011:09:29:39 -0700] - Backing up file 11 (/var/lib/dirsrv/slapd-test/bak/test-2011_06_16_09_29_38/userRoot/nsuniqueid.db4)
[16/Jun/2011:09:29:39 -0700] - Copying /var/lib/dirsrv/slapd-test/db/userRoot/nsuniqueid.db4 to /var/lib/dirsrv/slapd-test/bak/test-2011_06_16_09_29_38/userRoot/nsuniqueid.db4
[16/Jun/2011:09:29:39 -0700] - Backing up file 12 (/var/lib/dirsrv/slapd-test/bak/test-2011_06_16_09_29_38/log.0000000001)
[16/Jun/2011:09:29:39 -0700] - Copying /var/lib/dirsrv/slapd-test/db/log.0000000001 to /var/lib/dirsrv/slapd-test/bak/test-2011_06_16_09_29_38/log.0000000001
[16/Jun/2011:09:29:39 -0700] - Backing up file 13 (/var/lib/dirsrv/slapd-test/bak/test-2011_06_16_09_29_38/DBVERSION)
[16/Jun/2011:09:29:39 -0700] - Copying /var/lib/dirsrv/slapd-test/db/DBVERSION to /var/lib/dirsrv/slapd-test/bak/test-2011_06_16_09_29_38/DBVERSION
[16/Jun/2011:09:29:39 -0700] - All database threads now stopped

Comment 9 Amita Sharma 2011-06-17 06:49:10 UTC

[root@rhel61-ds90-amita slapd-M1]# ./db2bak -h
Usage: db2bak [archivedir] [-q] [-h]

[root@rhel61-ds90-amita slapd-M1]# ./db2bak -q
Back up directory: /var/lib/dirsrv/slapd-M1/bak/M1-2011_06_17_11_36_56


[root@rhel61-ds90-amita slapd-M1]# ./db2bak
Back up directory: /var/lib/dirsrv/slapd-M1/bak/M1-2011_06_17_11_37_17
[17/Jun/2011:11:37:17 +051800] 389-Directory/1.2.8.4 - debug level: backend (524288)
[17/Jun/2011:11:37:17 +051800] NSMMReplicationPlugin - changelog program - cl5WriteRUV: server (pid 14165) is already running; bail.
[17/Jun/2011:11:37:17 +051800] - db2archive: pre-backup-plugin failed (1).
[17/Jun/2011:11:37:17 +051800] - ERROR: Standalone db2bak is not supported when a multimaster replication enabled server is coexisting.
Please use db2bak.pl, instead.
[17/Jun/2011:11:37:17 +051800] - db2archive failed: removing /var/lib/dirsrv/slapd-M1/bak/M1-2011_06_17_11_37_17
[17/Jun/2011:11:37:17 +051800] - All database threads now stopped

Marking as VERIFIED.