Red Hat Bugzilla – Bug 90560
RPM database error: How to recover safely and fully
Last modified: 2007-04-18 12:53:38 EDT
Description of problem:
How to Repair an RPM database safely after:
rpmdb: fatal region error detected; run recovery
error: db4 error(-30982): Fatal error, run database recovery
Version-Release number of selected component (if applicable):
RPM version 4.2
Unknown in my /var/lib/rpm/Packages database (Fatal error)
Yes in rpmdb-redhat-9-0.20030313
Steps to Reproduce:
1. chdir /usr/lib/rpmdb/i386-redhat-linux/redhat
2. db_verify Packages
db_verify: region error detected; run recovery.
No errors, database ok
chdir /var/lib/rpm ; rpm --rebuilddb --> trimmed to 75% my Packages file
(see attached text file with more info)
You appear to have two problems.
Before doing anything else, save a copy of /var/lib/rpm
tar czvf rpmdb.tar.gz rpm
First eliminate stale locks by doing
rm -f /var/lib/rpm/__db*
say? If rpmdb_verify gives a recovery needed (or any other) error, then
the fix is to dump and reload the file by doing:
mv Packages Packages-ORIG
/usr/lib/rpm/rpmdb_dump Packages-ORIG | /usr/lib/rpm/rpmdb_load Packages
After Packages passes rpmdb_verify, then a rpm --rebuilddb will
regenerate the indices.
DOes that fix?
*** Bug 90589 has been marked as a duplicate of this bug. ***
It worked, but with my trimmed database (reduced Packages file after "rpm
--rebuild" I did yesterday).
But it works now with "/usr/lib/rpm/rpmdb_verify /var/lib/rpm/Packages", as well
as "/usr/lib/rpm/rpmdb_verify usr/lib/rpmdb/i386-redhat-linux/redhat/Packages"
(both without errors or warnings).
Yesterday I was doing "db_verify" and "db_dump", "db_load", in fact using the
old versions (in the path):
-r-xr-xr-x 1 root root 5548 feb 5 22:09 /usr/bin/db_archive
-r-xr-xr-x 1 root root 7092 feb 5 22:09 /usr/bin/db_checkpoint
-r-xr-xr-x 1 root root 6768 feb 5 22:09 /usr/bin/db_deadlock
-r-xr-xr-x 1 root root 10008 feb 5 22:09 /usr/bin/db_dump
-r-xr-xr-x 1 root root 51864 feb 5 22:09 /usr/bin/db_dump185
-r-xr-xr-x 1 root root 15924 feb 5 22:09 /usr/bin/db_load
-r-xr-xr-x 1 root root 6284 feb 5 22:09 /usr/bin/db_printlog
-r-xr-xr-x 1 root root 6564 feb 5 22:09 /usr/bin/db_recover
-r-xr-xr-x 1 root root 19140 feb 5 22:09 /usr/bin/db_stat
-r-xr-xr-x 1 root root 5548 feb 5 22:09 /usr/bin/db_upgrade
-r-xr-xr-x 1 root root 5704 feb 5 22:09 /usr/bin/db_verify
# /usr/bin/db_verify -V
Sleepycat Software: Berkeley DB 4.0.14: (November 18, 2001)
Instead of the new ones (not in path):
-rwxr-xr-x 1 rpm rpm 6948 feb 27 22:24 /usr/lib/rpm/rpmdb_deadlock
-rwxr-xr-x 1 rpm rpm 11156 feb 27 22:24 /usr/lib/rpm/rpmdb_dump
-rwxr-xr-x 1 rpm rpm 18032 feb 27 22:24 /usr/lib/rpm/rpmdb_load
-rwxr-xr-x 1 rpm rpm 24748 feb 27 22:24 /usr/lib/rpm/rpmdb_stat
-rwxr-xr-x 1 rpm rpm 39652 feb 27 22:24 /usr/lib/rpm/rpmdb_svc
-rwxr-xr-x 1 rpm rpm 7248 feb 27 22:24 /usr/lib/rpm/rpmdb_verify
# /usr/lib/rpm/rpmdb_verify -V
Sleepycat Software: Berkeley DB 4.1.24: (September 13, 2002)
It seems to me that the upgrade to RPM-4.2 in RH-9 does not erase the old
"/usr/bin/db_*" binaries of DB-4.0.14, neither installs the new binaries
"/usr/lib/rpm/rpmdb_*" of DB-4.1.24 in the user/root path.
I don't know if leaving the old "/usr/bin/db_*" binaries is harmful, as far as I
don't use them, but use the new "/usr/lib/rpm/rpmdb_*" ones.
On other hand, why the "db_recover" and other binaries are not in the new version?
Second, I still get the following error after erasing the __db.*, verifying ok
with /usr/lib/rpm/rpmdb_verify, and:
# rpm --rebuilddb
error: db4 error(16) from dbenv->remove: Dispositivo o recurso ocupado
(device or resource busy).
This means that --rebuilddb does not work well with the new databases?
Or perhaps it is not needed since the __db.* index files are generated
otherwise? (in this case the outdated option should not be deprecated?).
Finally, I still have the task of recovering my full initial rpm database, as I
told you in my first e-mail, that I update hereafter:
Now I am able to query only 726 rpm's in my trimmed database, so that I have
lost about 247 rpm's. Since my Packages file was reduced from 37376000 bytes to
26710016 bytes, it is to about 75 % (similar to 973 to 726), I assume that I
have lost the info about the missing rpm's.
Now, before going to the painful task of "reloading" all my missed rpm in the
database, I have 1 specific question left:
1. May I just "copy" the Packages file from a similar computer at home where I
have installed the same packages, more or less at the same times (since I
updated to RH-9 a few weeks ago in both machines), and just try to reload a few
of the rpms newly installed at this machine?.
A final remark is that I am afraid of not fixing my rpm database, because I
could get false dependencies faults in the future, or other troubles. Don't you
agree on this?
I have solved my problem by copying my full /var/lib/rpm directory with the one
in my home machine with quite similar packages. Thus I had to update only a few
different packages in both machines.
I have no idea why my /var/lib/rpm/Packages file got corrupted with a "region
fatal error", neither why when I did "rpm --rebuilddb" my Packages file was
trimmed and 25% of the packages were lost there.
Since I did not save my corrupted Packages file I can not check if the alternate
procedure, (given by jbj above:
cd /var/lib/rpm ; rm -f __db.* ; mv Packages Packages-ORIG ;
/usr/lib/rpm/rpmdb_dump Packages-ORIG | /usr/lib/rpm/rpmdb_load Packages ) would
have worked to recover my damaged Packages database.
On other hand, I found that the binaries /usr/bin/db_* , in the default path,
that were installed under RedHat-9 in the db4-utils-4.0.14-20.i386.rpm, are from
a lower version 4.0.14 of Berkeley DB4, while the binaries /usr/lib/rpm/rpmdb_*,
not in the standard path, were installed by rpm-devel-4.2-0.69.i386.rpm, and use
a newer version DB 4.1.24. So that doing "db_verify Packages" results in a
region error found in the new rpm-4.2 generated databases. Probably the use of
"db_dump" and "db_load" also has some potential problems.
I regard this report as NOT-A-BUG, resolving it, but I recommend Red Hat to
improve or update the documentation for "rpm --rebuilddb" and "rpmdb
--rebuilddb" man pages, giving there the recommendation to save the rpm database
directory "/var/lib/rpm/" before doing a "rebuilddb", deleting the "__db.00?"
files and running /usr/lib/rpm/rpmdb_dump and _load, as well as
"/usr/lib/rpm/rpmdb_verify /var/lib/rpm/Packages", instead of the "db_verify".
It would also be helpful to have some info in these man pages about the
maintenance of the "/var/log/rpmpkgs" files, with the full list of the installed
rpms, updated by the "/etc/cron.daily/rpm" file, and saved weekly by the
"/etc/logrotate.d/rpm" file, as distributed in rpm-4.2.
To have compatible versions of DB4 in future RH distros, would help to avoid
Many thanks to Jeff Johnson (firstname.lastname@example.org) for your quick attention.
I first noticed the rpm db corruption problem after I first installed rh9 and
then ran the up2date client. After the up2date client stalled out and crashed,
the rpm db has been corrupted and giving me troubles sense. Why does rpmdb --
rebuild always say device unavailable or give a small error message?