Bug 200670

Summary: rpm database unrecoverble
Product: [Fedora] Fedora Reporter: Sun Minchao <mcsun>
Component: rpmAssignee: Paul Nasrat <nobody+pnasrat>
Status: CLOSED WORKSFORME QA Contact: Mike McLean <mikem>
Severity: urgent Docs Contact:
Priority: medium    
Version: 3   
Target Milestone: ---   
Target Release: ---   
Hardware: i686   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2006-08-01 12:26:27 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Sun Minchao 2006-07-30 12:40:54 UTC 
Description of problem:
rpm database corrupted and unrecoverble

Version-Release number of selected component (if applicable):
rpm version 4.3.2

How reproducible:


Steps to Reproduce:
1.as root in single user mode: cd /var/lib/rpm
2.rm -f __db*
3.db_verify Packages (failed)
4.rpm --rebuilddb (failed)
  
Actual results:

[/var/lib/rpm] rm -f __db*
[/var/lib/rpm] db_verify Packages
db_verify: page 466: illegal page type or format
db_verify: page 466: illegal page type or format
db_verify: PANIC: Invalid argument
db_verify: Packages: pgin failed for page 466
db_verify: Packages: DB_RUNRECOVERY: Fatal error, run database recovery
db_verify: DB->verify: Packages: DB_RUNRECOVERY: Fatal error, run database
recovery
db_verify: PANIC: fatal region error detected; run recovery
db_verify: dbenv->close: DB_RUNRECOVERY: Fatal error, run database recovery
[/var/lib/rpm] mv Packages Packages.orig
[/var/lib/rpm] db_dump Packages.orig | db_load Packages
db_dump: page 466: illegal page type or format
db_dump: PANIC: Invalid argument
db_dump: Packages.orig: pgin failed for page 466
db_dump: DB->stat: DB_RUNRECOVERY: Fatal error, run database recovery
db_dump: PANIC: fatal region error detected; run recovery
db_dump: close: DB_RUNRECOVERY: Fatal error, run database recovery
db_dump: PANIC: fatal region error detected; run recovery
db_dump: dbenv->close: DB_RUNRECOVERY: Fatal error, run database recovery
db_load: line 4: unexpected format
[/var/lib/rpm] rpm --rebuilddb
rpmdb: PANIC: fatal region error detected; run recovery
error: db4 error(-30978) from dbenv->open: DB_RUNRECOVERY: Fatal error, run
database recovery
error: cannot open Packages index
[/var/lib/rpm]


Expected results:


Additional info:
the damaged /var/lib/rpm/Packages file is to large (39MB) to be attached here.
Comment 1 Sun Minchao 2006-07-30 12:47:19 UTC 
*** Bug 200669 has been marked as a duplicate of this bug. ***
Comment 2 Jeff Johnson 2006-07-31 12:01:55 UTC 
Can you add a ptr (i.e. a URL) to a tarball of your database?
    cd /var/lib
    tar czvf /tmp/rpmdb-200670.tar.gz ./rpm
Comment 3 Sun Minchao 2006-07-31 12:33:27 UTC 
(In reply to comment #2)
> Can you add a ptr (i.e. a URL) to a tarball of your database?
>     cd /var/lib
>     tar czvf /tmp/rpmdb-200670.tar.gz ./rpm
> 

Thanks a lot for your quick reply!
I now put the achieved /var/lib/rpm here:

ftp://minchao.dyndns.org/pub/download/rpmdb-20070727.tar.gz
Comment 4 Jeff Johnson 2006-08-01 12:16:43 UTC 
Reproduced the problem:
# /usr/lib/rpm/rpmdb_verify Packages 
rpmdb_verify: page 466: illegal page type or format
rpmdb_verify: PANIC: Invalid argument
rpmdb_verify: Packages: pgin failed for page 466
rpmdb_verify: Packages: DB_RUNRECOVERY: Fatal error, run database recovery
rpmdb_verify: PANIC: fatal region error detected; run recovery
rpmdb_verify: dbenv->close: DB_RUNRECOVERY: Fatal error, run database recovery
Comment 5 Jeff Johnson 2006-08-01 12:26:27 UTC 
This is the "standard" fix"

# rm -f __db*
# mv Packages Packages-ORIG
# /usr/lib/rpm/rpmdb_dump Packages-ORIG | /usr/lib/rpm/rpmdb_load Packages
rpmdb_dump: page 466: illegal page type or format
rpmdb_dump: PANIC: Invalid argument
rpmdb_dump: Packages-ORIG: pgin failed for page 466
rpmdb_dump: PANIC: fatal region error detected; run recovery
rpmdb_dump: close: DB_RUNRECOVERY: Fatal error, run database recovery
rpmdb_dump: PANIC: fatal region error detected; run recovery

After doing those operations, there is significant data loss:
# ls -al Pack*
-rw-r--r-- 1 root root  7307264 Aug  1 08:25 Packages
-rw-r--r-- 1 rpm  rpm  39190528 Jul 19 14:55 Packages-ORIG

But the above saves what can be saved. I've put what can be saved at
    ftp://wraptastic.org/pub/jbj/Packages-200670.gz
Download, uncompress, and do
    mv /var/lib/rpm/Packages /var/lib/rpm/Packages-SAVE
    mv Packages /var/lib/rpm
    rpm --rebuilddb

Proceed by rinstalling headers that have been lost. The packages
that were installed on your box are in /var/log/rpmpkgs*. Compare
that list with "rpm -qa" output, reinstalling any packages that have been
lost.

Another way to proceed is to do
    rpm -Va --nofiles
installing packages to meet any unsatisfied dependencies.
Comment 6 Sun Minchao 2006-08-01 18:23:58 UTC 
I'm now trying to reinstall the missing packages, a very very hard job,
especially because of the broken dependencies, a lesson will never be forgotten...

anyway, thank you very much for the quick help!