Bug 53188

Summary: rpm crashes with seg fault doing database operations
Product: [Retired] Red Hat Linux Reporter: Stan Bubrouski <stan>
Component: rpmAssignee: Jeff Johnson <jbj>
Status: CLOSED NOTABUG QA Contact:
Severity: medium Docs Contact:
Priority: medium    
Version: 7.1   
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
URL: http://66.31.233.44:81/rpm/rpmdb.tar.bz2
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2001-09-05 20:51:22 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Stan Bubrouski 2001-09-04 23:19:27 UTC
From Bugzilla Helper:
User-Agent: Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0)

Description of problem:
This a problem carried over from the rpm-list mailing list.  There appears 
to be a bug in rpm where it is trying to free a value that has already 
been freed (malloc stuff).

Version-Release number of selected component (if applicable):


How reproducible:
Always

Steps to Reproduce:
1. Download the database archive and decompress it(it is corrupt but that 
is not why rpm crash
2. Move the relevent database files to /var/lib/rpm
3. Run a rpm --rebuilddb
	

Actual Results:  [root@linux rpm]# rpm --rebuilddb
error: rpmdb: damaged header instance #3 retrieved, skipping.
error: rpmdb: damaged header instance #11 retrieved, skipping.
error: rpmdb: damaged header instance #19 retrieved, skipping.
error: rpmdb: damaged header instance #27 retrieved, skipping.
error: rpmdb: damaged header instance #35 retrieved, skipping.
error: rpmdb: damaged header instance #123 retrieved, skipping.
error: rpmdb: damaged header instance #131 retrieved, skipping.
error: rpmdb: damaged header instance #179 retrieved, skipping.
error: rpmdb: damaged header instance #195 retrieved, skipping.
error: rpmdb: damaged header instance #235 retrieved, skipping.
error: rpmdb: damaged header instance #344 retrieved, skipping.
error: rpmdb: damaged header instance #360 retrieved, skipping.
error: rpmdb: damaged header instance #384 retrieved, skipping.
error: rpmdb: damaged header instance #488 retrieved, skipping.
error: rpmdb: damaged header instance #569 retrieved, skipping.
error: rpmdb: damaged header instance #577 retrieved, skipping.
error: rpmdb: damaged header instance #633 retrieved, skipping.
error: rpmdb: damaged header instance #649 retrieved, skipping.
error: rpmdb: damaged header instance #753 retrieved, skipping.
error: rpmdb: damaged header instance #774 retrieved, skipping.
Segmentation fault

Expected Results:  [root@linux rpm]# rpm --rebuilddb
[root@linux rpm]# 

Additional info:

[root@linux stan]# gdb /usr/lib/rpm/rpmd
GNU gdb Red Hat Linux 7.x (5.0rh-12)
Copyright 2001 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you 
are
welcome to change it and/or distribute copies of it under certain 
conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-redhat-linux"...
(gdb) run --rebuilddb
Starting program: /usr/lib/rpm/rpmd --rebuilddb
error: rpmdb: damaged header instance #3 retrieved, skipping.
error: rpmdb: damaged header instance #11 retrieved, skipping.
error: rpmdb: damaged header instance #19 retrieved, skipping.
error: rpmdb: damaged header instance #27 retrieved, skipping.
error: rpmdb: damaged header instance #35 retrieved, skipping.
error: rpmdb: damaged header instance #123 retrieved, skipping.
error: rpmdb: damaged header instance #131 retrieved, skipping.
error: rpmdb: damaged header instance #179 retrieved, skipping.
error: rpmdb: damaged header instance #195 retrieved, skipping.
error: rpmdb: damaged header instance #235 retrieved, skipping.
error: rpmdb: damaged header instance #344 retrieved, skipping.
error: rpmdb: damaged header instance #360 retrieved, skipping.
error: rpmdb: damaged header instance #384 retrieved, skipping.
error: rpmdb: damaged header instance #488 retrieved, skipping.
error: rpmdb: damaged header instance #569 retrieved, skipping.
error: rpmdb: damaged header instance #577 retrieved, skipping.
error: rpmdb: damaged header instance #633 retrieved, skipping.
error: rpmdb: damaged header instance #649 retrieved, skipping.
error: rpmdb: damaged header instance #753 retrieved, skipping.
error: rpmdb: damaged header instance #774 retrieved, skipping.

Program received signal SIGSEGV, Segmentation fault.
chunk_free (ar_ptr=0x81af660, p=0x8201128) at malloc.c:3089
3089    malloc.c: No such file or directory.
        in malloc.c
(gdb) bt
#0  chunk_free (ar_ptr=0x81af660, p=0x8201128) at malloc.c:3089
#1  0x08120735 in __libc_free (mem=0x8201130) at malloc.c:3063
#2  0x080551e7 in headerFree (h=0x81f22f8) at header.c:84
#3  0x08070fb0 in rpmdbNextIterator (mi=0x81d0688) at ../lib/hdrinline.h:50
#4  0x08073332 in rpmdbRebuild (rootdir=0x8179de0 "/") at rpmdb.c:3162
#5  0x08048826 in main (argc=2, argv=0xbffffa84) at rpmqv.c:903
#6  0x0810126a in __libc_start_main (main=0x80482bc <main>, argc=2, 
ubp_av=0xbffffa84, init=0x80480d4 <_init>, 
    fini=0x8179da0 <_fini>, rtld_fini=0, stack_end=0xbffffa7c) 
at ../sysdeps/generic/libc-start.c:129
(gdb) list
3084    in malloc.c
(gdb) list header.c:84
79       * @return              NULL always
80       */
81      /*@unused@*/ static inline /*@null@*/ void *
82      _free(/*@only@*/ /*@null@*/ const void * p) /*@modifies *p @*/
83      {
84          if (p != NULL)      free((void *)p);
85          return NULL;
86      }
87
88      Header headerNew()
(gdb) list ../lib/hdrinline.h:50
45              /*@modifies h @*/
46      {
47          /*@-abstract@*/
48          if (h == NULL) return NULL;
49          /*@=abstract@*/
50          return (h2hv(h)->hdrfree) (h);
51      }
52
53      /** \ingroup header
54       * Reference a header instance.
(gdb) list rpmdb.c:3162
3157                    rpmError(RPMERR_INTERNAL,
3158                            _("cannot add record originally at %u\n"), 
_RECNUM);
3159                    failed = 1;
3160                    break;
3161                }
3162            }
3163
3164            mi = rpmdbFreeIterator(mi);
3165
3166        }
(gdb) list rpmqv.c:903
898         case MODE_INITDB:
899             (void) rpmdbInit(rootdir, 0644);
900             break;
901
902         case MODE_REBUILDDB:
903             ec = rpmdbRebuild(rootdir);
904             break;
905         case MODE_VERIFYDB:
906             ec = rpmdbVerify(rootdir);
907            break;

This is from a CVS build from Tue Sep  4 19:01:01 EDT 2001
The problem also occurs in the rpm included with Red Hat 7.1 and with the 
rpm in rawhide (rpm-4.0.3-0.91).

Jeff to get my db just download it from the URL included with this post 
via http and you should get it fine.

Comment 1 Jeff Johnson 2001-09-05 16:58:49 UTC
Hmmm, your database is majorly borked, I'm not going to
be able to do anything with it. I've looked at the
segfault, and hardcoded skips across bad records. The
next 15 or so records after the segfault are all bad, and I
have no reason to think that continuing to skip bad records
is going to lead to joy.

Yes, rpm will continue to segfault, as it relies on
intact headers being retrieved.

Meanwhile, look at /var/log/rpmpkgs. There should be a
list of the package that you had installed. If you
can find all the packages (or close equivalents) you
can recreate the rpmdb by doing
    cd /var/lib
    mv rpm rpm-SAVE
    mkdir rpm
    rpm --initdb
and then installing each package with the flags
    -Uv --justdb --notriggers --noscripts --nodeps

I'd also be interested in how your database was damaged.
All I can think of is that you ran out of space while
doing a large upgrade, causing many evil things to
happen.


Comment 2 Stan Bubrouski 2001-09-05 20:35:59 UTC
Disk space was never an issue.  I don't know how it happened.  I can try to 
investigate but chances are I'm not going to find anything.

Comment 3 Stan Bubrouski 2001-09-05 20:39:23 UTC
Looking at /var/log/rpmpkgs it's only has the same amount (260) as my damaged 
database shows me with rpm -qa, so now my guess would be the corruption 
occurred after installing one of the rawhide rpm packages.

Comment 4 Stan Bubrouski 2001-09-05 20:43:32 UTC
I do an automatic backup every week so I have an older database to compare to 
the new one.  The problem started (as far as I know on August 22nd and I have a 
backup from August 20th.)  The only downside is that I updated a few hundred 
packages between August 20th and 22nd, but on the good side, a lot of the 
packages listed in /var/log/rpmpkgs are the ones I updated between those 
dates).  If you'd like to compare the older one to the corrupt one I can post 
it on my webserver if you think it would assist you in any way.  It's not a 
problem.

Comment 5 Stan Bubrouski 2001-09-05 20:51:16 UTC
Unfortunately the database from the 20th is corrupt as well...and it is the 
only backup I had left.  Oh well.

Comment 6 Jeff Johnson 2001-09-06 19:15:02 UTC
Hmmm, from the guy who smashes rpm databases for a living :-)

You might wnat to rename /var/lib/rpm, and do an install.
If you carefully avoid doing mkfs on your existing file
systems, then most all the packages you wish to reinstall
will be reinstall'ed by anaconda, the Red Hat installed,
and most of your existing customizations will be saved
as .rpmorig files.

After getting through the install, look for unowned files
in all the usual (i.e. /bin /usr/bin /usr/sbin /usr/lib ...)
directories, looking for orphaned files with
    rpm -qf /some/file/path
For each orphaned file, then do
     rpm -q --redhatprovides /some/file/path
to find which package contains the file. You need
the rpmdb-redhat package installed for this to work.

I'm gonna close this bug, but feel free to ask on rpm-list
or reopen this bug if you need help.