Bug 215127

Created attachment 140956 [details]
Anaconda logs.

Created attachment 140967 [details]
anaconda logs and rpm list from partial FC6 upgrade

I think I have this issue.  Here are my anaconda logs and rpm lists.  I will
wait until Wednesday 15 Nov until trying again, so if anyone needs more info,
ask before then.

My installation is FC5 upgraded from FC3.  I have two install partitions (LV00
and LV02, LV01 is swap); I upgrade by copying my active installation to the
other partition, getting that working again, and then upgrading via the
installer.

I started the install from DVD with: "linux askmethod hda=noprobe hdb=noprobe",
and told it to use the .iso on my C drive (hde1).  When I got to the screen
that would start the upgrade, I backed up, in hopes that that would show the
repo selection screen (gee, is there really no way to select repos during an
upgrade?); when that just took me to the package-finding screen, I gave up and
pressed Next (waiting again) and then Next to start the upgrade.  The upgrade
appeared to succeed, but when I checked my /boot volume to reinstall any
deleted kernels for the FC5 install and grab a new copy of grub stage 1 for
NTLDR, I found that no changes had been made (even though I had chosen to
upgrade grub).	I then rebooted and noticed that it didn't seem quite upgraded.
 Notably, fedora-release still said "Fedora Core release 5 (Bordeaux)".

I rebooted into my original FC5 and mounted the other install.	I see in
/var/log/anaconda.log that what seems like the proper set of packages was
selected (there are 1086 "Adding Package" lines, though some are repeated), but
"rpm -qa --last" on the new install, when compared with the original one, shows
that 151 FC6 packages have been added while no FC5 packages have been removed.

The .iso file has the proper sha1sum, and the DVD verifies with dd'ing it back
in.  I had run memtest86 for 5 passes before upgrading.

Ehh, forgot to say that my RPM database does not seem to be corrupted, either on
the original install or in the new hybrid.

OK, Wednesday has arrived and I'm blowing away the odd update.  I did a full
dump of it, so specific files may still be requested if they would help in
debugging this.

I think the problem is due to a (slightly) corrupt RPM database.  While it
appears to work on my FC5 installation, after re-copying the installation and
getting the copy working again, I tried to yum remove a package that looked
problematic (forget which one) and the RPM database blew up.  I rebuilt the RPM
database, and then upgrading to FC6 succeeded, with nearly all packages
upgraded.  This leads me to suspect that the problem stems from having had a
corrupt RPM database that was still limping along.  Note that this time I also
didn't try backing up from the screen that would start the upgrade.

The logs I posted before were only some of the logs.  Anaconda puts its logs in
/var/log/anaconda* and also in /root/upgrade.log*, and I only noticed the
/var/log/ ones at first.  /root/upgrade.log is huge, and full of db4 errors
telling me to "run database recovery", whatever that is.  You probably don't
want it.  I also have the corrupt RPM database files.

Anaconda should clearly show at the end of installation that the installation
failed and how many packages were not upgraded.  Possibly Anaconda should check
the RPM database for integrity before starting an upgrade, if there is a way to
do it and it isn't too slow.

... Or at least, check the RPM database -before- trying to upgrade.
Leaving the machine in an inconsistent state (without telling the user anything
about it) is a big no-no.

- Gilboa

Pritam Ghanghas on fedora-list suggested using the tools in /usr/lib/rpm/,
especially rpmdb_verify.  I tried the following, and it is fast enough (and
found new corruption, hmm):

cd /var/lib/rpm ; for name in * ; do echo $name ; /usr/lib/rpm/rpmdb_verify
$name ; done

Anaconda could do something slightly more sophisticated (skip any __db* lock
files), and, if there is corruption but the Packages file is OK, offer to
rebuild the database with "rpm --rebuilddb" before proceeding with an upgrade.

User pnasrat's account has been closed

Does anyone know if this bug was fixed F7/8/rawhide?

- Gilboa

Fedora apologizes that these issues have not been resolved yet. We're
sorry it's taken so long for your bug to be properly triaged and acted
on. We appreciate the time you took to report this issue and want to
make sure no important bugs slip through the cracks.

If you're currently running a version of Fedora Core between 1 and 6,
please note that Fedora no longer maintains these releases. We strongly
encourage you to upgrade to a current Fedora release. In order to
refocus our efforts as a project we are flagging all of the open bugs
for releases which are no longer maintained and closing them.
http://fedoraproject.org/wiki/LifeCycle/EOL

If this bug is still open against Fedora Core 1 through 6, thirty days
from now, it will be closed 'WONTFIX'. If you can reporduce this bug in
the latest Fedora version, please change to the respective version. If
you are unable to do this, please add a comment to this bug requesting
the change.

Thanks for your help, and we apologize again that we haven't handled
these issues to this point.

The process we are following is outlined here:
http://fedoraproject.org/wiki/BugZappers/F9CleanUp

We will be following the process here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping to ensure this
doesn't happen again.

And if you'd like to join the bug triage team to help make things
better, check out http://fedoraproject.org/wiki/BugZappers

AIUI, the original bug that corrupted RPM databases is fixed, but there is still
no check of RPM database integrity in Anaconda, so no one can know for sure. 
Properly fixing the bug needs a change to Anaconda to check the RPM database
before using it.  I know that the checks are fairly weak, only checking each
table for integrity and not for referential integrity across the database, but
the checks have proven to be able to find many problems in the field.  Just
because they aren't perfect is a poor reason not to do useful checks before
starting a long process that depends on the RPM database.

Since the original bug is fixed, closing this bug.

Please open a new bug for the other issue.

Thanks