Bug 671377

Note that we still want live usb devices to be able to run yum or rpm.

Whether the media is read-only or not is not relevant for the __db* files: they get created as needed, but they should not be copied from one system to another, much less included in any images.

yum and rpm should work to some extent even on read only media, though the changes won't persist. For usb devices, you can do permanent upgrades using the overlay if you created one on the device.

Does this problem show up when doing live installs? I wouldn't expect there to be a problem with having those files for a live image even when you won't normally be updating, as the live image is the host. Even when doing live installs, wouldn't the copy be valid, since you'll be installing to the same architecture and the same package set.

Isn't creating them on the fly going to make running yum on a live image pretty slow?

Another note is that if you do an update of a live usb aren't those files going to be recreated and again used for the install process if you do a live install from an updated live usb?

I don't know the exact context where Matthias' problem came up, or much about live usb etc specifics. But you do not ever want to include the __db* files on installation images of any kind, period. Besides the potential problems they can cause, on read-only media they do NOTHING but waste space as the environment needs to be writable (even for read-only access to the actual database) to be usable. When it's not writable, a private (ie in-memory) environment is used.

I thought I had a sense of deja-vu wrt this, and indeed there's this in livecd-tools yuminst.py already:

    def _cleanupRpmdbLocks(self, installroot):
        # cleans up temporary files left by bdb so that differing
        # versions of rpm don't cause problems
        for f in glob.glob(installroot + "/var/lib/rpm/__db*"):
            os.unlink(f)

Makes me wonder what has made this to resurface, is there some new method to create images which bypasses this or something...

Live images aren't just installation media, they are also running systems with overlays so they are updatable even when the media is read only.

Doing the removal in yuminst.py seems like the right place to deal with this, though the way installs work you probably also should guard against updates running concurrently with the install. (It may in fact do that; I don't know.)

Well, the __db* files belong to the writable overlay part along with other updatable parts of the system. Not in distributed images.

I disagree. I agree that is a good idea not to copy them for a live install. They could be in some intermediate state at that time if something using rpm/yum is running while the copy is happening. I see no reason that they can't be part of the live image. Removing them from a live image has performance implications. Recreating those files takes a significant amount of time and for cases where the update is done on an in memory overlay, you need to rebuild them again after rebooting if you need them again.
If they  really don't work on live images, I would expect that they also wouldn't work after normal systems reboot and that aren't currently getting rebuilt after every reboot.

You know, those files get removed on each system boot anyway... so again, you're literally doing nothing but wasting space by including the __db* environment files on images.

No I didn't know that. That explains why sometimes yum appears to be much slower than usual. Given that, I have no objection to stripping those files out.

I'll look at doing this if no one beats me to it. These files take up a significant amount of space when uncompressed. Even compressed on the live image, its probably a noticeable amount of space.
I won't remove the code in liveinst.py that also removes these files, as that will still be needed if yum has been run on the live image.

I started taking a look at this and there is already code that appears to try to remove the db files after yum install is done, but it doesn't work. It might be something obvious to people who are more used to python than I am. I'm going to see if the code is actually getting called and if so what exact path it is trying to unlink.

It looks like the files do get properly removed part way through building the iso. So I suspect that something is recreating them again after that step.

The files are back by the time the man page stuff is done. I notice that a package list is printed out inbetween. I bet whatever generates the package list is what is causing the problem.

It looks like the problem is really in the live-base ks file. I'll be testing that.

I added a command to remove the db files in the ks files that ran rpm in the %post section. This change is upstream, but not in a released spin-kickstarts package. The nightlies should start using the fixed version within a day or so.

This should be fixed with spin-kickstarts-0.15.1-1.fc15 which is in rawhide and this version or the one from the mass rebuild should show up in F15 as soon as that is refreshed.