Bug 679880

/usr/bin/readelf is in the binutils package.

I believe this could be accomplished entirely in the kickstart that creates the spin.

The disk usage was 235MB, as processed by the first automatic prelink on gfx_test_week_20110221_x86-64.

I think disabling prelink on the live media makes sense, but not sure it makes sense to run it as part of the compose. I think that would be adding a lot of time to compose and not really getting us much.

Prelinking saves about 9% of boot time.  That's F15 alpha XFCE desktop after Install-to-harddrive from Live.iso and first boot; comparing second boot (35 seconds to start of X11) to third boot (32 seconds) where /etc/cron.daily/prelink was run by hand between second and third boot [1.6GHz Pentium 4 uniprocessor, 1GB PC2100 SDRAM.]

Prelinking touched 176MB of libraries, and 324MB total (libraries plus apps; see lists in /var/log/prelink/prelink.log.)  It took only a couple minutes.

I've commited a change to run it at compose time... 

I'm not sure turning it off on the live copy is worth while. If it's already run at compose time, the amount of changes if it runs after that should be pretty small, right?

(In reply to comment #6)
>  If it's already run at compose time, the amount of changes if it runs after
>  that should be pretty small, right?

No; each time prelink runs it assigns the addresses randomly.

I think this should be run with || : so you can still compose live images with
"-prelink".

I made the following changes.

I added the -m option which is needed in some cases for -i686.

I added &>/dev/null || : to the prelink commands.

I added a check to livesys to turn off running prelink using code from live-kde-base.

Maybe you could just run `/etc/cron.daily/prelink'.  It will do the full prelink on initial run - as /etc/prelink.cache does not exist that time.
For example the default option -R has security benefits and it is present in `/etc/sysconfig/prelink' as used by `/etc/cron.daily/prelink'.

I am not sure what the cost is of using -R, but the benefit seems low here, since the live images are public. If it's nearly free, it's reasonable to add it.

I am not sure how running cron really helps; we aren't planning to run prelink again later so worrying about caching seems like extra work.

(In reply to comment #11)
> the benefit seems low here, since the live images are public.

oops, yes.

> If it's nearly free, it's reasonable to add it.

It may have some higher page tables cost, it should be negligible.

> I am not sure how running cron really helps; we aren't planning to run prelink
> again later so worrying about caching seems like extra work.

I do not talk about crond itself but about the script `/etc/cron.daily/prelink' which is prepared to be run by crond but one can run it even directly.

When -R is not useful it may no longer bring any real benefit, it creates a log which may be considered both positive or negative etc.

Uh, does prelink bloat the size of the live image? The latest nightly compose of the KDE spin came out oversized, with i686 having grown by almost 30 MiB between April 2 and April 3!

(But I'm not sure whether the size increase is really prelink's fault or whether it's just some strange stochastic variance. Unfortunately, there have been no nightly composes since April 3, so it's hard to tell.)

FYI:
20110401.17 didn't use prelink, the KDE image sizes were 689 MiB for i386, 688 MiB for x86_64.
20110402.18 used prelink -a, the KDE image sizes were 689 MiB for i386, 703 MiB for x86_64. (Did prelink fail on i386 there?)
20110403.17 used prelink -am, the KDE image sizes were 713 MiB for i386, 705 MiB for x86_64.

The KDE spin cannot take such a size increase.

I committed these:
http://git.fedorahosted.org/git/?p=spin-kickstarts.git;a=commitdiff;h=36271446ffc9384263e4ba9c9a4881521014b4b0
http://git.fedorahosted.org/git/?p=spin-kickstarts.git;a=commitdiff;h=596ef8e70cb7369826e8b9d45a0c56e2cc39ddd0

to drop the prelinking again. Sorry, but the size increase is really not acceptable for the KDE spin. It might make sense to prelink some spins, but forcing this in -base for everyone is a very bad idea.

To answer the obvious question:
Yes, now, I am sure it is prelink which causes this:
* The difference between i386 and x86_64 on April 2 is easily explained by the fact that prelink was run without -m, which doesn't always work on i386 according to the commit messages and thus the i386 image wasn't actually prelinked on April 2, it was on April 3.
* Considering the above, the size increases for i386 and x86_64 are easily correlated: 24 MiB for i386, 17 MiB for x86_64, that's in the same order of magnitude.
* I also looked at the differences in content between April 2 and 3 for the i386 spin. There are no added packages, only updated ones, and the only package I considered suspicious at first (lovelock-backgrounds-single) has hardly changed in size.

For the other spins, I think we want to be careful about doing changes right around the beta release, but I agree the size increase is a problem.
I'd like to find out if the size increase is due to small amounts of increase per binary or library or if there are some scratch files being left behind that could be cleaned up.

There are several things which could cause size increases:
* the fact that the relocated addresses don't compress so well,
* the undo information needed to support prelink -u,
* /etc/prelink.cache,
* from the manpage: "It also stores a list of all dependent libraries together with their checksums into the binary or library."
* also from the manpage: "For binaries, it also computes a list of conflicts (relocations that resolve differently in the binary's symbol search scope than in  the smaller search scope in which the dependent library was resolved) and stores it into a special ELF section."

On i686 prelink sometimes converts relocations REL -> RELA (thus being larger).
Yes, i686 should be abandoned.

Please remember the other part of my original report: disable /etc/cron.daily/prelink so that it does not run by default after booting a Live spin.  The cron job is an unexpected and noticeable disruption (interactivity slows significantly), and it eats memory (tmpfs) on a Live system: about 340MB of RAM to store the results.  This is very bad on a minimum system with 512MB RAM and still unreasonable for 1GB RAM.

That part (disabling prelink runs on the uninstalled Live system) has been implemented and is not controversial

Re comment #20 (Jan Kratochvil): That could explain the extra 7 MiB penalty (for a total of 24 MiB) on i686, but even on x86_64 we have a 17 MiB penalty.

A prelink of just the shared libraries that are used most often can reap measurable benefits in speed at small cost in space (usually a couple Kbytes per library.)  One likely list is (sorted by most important first):
   ld-linux.so.2
   libc.so.6
   libgcc_s.so.1
   libdl.so.1
   librt.so.1
   libpthread.so.0
   libtinfo.so.5   [for bash]
   libselinux.so.1, libacl.so.1, libattr.so.1  [for mv/cp/ls]
Even prelinking just the first three (ld-linux, libc, libgcc_s) would help a lot.

See LD_DEBUG=all.  If any of the executable or any of its dynamically linked libraries is not properly up to date with valid checksum the whole prelinked data are dropped and the normal ld.so dynamic linking process starts.

After my commits from comment #16 (i.e. with prelink not run anymore), the latest live composes (Nightly-20110407.23, F15 Beta RC1) are back to the original size (690 MiB ± 1 MiB for fedora-livecd-kde), so it was really prelink causing the bloat, not some other change.

This message is a notice that Fedora 15 is now at end of life. Fedora
has stopped maintaining and issuing updates for Fedora 15. It is
Fedora's policy to close all bug reports from releases that are no
longer maintained. At this time, all open bugs with a Fedora 'version'
of '15' have been closed as WONTFIX.

(Please note: Our normal process is to give advanced warning of this
occurring, but we forgot to do that. A thousand apologies.)

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, feel free to reopen
this bug and simply change the 'version' to a later Fedora version.

Bug Reporter: Thank you for reporting this issue and we are sorry that
we were unable to fix it before Fedora 15 reached end of life. If you
would still like to see this bug fixed and are able to reproduce it
against a later version of Fedora, you are encouraged to click on
"Clone This Bug" (top right of this page) and open it against that
version of Fedora.

Although we aim to fix as many bugs as possible during every release's
lifetime, sometimes those efforts are overtaken by events. Often a
more recent Fedora release includes newer upstream software that fixes
bugs or makes them obsolete.

The process we are following is described here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping