Bug 679880 - Live spins vs prelink
Summary: Live spins vs prelink
Alias: None
Product: Fedora
Classification: Fedora
Component: spin-kickstarts
Version: 15
Hardware: Unspecified
OS: Unspecified
Target Milestone: ---
Assignee: Jeroen van Meeuwen
QA Contact: Fedora Extras Quality Assurance
Depends On:
TreeView+ depends on / blocked
Reported: 2011-02-23 18:44 UTC by John Reiser
Modified: 2013-01-10 06:28 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Last Closed: 2012-08-07 19:53:42 UTC
Type: ---

Attachments (Terms of Use)

Description John Reiser 2011-02-23 18:44:13 UTC
Description of problem: Prelink should be run once during the build of Live spins, and then disabled so that prelink does not run in a Live session with readonly media.  Currently prelink is not run during build, so invocation of any Live compiled binary program suffers relocation on-the-fly, which can be expensive for shared libraries (including almost anything that uses graphics.)  Running prelink (initiated by a cron job) during a Live session from readonly media either wastes RAM (to store the newly-written results), or wastes time and causes headaches (if there is no place to store the dozens of megabytes that a prelink often generates) 

Version-Release number of selected component (if applicable):

How reproducible: every time

Steps to Reproduce:
1. boot gfx_test_week_20110221_x86-64.iso
2. yum install elfutils (or manually download 'readelf')
3. readelf --headers /lib64/libc.so.6
Actual results: .p_vaddr is 0 (zero), indicating prelink was not run.
Also look in /var/log/prelink/prelink.log after the cron job has run (about one hour or so [?] into Live session.)

Expected results: .p_vaddr is non-zero, indicating prelink was run before freezing the Live filesystem.

Additional info:

Comment 1 John Reiser 2011-02-23 19:02:12 UTC
/usr/bin/readelf is in the binutils package.

Comment 2 Bill Nottingham 2011-02-23 19:03:11 UTC
I believe this could be accomplished entirely in the kickstart that creates the spin.

Comment 3 John Reiser 2011-02-24 00:08:42 UTC
The disk usage was 235MB, as processed by the first automatic prelink on gfx_test_week_20110221_x86-64.

Comment 4 Kevin Fenzi 2011-02-24 17:52:18 UTC
I think disabling prelink on the live media makes sense, but not sure it makes sense to run it as part of the compose. I think that would be adding a lot of time to compose and not really getting us much.

Comment 5 John Reiser 2011-02-26 16:18:43 UTC
Prelinking saves about 9% of boot time.  That's F15 alpha XFCE desktop after Install-to-harddrive from Live.iso and first boot; comparing second boot (35 seconds to start of X11) to third boot (32 seconds) where /etc/cron.daily/prelink was run by hand between second and third boot [1.6GHz Pentium 4 uniprocessor, 1GB PC2100 SDRAM.]

Prelinking touched 176MB of libraries, and 324MB total (libraries plus apps; see lists in /var/log/prelink/prelink.log.)  It took only a couple minutes.

Comment 6 Kevin Fenzi 2011-04-03 00:57:32 UTC
I've commited a change to run it at compose time... 

I'm not sure turning it off on the live copy is worth while. If it's already run at compose time, the amount of changes if it runs after that should be pretty small, right?

Comment 7 John Reiser 2011-04-03 01:21:23 UTC
(In reply to comment #6)
>  If it's already run at compose time, the amount of changes if it runs after
>  that should be pretty small, right?

No; each time prelink runs it assigns the addresses randomly.

Comment 8 Kevin Kofler 2011-04-03 05:07:16 UTC
I think this should be run with || : so you can still compose live images with

Comment 9 Bruno Wolff III 2011-04-03 13:43:49 UTC
I made the following changes.

I added the -m option which is needed in some cases for -i686.

I added &>/dev/null || : to the prelink commands.

I added a check to livesys to turn off running prelink using code from live-kde-base.

Comment 10 Jan Kratochvil 2011-04-03 13:55:21 UTC
Maybe you could just run `/etc/cron.daily/prelink'.  It will do the full prelink on initial run - as /etc/prelink.cache does not exist that time.
For example the default option -R has security benefits and it is present in `/etc/sysconfig/prelink' as used by `/etc/cron.daily/prelink'.

Comment 11 Bruno Wolff III 2011-04-03 21:36:19 UTC
I am not sure what the cost is of using -R, but the benefit seems low here, since the live images are public. If it's nearly free, it's reasonable to add it.

I am not sure how running cron really helps; we aren't planning to run prelink again later so worrying about caching seems like extra work.

Comment 12 Jan Kratochvil 2011-04-03 21:43:54 UTC
(In reply to comment #11)
> the benefit seems low here, since the live images are public.

oops, yes.

> If it's nearly free, it's reasonable to add it.

It may have some higher page tables cost, it should be negligible.

> I am not sure how running cron really helps; we aren't planning to run prelink
> again later so worrying about caching seems like extra work.

I do not talk about crond itself but about the script `/etc/cron.daily/prelink' which is prepared to be run by crond but one can run it even directly.

When -R is not useful it may no longer bring any real benefit, it creates a log which may be considered both positive or negative etc.

Comment 13 Kevin Kofler 2011-04-07 07:07:16 UTC
Uh, does prelink bloat the size of the live image? The latest nightly compose of the KDE spin came out oversized, with i686 having grown by almost 30 MiB between April 2 and April 3!

Comment 14 Kevin Kofler 2011-04-07 07:31:42 UTC
(But I'm not sure whether the size increase is really prelink's fault or whether it's just some strange stochastic variance. Unfortunately, there have been no nightly composes since April 3, so it's hard to tell.)

Comment 15 Kevin Kofler 2011-04-07 07:49:13 UTC
20110401.17 didn't use prelink, the KDE image sizes were 689 MiB for i386, 688 MiB for x86_64.
20110402.18 used prelink -a, the KDE image sizes were 689 MiB for i386, 703 MiB for x86_64. (Did prelink fail on i386 there?)
20110403.17 used prelink -am, the KDE image sizes were 713 MiB for i386, 705 MiB for x86_64.

The KDE spin cannot take such a size increase.

Comment 16 Kevin Kofler 2011-04-07 08:00:27 UTC
I committed these:

to drop the prelinking again. Sorry, but the size increase is really not acceptable for the KDE spin. It might make sense to prelink some spins, but forcing this in -base for everyone is a very bad idea.

Comment 17 Kevin Kofler 2011-04-07 09:23:31 UTC
To answer the obvious question:
Yes, now, I am sure it is prelink which causes this:
* The difference between i386 and x86_64 on April 2 is easily explained by the fact that prelink was run without -m, which doesn't always work on i386 according to the commit messages and thus the i386 image wasn't actually prelinked on April 2, it was on April 3.
* Considering the above, the size increases for i386 and x86_64 are easily correlated: 24 MiB for i386, 17 MiB for x86_64, that's in the same order of magnitude.
* I also looked at the differences in content between April 2 and 3 for the i386 spin. There are no added packages, only updated ones, and the only package I considered suspicious at first (lovelock-backgrounds-single) has hardly changed in size.

Comment 18 Bruno Wolff III 2011-04-07 11:46:03 UTC
For the other spins, I think we want to be careful about doing changes right around the beta release, but I agree the size increase is a problem.
I'd like to find out if the size increase is due to small amounts of increase per binary or library or if there are some scratch files being left behind that could be cleaned up.

Comment 19 Kevin Kofler 2011-04-07 11:58:04 UTC
There are several things which could cause size increases:
* the fact that the relocated addresses don't compress so well,
* the undo information needed to support prelink -u,
* /etc/prelink.cache,
* from the manpage: "It also stores a list of all dependent libraries together with their checksums into the binary or library."
* also from the manpage: "For binaries, it also computes a list of conflicts (relocations that resolve differently in the binary's symbol search scope than in  the smaller search scope in which the dependent library was resolved) and stores it into a special ELF section."

Comment 20 Jan Kratochvil 2011-04-07 12:07:50 UTC
On i686 prelink sometimes converts relocations REL -> RELA (thus being larger).
Yes, i686 should be abandoned.

Comment 21 John Reiser 2011-04-07 14:38:42 UTC
Please remember the other part of my original report: disable /etc/cron.daily/prelink so that it does not run by default after booting a Live spin.  The cron job is an unexpected and noticeable disruption (interactivity slows significantly), and it eats memory (tmpfs) on a Live system: about 340MB of RAM to store the results.  This is very bad on a minimum system with 512MB RAM and still unreasonable for 1GB RAM.

Comment 22 Kevin Kofler 2011-04-07 15:25:51 UTC
That part (disabling prelink runs on the uninstalled Live system) has been implemented and is not controversial

Re comment #20 (Jan Kratochvil): That could explain the extra 7 MiB penalty (for a total of 24 MiB) on i686, but even on x86_64 we have a 17 MiB penalty.

Comment 23 John Reiser 2011-04-08 03:34:36 UTC
A prelink of just the shared libraries that are used most often can reap measurable benefits in speed at small cost in space (usually a couple Kbytes per library.)  One likely list is (sorted by most important first):
   libtinfo.so.5   [for bash]
   libselinux.so.1, libacl.so.1, libattr.so.1  [for mv/cp/ls]
Even prelinking just the first three (ld-linux, libc, libgcc_s) would help a lot.

Comment 24 Jan Kratochvil 2011-04-08 15:02:31 UTC
See LD_DEBUG=all.  If any of the executable or any of its dynamically linked libraries is not properly up to date with valid checksum the whole prelinked data are dropped and the normal ld.so dynamic linking process starts.

Comment 25 Kevin Kofler 2011-04-08 16:50:39 UTC
After my commits from comment #16 (i.e. with prelink not run anymore), the latest live composes (Nightly-20110407.23, F15 Beta RC1) are back to the original size (690 MiB ± 1 MiB for fedora-livecd-kde), so it was really prelink causing the bloat, not some other change.

Comment 26 Fedora End Of Life 2012-08-07 19:53:44 UTC
This message is a notice that Fedora 15 is now at end of life. Fedora
has stopped maintaining and issuing updates for Fedora 15. It is
Fedora's policy to close all bug reports from releases that are no
longer maintained. At this time, all open bugs with a Fedora 'version'
of '15' have been closed as WONTFIX.

(Please note: Our normal process is to give advanced warning of this
occurring, but we forgot to do that. A thousand apologies.)

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, feel free to reopen
this bug and simply change the 'version' to a later Fedora version.

Bug Reporter: Thank you for reporting this issue and we are sorry that
we were unable to fix it before Fedora 15 reached end of life. If you
would still like to see this bug fixed and are able to reproduce it
against a later version of Fedora, you are encouraged to click on
"Clone This Bug" (top right of this page) and open it against that
version of Fedora.

Although we aim to fix as many bugs as possible during every release's
lifetime, sometimes those efforts are overtaken by events. Often a
more recent Fedora release includes newer upstream software that fixes
bugs or makes them obsolete.

The process we are following is described here:

Note You need to log in before you can comment on or make changes to this bug.