Bug 248082
Summary: | anaconda liveinst livecd installer is inefficient and doesn't support barely large enough destination volumes | ||
---|---|---|---|
Product: | [Fedora] Fedora | Reporter: | Jane Dogalt <jdogalt> |
Component: | LiveCD | Assignee: | David Zeuthen <davidz> |
Status: | CLOSED WONTFIX | QA Contact: | |
Severity: | low | Docs Contact: | |
Priority: | low | ||
Version: | 7 | CC: | dcantrell, katzj, mclasen |
Target Milestone: | --- | ||
Target Release: | --- | ||
Hardware: | All | ||
OS: | Linux | ||
Whiteboard: | |||
Fixed In Version: | Doc Type: | Bug Fix | |
Doc Text: | Story Points: | --- | |
Clone Of: | Environment: | ||
Last Closed: | 2007-07-16 20:00:10 UTC | Type: | --- |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: |
Description
Jane Dogalt
2007-07-13 03:49:42 UTC
I haven't quite got around to fixing this. I got distracted and had to fix that redundant rootfs format bug first. Since I went to the trouble of collecting this data, which is also relevant for that bug as well, I'll go ahead and post it here. I performed 5 installs onto my vaio vgn-n250 laptop (core duo, sata laptop drive (hdparm -t == 40 MB/s). The flash used was 8.5MB/s. cdrom used is 1.6MB/s. 3 live isos were used. stock f7, replicated f7 (with livecd tools), and replicated f7 with a 2.5G container instead of the default 4.0G container. about 2075MB of data within the filesystem. each test was also performed after doing livecd-iso-to-usb. (thus the 6(-1) tests). I measured the time the user waits for the install, this being 3 phases- 1) format rootfs 2) copy live image to drive 3) postinst stuff I did not do a stock-f7 usb test, as mainly I used the stock-f7 cd test to verify that my respun numbers were not off base. 30G rootfs was used in all tests. respin-flash-2.5G format 41s copy 99s postinst 75s respin-flash-4.0G format 41s copy 131s postinst 75s respin-cdrom-2.5G format 49s copy 262s postinst 89s respin-cdrom-4.0g format 49s copy 290s postinst 89s stockf7-cdrom-4.0G format 47s copy 291s postinst 89s conclusions ----------- copy benefit interpolated up by ~25% to factor for optimal case, i.e. 4.0G->2.1G, rather than 4.0G->2.5G for flash case, overall 89+75=164 vs 41+131+75=247, 83/164= "F7 took 50% longer to install than it could have" 41s saved from format, 42s saved from copy for cdrom 253+89= 342 vs 49+290+89=428, 86/342= "F7 took 25% longer to install than it could have" 49s saved from format, 37s saved from copy I mentioned on the fedora-livecd-list to markmc that my solution involving on-the-fly resize2fs-ing down to minimal isn't going to work. Not only is there the issue of ram usage, the fatal flaw with that theory is that the resize2fs-minimization process will likely(??) require lots of reads hitting the install media(e.g. cdrom), which I suspect will be atrocious. The final (I say a bit prematurely) solution then hit me- During livecd-creator compose time 1) after done composing the 4.0G os.img as happens now, do a resize2fs down to minimal, truncate the file to minimal, resparsify it back to 4.0G. This step by itself, while time consuming (make it a flag to livecd-creator if this is an issue), is useful in that it will remove any data taken up by files that were written during the install and then deleted for some reason (I'm guessing there are at least a few). 2) now, still during livecd-creator compose, snapshot the image, overlay file on disk, and run a resize2fs minimize pass. Then (here is the magic) include that snapshot file (which by all rights should be very small, if not, a pretty bad bug with resize2fs) on the cdrom alongside os.img. 3) now, during anaconda liveinst time, recreate the snapshot device, thus you have access to a minimal os.img for use in copying efficiently to drive. If for some reason having that overlay file on cdrom slows things down, read it into /dev/shm(tmpfs) first. This design does hinge on the overlay file being small, and the snapshot device not imposing a serious penalty. But my fairly strong hunch is that that will be the case. oops, forgot in step 1, after resparsifying back to 4.0G, resize2fs back to 4.0G as well. I have to wonder a bit if this is getting to the point of over-optimization -- the cost of doing all of the resizing + snapshotting is going to be pretty large. And being able to build live CD fast is going to be increasingly important. For cases where you really care about making this level of optimization, we already let you specify the size of the rootfs image and so that's going to likely be the better approach. I disagree. I think the time measurements show the benefits. All the resizing + snapshotting happens at livecd creator time, which can be a flag, so that it needs only be done for mass shipping spins. The snapshot does get _used_ at install time, but I suspect it won't be that bad. Anyway, I may still produce a patch and do some tests, and post. Just setting the size at creation time, isn't an answer I think. I think you need the +2G space, such that livecd users with 2G of ram are allowed to utilize it. Actually, the issues with that are also related to issues with the currently undefined persistence livecd wishlist feature. I.e. do we support persistance to an 8GB usbstick? If so, how does that get implemented? (though that discussion is I think for fedora-livecd-list, and not this bug) Douglas McClendon :) posted a patch that takes care of much of this, as well as reclaiming 5.5% wasted space on the F7-i686-LiveCD. Please review that patch ASAP. If it looks like it will get applied more or less as is, I'll go forward implementing the rest of the above, hopefully in time for the F8T1 devel freeze. I can implement in such a way that it is easily verifiable that by default, nothing will happen differently, either with the boot sequence, or anaconda. If however a boot arg is given (e.g. by f8t1 testers actively testing this feature/bugfix), then a dm-linear layer will be inserted on top of the os.img loop device (so it can be simultaneously used as part of the live-rw-rootfs, and as part of the minimized-installable-rootfs), and anaconda will take advantage of it, and the <<1MB extra minimized-fs-overlay file provided by livecd-creator will be utilized. As always, a little more mind-bending thought with devicemapper, and I realize that no bootarg or futzing with the existing boot sequence will be needed. I had been happy to discover that dm allows you to use a single device to create multiple mountable devices. (i.e. that rebootless theoretical workaround #5...). But what I just realized, is that because the loop device associated with os.img, is _already_ a component of devicemapper in the livecd scenario (i.e. a part of the snapshotted /dev/mapper/live-rw), that it can be reused to build the minimized image at runtime (without any extra stuff happening during early boot) |