Bug 476696 - RFE: Allow larger spins by using udf
Summary: RFE: Allow larger spins by using udf
Keywords:
Status: CLOSED NEXTRELEASE
Alias: None
Product: Fedora
Classification: Fedora
Component: livecd-tools
Version: rawhide
Hardware: All
OS: Linux
low
medium
Target Milestone: ---
Assignee: Jeremy Katz
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: F11Target
TreeView+ depends on / blocked
 
Reported: 2008-12-16 16:01 UTC by Bruno Wolff III
Modified: 2009-01-10 04:53 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2009-01-08 21:54:11 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)
A patch that implements the requested feature (2.78 KB, patch)
2008-12-16 16:01 UTC, Bruno Wolff III
no flags Details | Diff
Updated patch file (3.36 KB, patch)
2009-01-06 15:55 UTC, Bruno Wolff III
no flags Details | Diff
F11 version of the patch (3.40 KB, patch)
2009-01-08 12:53 UTC, Bruno Wolff III
no flags Details | Diff

Description Bruno Wolff III 2008-12-16 16:01:58 UTC
Created attachment 327118 [details]
A patch that implements the requested feature

Description of problem:
I have been working on the games spin and it is currently extremely close to the current 4GiB limit. However the smallest standard size DVD is supposed to be able to hold 4.7GB (this is the lowest maximum size allowed by the various types of DVD formats) which would add about 400MB of compressed space to what is currently available. Also with more people making live usbs there is going to be demand for significantly larger spins in the future.
One way to provide for this feature is to use udf format. I have tested this with the games spin with a couple of extra games that put it over the 4GiB limit and the system booted and testing a couple of games seemed to work.
While there may be some cases where using udf will prevent people from copying the squashfs file system over on other OS's, this doesn't seem like it would be common. I am not an expert on how widely udf is supported on other OS's, so I might be misjudging this. I expect the only thing we might care about is the windows version of the livecd to usb tool.
The test implementation I used for this added the -allow-limited-size option to the mkisofs call (which according to the man page implies the -ufs option), changed the root file system to udf and added udf to the list of modules to include. (I am not completely sure if I did the last correctly, but it seemed to work.)
If making livecd-tools always use udf is a problem, perhaps an option could be added to allow one to request it.

Version-Release number of selected component (if applicable):
The patch is against livecd-tools-020-1.fc10.i386. Though the feature request is for rawhide.

How reproducible:


Steps to Reproduce:
1.
2.
3.
  
Actual results:


Expected results:


Additional info:

Comment 1 Bruno Wolff III 2008-12-17 12:22:58 UTC
I got some feedback on this and it was suggested that rather than have livecd-creator just use udf, that it detect that the squashfs file was 4GB or greater in size and then switch to udf and emit a message indicating that.
I don't really know python yet, but I'll attempt to rework my patch and see if I can get something to work which can be reviewed.

Comment 2 Jeremy Katz 2008-12-19 18:48:43 UTC
Yeah, I think we're less likely to trigger problems that way.  As I said on the list, it should be pretty straight-forward.  If you run into roadblocks, let me know.

And I'm going to stick this onto the list of things for F11

Comment 3 Bruno Wolff III 2009-01-05 06:17:28 UTC
I haven't got as far with this as I would have liked. But I do have some notes for how to approach it that I want saved.

There is a potential issue of timing with checking the size of the mksquashfs image and the boot image. If the boot image is built before the mksquashfs image (or the ext3 image if no compression is requested) then the file system is type is already saved before we know which one to use. Rather than syncronizing their creation my idea is to use a root file system type of udf,iso9660. If the image is not udf and is iso9660 then the iso9660 driver should get used. If the image is both iso9660 and udf (with iso9660 lying about the live image size), then I am not sure that putting udf first guaranties that udf will be used in preference. However if it doesn't, then we should be able to specify an order with /etc/filesystems.

Assuming that the above will work, then I can use os.walk to walk the directory that is going to be passed to mkisofs and see if any of the files is 4 GiB or larger using os.stat.st_size and if so add the -allow-limited-size to the arguments.

Assuming the file system type checking order can be set to test for udf before iso9660, doing the change the above way seems pretty safe. I don't have to worry about timing of creations of live image versus other things that need to be in the iso file. Nor do I need to worry about the exact name of the live image, so that future additions won't break the test.

I still plan on working up a patch to implement the above approach. I also want to see if I can determine if the order in which file system types are list after the -t option, are the order in which they are checked. So far I haven't found anything using google, so I'll take a look at the source for mount and see what it does.

Comment 4 Bruno Wolff III 2009-01-05 06:55:49 UTC
I looked through the mount source code. While I don't understand it fully, I only found one place where it dealt with a comma separated list of types and it tried doing a mount of each type in order. So while this behavior isn't promised it looks like it will work. And I expect it is unlikely to change.

Comment 5 Bruno Wolff III 2009-01-06 15:55:19 UTC
Created attachment 328289 [details]
Updated patch file

The updated patch should outline my approach and might even work. I keep making types in the logging statement and it takes a couple of hours to hit them. So I didn't get the patch as tested as I would have liked before posting. I'll do more tonight, but I wanted to get it out there early to get feedback on the approach. One incarnation (it had a typo, but the typo didn't cause the build to fail) built an iso file. I didn't get a chance to burn it to make sure it booted though.

Comment 6 Bruno Wolff III 2009-01-07 15:14:56 UTC
I tested the F10 version of the games spin and it booted.
F11 has dependency problems with poker2d right now and I won't be able to try out the full games spin for it until the issue is fixed.
I will try out a smaller (< 4GiB) for F10 tonight (and probably for F11 as well).
The main thing I wanted to make sure worked (specifying two file system types on the rootfstype parameter) seems to be working.
Note that the patch applies cleanly to the F10 version of live.py. In F11 there was a change to the statements my patch changes so the patch will fail. But it's clear what to do (change iso9660 to udf,iso9660).
I'd like to get this change into F11 soon. I don't want to ask for a change to fedora-livedvd-games.ks until after live.py has been updated, since it won't work otherwise. And I'd like to get fedora-livedvd-games.ks updated soon so that there is a chance to get a sample games spin out for the alpha.

Comment 7 Bruno Wolff III 2009-01-07 16:03:55 UTC
Building and burning the live desktop spin went fast enough that I was able to test it this morning and it booted. So it looks like using a rootfstype of udf,iso9660 works for both udf and iso9660 images.

Comment 8 Bruno Wolff III 2009-01-08 12:53:44 UTC
Created attachment 328461 [details]
F11 version of the patch

I have attached a version of the patch that will apply cleanly to the F11 version of live.py. Otherwise the patch is essentially the same as the previously posted patch.

Comment 9 Jeremy Katz 2009-01-08 21:54:11 UTC
Applied to master, thanks!  And I'll either get a build done for the alpha freeze or help ensure that we build the images with what's in git currently.

Comment 10 Bruno Wolff III 2009-01-09 19:16:51 UTC
This is FYI related to this feature. It has been suggested because vfat is limited to 4 GiB for file sizes, that I get the iso image size under this for the games spin. Considering the target audience, this seems like a reasonable request. So by the time the alpha spin is made I'll probably not require this feature for the published games spin.
However, I do want the feature for myself when making custom spins. So I will continue to test it with a private version of the games spin in parallel.

Comment 11 Bruno Wolff III 2009-01-10 04:53:09 UTC
More feedback has suggested that few people will be using vfat except for usb drives which probably isn't a big deal for spins meant for DVDs. So I may or may not have the games spin slightly over 4 GiB for the alpha.


Note You need to log in before you can comment on or make changes to this bug.