Bug 448030 - RFE: create a bootable Live USB stick from the running livecd
RFE: create a bootable Live USB stick from the running livecd
Status: CLOSED ERRATA
Product: Fedora
Classification: Fedora
Component: livecd-tools (Show other bugs)
rawhide
All Linux
low Severity low
: ---
: ---
Assigned To: Brian Lane
Fedora Extras Quality Assurance
: FutureFeature
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2008-05-22 21:21 EDT by Rahul Sundaram
Modified: 2013-03-13 01:43 EDT (History)
9 users (show)

See Also:
Fixed In Version: livecd-tools-15.5-1.fc15
Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2011-03-11 23:39:38 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:


Attachments (Terms of Use)
a modified livecd-iso-to-disk script (32.24 KB, application/octet-stream)
2010-04-06 13:48 EDT, Frederick Grose
no flags Details
modified_livecd-iso-to-disk (37.46 KB, text/plain)
2010-04-13 02:48 EDT, Frederick Grose
no flags Details
help file (8.34 KB, text/plain)
2010-04-13 02:51 EDT, Frederick Grose
no flags Details
Add --copy-overlay & --copy-home options (46.27 KB, patch)
2010-07-18 02:13 EDT, Frederick Grose
no flags Details | Diff
man page changes for the copy.patch (13.10 KB, patch)
2010-07-18 02:15 EDT, Frederick Grose
no flags Details | Diff
Add --builder-message & --releasefile options (2.63 KB, patch)
2010-07-18 02:18 EDT, Frederick Grose
no flags Details | Diff
man page changes for the builder & releasefile patches (1.93 KB, patch)
2010-07-18 02:20 EDT, Frederick Grose
no flags Details | Diff
add --help option to the livecd-iso-to-disk script (26.55 KB, patch)
2010-07-18 02:22 EDT, Frederick Grose
no flags Details | Diff
man page for all help file changes (14.05 KB, patch)
2010-07-18 02:32 EDT, Frederick Grose
no flags Details | Diff
The proposed, new help file complete (11.33 KB, text/plain)
2010-07-18 02:35 EDT, Frederick Grose
no flags Details
combined changes to livecd-iso-to-disk.sh (44.31 KB, text/plain)
2010-07-19 16:51 EDT, Frederick Grose
no flags Details
changes to edit-livecd to permit cloning of live image (17.27 KB, text/plain)
2010-08-13 20:03 EDT, Frederick Grose
no flags Details
incomplete edit-liveos version of edit-livecd (18.22 KB, text/plain)
2010-08-13 20:05 EDT, Frederick Grose
no flags Details
Rename source- and target-related variables (28.28 KB, patch)
2010-12-05 21:59 EST, Frederick Grose
no flags Details | Diff
Add support for Live image propagation from installed sources (7.63 KB, patch)
2010-12-05 22:02 EST, Frederick Grose
no flags Details | Diff
Update usage manual & add it to the script (29.30 KB, patch)
2010-12-05 22:10 EST, Frederick Grose
no flags Details | Diff
alpha version of edit-livecd variant (21.40 KB, text/x-python)
2011-01-28 04:19 EST, Frederick Grose
no flags Details
changes to edit-livecd to permit cloning of live images (26.00 KB, patch)
2011-02-08 03:36 EST, Frederick Grose
no flags Details | Diff
Enable reading of SquashFS compression type. (4.28 KB, patch)
2011-02-08 03:41 EST, Frederick Grose
no flags Details | Diff
late alpha version of edit-liveos (22.27 KB, text/plain)
2011-02-08 03:48 EST, Frederick Grose
no flags Details
Support attached LiveOS devices, adjust iso naming, restore script option (9.17 KB, patch)
2011-02-15 22:43 EST, Frederick Grose
no flags Details | Diff
Enable live-mounting of LiveOS device together with an active overlay (12.80 KB, patch)
2011-03-17 00:39 EDT, Frederick Grose
no flags Details | Diff
Support usage within a booted LiveOS instance (10.08 KB, patch)
2011-04-14 11:20 EDT, Frederick Grose
no flags Details | Diff
Support refreshing of attached LiveOS devices (51.52 KB, patch)
2011-04-14 11:26 EDT, Frederick Grose
no flags Details | Diff
Beta version of editliveos.py (47.45 KB, text/plain)
2011-04-14 11:33 EDT, Frederick Grose
no flags Details
Add --copy-home and --include options to livecd-iso-to-disk (10.99 KB, patch)
2011-04-14 11:36 EDT, Frederick Grose
no flags Details | Diff
Document --copy-home and --include options (2.15 KB, patch)
2011-04-14 11:37 EDT, Frederick Grose
no flags Details | Diff

  None (edit)
Description Rahul Sundaram 2008-05-22 21:21:50 EDT
Description of problem:

This would be a very nice enhancement. Refer http://klik.atekon.de/liveusb/ for
a potential implementation.
Comment 1 Jeremy Katz 2008-05-22 22:09:50 EDT
This actually would work today in the basic case --  just livecd-iso-to-disk
/dev/live /path/to/usb/stick.  Doing more would be more suited for Luke's tool
rather than the main livecd-tools
Comment 2 Sebastian Dziallas 2009-08-22 11:17:00 EDT
/me seconds the request.

Jeremy, is this still supposed to work? I can't seem to get it to work, as livecd-iso-to-disk just displays the help.
Comment 3 Jeremy Katz 2009-08-24 11:17:28 EDT
Can you run with sh -x and show how it fails?

(although again, liveusb-creator should be preferred -- not the shell script)
Comment 4 Luke Macken 2010-04-05 22:56:10 EDT
From what I remember, we did not want to include the liveusb-creator on the live images due to it pulling in the fairly large PyQt.  Since it looks like we're not longer targeting normal CD-R's for Fedora 13, we could potentially add the liveusb-creator to do the default image.
Comment 5 Frederick Grose 2010-04-06 13:04:38 EDT
Please see http://lists.sugarlabs.org/archive/soas/2010-April/000998.html
for a method based on a modified livecd-iso-to-disk.

Review and testing needed.

(Re: http://bugs.sugarlabs.org/ticket/74)
Comment 6 Frederick Grose 2010-04-06 13:48:35 EDT
Created attachment 404761 [details]
a modified livecd-iso-to-disk script
Comment 7 Frederick Grose 2010-04-06 13:53:38 EDT
See also comments in this post to the livecd list:
http://lists.fedoraproject.org/pipermail/livecd/2010-April/005763.html
Comment 8 Frederick Grose 2010-04-13 02:48:16 EDT
Created attachment 406157 [details]
modified_livecd-iso-to-disk

newer version with expanded usage sections and some bug fixes (will submit them as individual patches to tools_livecd-iso-to-disk.sh to make review easier.)
Comment 9 Frederick Grose 2010-04-13 02:51:31 EDT
Created attachment 406158 [details]
help file

This is an updated help file in .pod format.
Comment 10 Fedora Admin XMLRPC Client 2010-05-07 11:41:06 EDT
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.
Comment 11 Frederick Grose 2010-07-18 02:13:21 EDT
Created attachment 432663 [details]
Add --copy-overlay & --copy-home options

The --copy-overlay and --copy-home additions extend the utility of the script to allow the user/learner to replicate their personally-modified image.

There are many white-space changes that I couldn't avoid in order to clarify the local variable name changes made to match the broader scope of the script (for example, $ISO to $SRC and $USBDEV to $TGTDEV). In addition, there are some block order changes that make use and documentation easier to follow.

I also standardized the Bash conditional expression and list pipeline style to the more modern, [[ Expression ]] && [[ Expression ]] format. (See http://wiki.bash-hackers.org/syntax/ccmd/conditional_expression.)
This means that variables within the special double brackets need not be quoted and the old test-type operators -a -o should not be used.

I'm sorry that all the changes may make review more critical, but it was practically necessary.
Comment 12 Frederick Grose 2010-07-18 02:15:40 EDT
Created attachment 432664 [details]
man page changes for the copy.patch

man page changes to accompany the copy.patch
Comment 13 Frederick Grose 2010-07-18 02:18:07 EDT
Created attachment 432665 [details]
Add --builder-message & --releasefile options

The --builder-message and --releasefile options label the remixed image to prevent confusion with the pristine versions, and to credit the builder for their changes.
Comment 14 Frederick Grose 2010-07-18 02:20:32 EDT
Created attachment 432666 [details]
man page changes for the builder & releasefile patches

Man page patches to accompany the builder-message and releasefile patch.
Comment 15 Frederick Grose 2010-07-18 02:22:43 EDT
Created attachment 432667 [details]
add --help option to the livecd-iso-to-disk script

Because the livecd-iso-to-disk script is so often redistributed in standalone fashion, it is worth the space and effort to include usage documentation and help in the script itself.  This should help the general understanding of the script and its methods, given its presently-limited documentation.
Comment 16 Frederick Grose 2010-07-18 02:32:48 EDT
Created attachment 432669 [details]
man page for all help file changes

This is a patch to update the man page help file with all the changes.
Comment 17 Frederick Grose 2010-07-18 02:35:04 EDT
Created attachment 432670 [details]
The proposed, new help file complete

A complete version of the help file, livecd-iso-to-disk.pod
Comment 18 Frederick Grose 2010-07-18 02:43:40 EDT
Mailing list post for cross referencing list discussion,
http://lists.fedoraproject.org/pipermail/livecd/2010-July/005977.html
Comment 19 Frederick Grose 2010-07-19 16:51:56 EDT
Created attachment 432997 [details]
combined changes to livecd-iso-to-disk.sh

I've attached a full script with all of the proposed changes including some bug fixes, help message and code format refinements (especially on combined conditional expressions).

This should make smoke testing easier.
Comment 20 Frederick Grose 2010-08-13 20:03:39 EDT
Created attachment 438775 [details]
changes to edit-livecd to permit cloning of live image

This is an incomplete change to show work in progress.

The goal is to support cloning of an existing image along with resparsing the operating system so that the overlay can be reset and a livecd.iso file could be generated, if desired.

The attached edit-liveos and .diff files will show the approach and current state of development.

Due to bugs in the shutil.copytree() function, I've used rsync to copy the filesystem.  This seems fast and allows greater customization, if desired, by use of per directory .rsync-filter files.
Comment 21 Frederick Grose 2010-08-13 20:05:22 EDT
Created attachment 438777 [details]
incomplete edit-liveos version of edit-livecd

See previous comment.
Comment 22 Fedora Admin XMLRPC Client 2010-10-21 13:53:30 EDT
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.
Comment 23 Brian Lane 2010-11-09 17:08:57 EST
These changes are hard for me to follow, it would be easier to track if you posted patches of distinct changes, made against the master branch of the livecd tools git repository at http://git.fedorahosted.org/git/livecd
Comment 24 Frederick Grose 2010-12-05 21:59:20 EST
Created attachment 464912 [details]
Rename source- and target-related variables

Code variable name changes to remove ambiguity resulting from their new roles in supporting propagation of Live images from installed USB/SD devices.

See the following patches for the logic and documentation changes.
Comment 25 Frederick Grose 2010-12-05 22:02:33 EST
Created attachment 464913 [details]
Add support for Live image propagation from installed sources
Comment 26 Frederick Grose 2010-12-05 22:10:04 EST
Created attachment 464916 [details]
Update usage manual & add it to the script

The vastly expanded documentation is provided to support the propagation of images from an installed instance and because the script is distributed independently.

(The script options were reordered to support following the onboard documentation.)

Mailing list posts:

http://lists.fedoraproject.org/pipermail/livecd/2010-December/006542.html
http://lists.fedoraproject.org/pipermail/livecd/2010-December/006543.html
http://lists.fedoraproject.org/pipermail/livecd/2010-December/006544.html
Comment 27 Frederick Grose 2011-01-28 04:19:31 EST
Created attachment 475754 [details]
alpha version of edit-livecd variant

An alpha working version of a proposed replacement for edit-livecd.

This script allows one to rebuild a custom iso image from a running image by staging the build on an attached hard drive and using only the content already on the current image.

See this mailing list posting,
http://lists.fedoraproject.org/pipermail/livecd/2011-January/006616.html
for more details.

Review and testing would be appreciated.

I had to hack the system /tmp directory to bypass the hardwired use of /tmp in fs.py.resize2fs().  More robust alternatives would be appreciated.
Comment 28 Bruno Wolff III 2011-01-28 09:59:09 EST
I suggest that you add a way to specify compression type and switch the default type to xz to match livecd-creator. Several official live images are looking like they will be oversize without using xz. (If this is going to get included in older releases, gzip should remain the default in those releases.)
Comment 29 Bruno Wolff III 2011-01-28 10:05:29 EST
Another option for the default is to match the compression type used on the iso you are starting with. That probably makes even more sense than a fixed default. You can use unsquashfs -s to retrieve the compression type used for squashfs-tools 4.1 and 4.2; I am not sure about 4.0. (F13 has 4.0, F14 4.1 and F15 4.2.)
Comment 30 Frederick Grose 2011-02-08 03:36:58 EST
Created attachment 477571 [details]
changes to edit-livecd to permit cloning of live images

A late alpha version patch to edit-livecd to enable cloning of a running LiveOS image into a fresh iso.

Depends on pending patches to live.py and fs.py for squashfs_compression_type().
Comment 31 Frederick Grose 2011-02-08 03:41:35 EST
Created attachment 477572 [details]
Enable reading of SquashFS compression type.

Patch submitted to the mailing list:
http://lists.fedoraproject.org/pipermail/livecd/2011-February/006627.html

Support edit-livecd & livecd-creator base_on image options by
adding the capability in fs.py to read the base_on image
compression type.  Specifying a compressor type will override
the default; specifying 'None' will trigger reading and using
the base_on image's compression type.
Comment 32 Frederick Grose 2011-02-08 03:48:40 EST
Created attachment 477575 [details]
late alpha version of edit-liveos

This is the full edit-liveos.py script.

Note: the imports have temporary versions specified for fs, live, & creator.py due to the pending updates for squashfs_compression_type().
Comment 33 Fedora Update System 2011-02-15 19:23:55 EST
livecd-tools-15.4-1.fc15 has been submitted as an update for Fedora 15.
https://admin.fedoraproject.org/updates/livecd-tools-15.4-1.fc15
Comment 34 Fedora Update System 2011-02-15 20:30:18 EST
livecd-tools-15.4-1.fc15 has been pushed to the Fedora 15 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update livecd-tools'.  You can provide feedback for this update here: https://admin.fedoraproject.org/updates/livecd-tools-15.4-1.fc15
Comment 35 Frederick Grose 2011-02-15 22:43:13 EST
Created attachment 479013 [details]
Support attached LiveOS devices, adjust iso naming, restore script option

Wasn't expecting the previous commit, so here is a catchup patch:

Support attached LiveOS devices as well as image files for LiveOS editing.
Adjust naming of iso, log the iso name, adjust usage text, restore script
option.
Comment 36 Fedora Update System 2011-02-18 19:52:42 EST
livecd-tools-15.5-1.fc15 has been submitted as an update for Fedora 15.
https://admin.fedoraproject.org/updates/livecd-tools-15.5-1.fc15
Comment 37 Fedora Update System 2011-03-11 23:38:58 EST
livecd-tools-15.5-1.fc15 has been pushed to the Fedora 15 stable repository.  If problems still persist, please make note of it in this bug report.
Comment 38 Frederick Grose 2011-03-17 00:39:09 EDT
Created attachment 485909 [details]
Enable live-mounting of LiveOS device together with an active overlay

A standalone tool to help in working with attached LiveOS-bearing partitions.
Comment 39 Frederick Grose 2011-04-14 11:20:19 EDT
Created attachment 492141 [details]
Support usage within a booted LiveOS instance

Provided functions to add new loop devices (and remove them before exit) to support using this script in a running LiveOS instance.
Comment 40 Frederick Grose 2011-04-14 11:26:44 EDT
Created attachment 492146 [details]
Support refreshing of attached LiveOS devices

Added features to support refreshing an attached LiveOS source image with a renewed rootfs and freshed overlay and, optionally, skip building a new .iso file.

Still need code for refreshing a running LiveOS instance (by switching the rootfs).
Comment 41 Frederick Grose 2011-04-14 11:33:25 EDT
Created attachment 492152 [details]
Beta version of editliveos.py

Full beta version of editliveos.py.

Still lacking code to support refreshing of running LiveOS image (by switching the
rootfs back and forth).

Propose that the filename be changed from edit-livecd to editliveos.py to reflect its broader capability and support importing in Python interactive.
Comment 42 Frederick Grose 2011-04-14 11:36:30 EDT
Created attachment 492153 [details]
Add --copy-home and  --include options to livecd-iso-to-disk

Support installation of customized builds with --copy-home and --include options.

This complements new editliveos.py capabilities.
Comment 43 Frederick Grose 2011-04-14 11:37:59 EDT
Created attachment 492154 [details]
Document --copy-home and --include options

Document --copy-home and --include options for manual.

Note You need to log in before you can comment on or make changes to this bug.