Bug 1105735

Summary: No persistence on Live USB
Product: [Fedora] Fedora EPEL Reporter: Todd <ToddAndMargo>
Component: livecd-toolsAssignee: Brian Lane <bcl>
Status: CLOSED WONTFIX QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: high Docs Contact:
Priority: unspecified    
Version: el6CC: apevec, bcl, fgrose, katzj, metherid, ToddAndMargo, vanmeeuwen+fedora
Target Milestone: ---Keywords: Reopened
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-06-15 19:23:12 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Todd 2014-06-06 21:19:14 UTC
Hi All,

Host machine: Scientific Linux 6.5, 64 bit

# rpm -qa \*live\*
liveusb-creator-3.12.0-1.el6.noarch
livecd-tools-13.4.5-1.el6.x86_64


"livecd-iso-to-disk" does not give "persistence".  Boot up, put something on the desktop, reboot, it is gone.

Note that persistence works fine with "liveusb-creator" (the GUI), but that you can not get more than 4 GB of persistence (a bug for someone else).

I presume this problem is because "livecd-iso-to-disk" is so horribly out-of-date (version 13 versus version 21).


~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

From the host machine ("#" is root's prompt):

The USB stick after being erase and repartitioned:
# fdisk -l /dev/sdc
...
   Device Boot      Start         End      Blocks   Id  System
/dev/sdc1              85       71954     8049440   83  Linux


The creation command:
# livecd-iso-to-disk --reset-mbr --format --overlay-size-mb 6624 --unencrypted-home ./Fedora-Live-Xfce-x86_64-20-1.iso /dev/sdc



~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

From the Live USB with missing persistence:

# dmsetup status
live-base: 0 6291456 linear
live-osimg-min: 0 6291456 snapshot 1888/1888 16
live-rw: 0 6291456 snapshot 42352/1048576 176

# dmsetup table
live-base: 0 6291456 linear 7:3 0
live-osimg-min: 0 6291456 snapshot 7:3 7:1 P 8
live-rw: 0 6291456 snapshot 7:3 7:4 P 8


# losetup -a
/dev/loop0: [0001]:644 (/osmin.img (deleted))
/dev/loop1: [1792]:1 (/osmin)
/dev/loop2: [64513]:398274 (/run/initramfs/live/LiveOS/squashfs.img)
/dev/loop3: [1794]:2 (/LiveOS/ext3fs.img)
/dev/loop4: [0001]:10662 (/overlay (deleted))

# ls -lA /run/initramfs/live/LiveOS/
total 595580
-r--r--r--. 1 root root       8192 Jun  6 16:00 osmin.img
-rw-r--r--. 1 root root 6945767424 Jun  6 16:01 overlay-LIVE-c712eef2-2e8b-4d04-b5a8-3361566b8efd
-r--r--r--. 1 root root  609263616 Jun  6 16:00 squashfs.img

Comment 1 Brian Lane 2014-10-09 19:10:28 UTC
Is this still a problem?

Comment 2 Todd 2014-10-09 20:04:56 UTC
I can get persistence with ext4 and "Fedora Live USB Creator" but not
from the command.  So I haven't looked at the command line for a while.  Has there been any changes that you would like me to test?

An off topic remark (hopefully you will forgive me): I have found that I have to have "Legacy USB" enabled in the BIOS for flash drives created with "Fedora Live USB Creator" to boot.  And, have the BIOS updated to the most recent version.  Most new Supermicro motherboards won't boot until you update their BIOS.

Comment 3 Brian Lane 2015-06-08 18:04:31 UTC
*** Bug 1220015 has been marked as a duplicate of this bug. ***

Comment 4 Brian Lane 2015-06-08 19:28:18 UTC
You need to pass --home-size-mb not --overlay-size-mb and it should work.

Also, livecd-iso-to-disk is generally backwards compatible with older releases, but it isn't always forward compatible further than n+1, so making a live usb of a fedora release > f14 from a el6 system isn't necessarily going to work -- you may get lucky, but really you should use a newer release to make a newer usb stick.

Comment 5 Todd 2015-06-08 21:36:23 UTC
Reopening.  

In attempting to verify this was corrected, one more bug popped up to fix.  Possibly a second.  The first bug block my ability to verify that this bug was indeed corrected.

You may wish to adjust the Title of this bug to reflect the new problem(s).


Bug #1:
# livecd-iso-to-disk --home-size-mb 13038  Fedora-Live-Xfce-x86_64-21-5.iso /dev/sdc1

This one blocked my ability to verify to "--home-size-mb" from "--overlay-size-mb" solved the original issue.

When running this iso directly (KVM), it auto logs in without a password.  Root also has no password.  The Live USB created prompts for a username and password.  "liveuser" and "root" do not work.  Using the password prompted for the home directory also does not work.  (No LUKS prompt occurred for /home when the stick booted either.)  <ctrl><alt><f2> and trying to log in with the command line did not work with either user.

Bug #2 (perhaps): 

Unable to do a "df" to verify "--home-size-mb" corrected the original issue

Comment 6 Brian Lane 2015-06-08 22:09:17 UTC
If there is a different issue please open a new bug -- unless you are still trying to do this from the el6 version which isn't guaranteed (or really expected) to work with a F21 iso.

If you are having problems with df that's a spin issue, although I can't see how it would get left out of an image.

Comment 7 Todd 2015-06-09 00:37:47 UTC
(In reply to Brian Lane from comment #6)
> If there is a different issue please open a new bug -- unless you are still
> trying to do this from the el6 

EL6 is supported until the year 2020, so it is support to work for at least the next 4-1/2 years.

> version which isn't guaranteed (or really expected) to work with a F21 iso.

So FC12 that EL6 was originally based on?

What does the base OS have anything to do with copying and configuring an ISO?  That is the program's (livecd-iso-to-disk) doing.
 
> If you are having problems with df that's a spin issue, although I can't see
> how it would get left out of an image.

You could always try it yourself.  My stick was 16 GB.

Do you still want me to open a new bug or will this one do?

Many thanks,
-T

Comment 8 Brian Lane 2015-06-09 15:25:21 UTC
EL6 is based on F13, but just because RHEL6 is supported for the long term doesn't mean that every possible permutation of use is supported. New releases have different ways of doing things, different file locations, different kernel flags, etc.

If you have different bugs, file them with full details and logs.

As far as I can tell *this* bug was user error. I've use the rhel6 litd to create a f20 live iso with persistent home and confirmed that it boots.

Comment 9 Todd 2015-06-09 17:15:06 UTC
(In reply to Brian Lane from comment #8)

> 
> As far as I can tell *this* bug was user error. I've use the rhel6 litd to
> create a f20 live iso with persistent home and confirmed that it boots.

1) it boots.  That was not the issue.  It won't go past the password prompt

2) the LUKS password for /home is never asked for.

I will re-post as a new bug

Comment 10 Todd 2015-06-09 17:26:26 UTC
(In reply to Brian Lane from comment #8)

> As far as I can tell *this* bug was user error. I've use the rhel6 litd to
> create a f20 live iso with persistent home and confirmed that it boots.

You can not tell that unless you recreate with the same ISO I used (Fedora-Live-Xfce-x86_64-21-5.iso)

Comment 11 Frederick Grose 2015-06-11 01:52:57 UTC
Persistence confirmed with Fedora-Live-Xfce-x86_64-21-5.iso installed as Live USB (formatted as ext4) with the onboard livecd-iso-to-disk script.

curl -OC - http://dl.fedoraproject.org/pub/fedora/linux/releases/21/Live/x86_64/Fedora-Live-Xfce-x86_64-21-5.iso
...
sudo mount Fedora-Live-Xfce-x86_64-21-5.iso /mnt/
mount: /dev/loop0 is write-protected, mounting read-only
sudo /mnt/LiveOS/livecd-iso-to-disk --reset-mbr --overlay-size-mb 6624 --unencrypted-home /dev/loop0 /dev/sdb1

(Note: --unencrypted-home is a secondary option to --home-size-mb <size> and does not have any effect without a --home-size-mb request.)  I left this option in the command line for testing purposes.

[liveuser@localhost ~]$ sudo dmsetup status
live-base: 0 6291456 linear 
live-osimg-min: 0 6291456 snapshot 1960/1960 16
live-rw: 0 6291456 snapshot 79440/13565952 320
[liveuser@localhost ~]$ sudo losetup -l
NAME       SIZELIMIT OFFSET AUTOCLEAR RO BACK-FILE
/dev/loop0         0      0         0  1 /osmin.img (deleted)
/dev/loop1         0      0         0  1 /osmin
/dev/loop2         0      0         0  1 /run/initramfs/live/LiveOS/squashfs.img
/dev/loop3         0      0         0  1 /LiveOS/ext3fs.img
/dev/loop4         0      0         0  0 /LiveOS/overlay-LIVE-e2f445aa-eb8b-4b9a-acdb-8d1bf0449849
[liveuser@localhost ~]$ sudo df -T
Filesystem          Type     1K-blocks     Used Available Use% Mounted on
/dev/mapper/live-rw ext4       3030800  2437268    545696  82% /
devtmpfs            devtmpfs   2003672        0   2003672   0% /dev
tmpfs               tmpfs      2023860       72   2023788   1% /dev/shm
tmpfs               tmpfs      2023860     1064   2022796   1% /run
tmpfs               tmpfs      2023860        0   2023860   0% /sys/fs/cgroup
/dev/sdb1           ext4      15299276  1347440  13151620  10% /run/initramfs/live
tmpfs               tmpfs      2023860       12   2023848   1% /tmp
varcacheyum         tmpfs      2023860        0   2023860   0% /var/cache/yum
vartmp              tmpfs      2023860       12   2023848   1% /var/tmp
tmpfs               tmpfs       404776        8    404768   1% /run/user/1000

I copied an application icon on the desktop on the first boot and it was still there on the second.

[liveuser@localhost ~]$ ls -lA /home/liveuser/Desktop/
total 8
-rw-r--r--. 1 liveuser liveuser 5837 Aug 19  2014 xfce4-screenshooter.desktop

Comment 12 Todd 2015-06-11 03:59:34 UTC
Very cool.  I get told I have to use fat32 or ext3.  How did you get ext4 to work?

What revision of livecd-tools are you on?  Maybe that is the problem.

Comment 13 Frederick Grose 2015-06-11 15:48:52 UTC
(In reply to Todd from comment #12)
> How did you get ext4 to work?

See above. I used the onboard copy of livecd-iso-to-disk in the LiveOS directory of the root .iso filesystem.

> What revision of livecd-tools are you on?  Maybe that is the problem.

[liveuser@localhost ~]$ extlinux -v
extlinux 6.03  Copyright 1994-2014 H. Peter Anvin et al

I repeated the installation with .iso dd'ed to a second USB flash drive and then booted that drive. Here is the installation transcript:

[liveuser@localhost ~]$ sudo /run/initramfs/live/LiveOS/livecd-iso-to-disk --reset-mbr --overlay-size-mb 6624 --home-size-mb 1024 --unencrypted-home /run/initramfs/livedev /dev/sdd1
Verifying image...
/dev/sdc1:   e3ae85c13c4e8c8da9f21744fa416a21
Fragment sums: 183e8f77d5268df13d6a814a3d6d6cbcc62be4455b9f1e84451a297be956
Fragment count: 20
Supported ISO: no
Press [Esc] to abort check.
Checking: 100.0%

The media check is complete, the result is: PASS.

It is OK to use this media.
Copying live image to target device.
squashfs.img
    858,660,864 100%   10.32MB/s    0:01:19 (xfr#1, to-chk=0/1)
osmin.img
          8,192 100%    0.00kB/s    0:00:00 (xfr#1, to-chk=0/1)
Updating boot config file
Initializing persistent overlay file
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.000163429 s, 0.0 kB/s
Initializing persistent /home
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.000197442 s, 0.0 kB/s
Formatting unencrypted /home
mke2fs 1.42.11 (09-Jul-2014)
Discarding device blocks: done                            
Creating filesystem with 262144 4k blocks and 65536 inodes
Filesystem UUID: ddf19da5-d664-4529-bdc4-fa38090a8ace
Superblock backups stored on blocks: 
	32768, 98304, 163840, 229376

Allocating group tables: done                            
Writing inode tables: done                            
Creating journal (8192 blocks): done
Writing superblocks and filesystem accounting information: done

tune2fs 1.42.11 (09-Jul-2014)
Setting maximal mount count to -1
Setting interval between checks to 0 seconds
Installing boot loader
/media/tgttmp.8McipX/syslinux is device /dev/sdd1
Target device is now set up with a Live image!

I booted the freshly installed LiveUSB, copied an application icon
to the Desktop and rebooted.  The file was persistent

[liveuser@localhost ~]$ sudo dmsetup status
live-base: 0 6291456 linear 
live-osimg-min: 0 6291456 snapshot 1960/1960 16
live-rw: 0 6291456 snapshot 38464/13565952 160

[liveuser@localhost ~]$ sudo losetup -l
NAME       SIZELIMIT OFFSET AUTOCLEAR RO BACK-FILE
/dev/loop0         0      0         0  1 /osmin.img (deleted)
/dev/loop1         0      0         0  1 /osmin
/dev/loop2         0      0         0  1 /run/initramfs/live/LiveOS/squashfs.img
/dev/loop3         0      0         0  1 /LiveOS/ext3fs.img
/dev/loop4         0      0         0  0 /LiveOS/overlay-LIVE-e2f445aa-eb8b-4b9a-acdb-8d1bf0449849
/dev/loop5         0      0         0  0 /run/initramfs/live/LiveOS/home.img

[liveuser@localhost ~]$ sudo ls -lA /run/initramfs/live/LiveOS/
total 891400
-rw-r--r--. 1 root root 1073741824 Jun 11 07:33 home.img
-r-xr-xr-x. 1 root root      49592 Jun 11 07:23 livecd-iso-to-disk
-r--r--r--. 1 root root       8192 Jun 11 07:21 osmin.img
-rw-r--r--. 1 root root 6945767424 Jun 11 07:33 overlay-LIVE-e2f445aa-eb8b-4b9a-acdb-8d1bf0449849
-r--r--r--. 1 root root  858660864 Jun 11 07:21 squashfs.img

[liveuser@localhost ~]$ extlinux -v
extlinux 6.03  Copyright 1994-2014 H. Peter Anvin et al

[liveuser@localhost ~]$ ls -lA /usr/share/applications/liveinst.desktop 
-rw-r--r--. 1 root root 10590 Jun 11 07:27 /usr/share/applications/liveinst.desktop
[liveuser@localhost ~]$ 

[liveuser@localhost ~]$ sudo df -T
Filesystem          Type     1K-blocks     Used Available Use% Mounted on
/dev/mapper/live-rw ext4       3030800  2429748    553216  82% /
devtmpfs            devtmpfs   2003672        0   2003672   0% /dev
tmpfs               tmpfs      2023860       72   2023788   1% /dev/shm
tmpfs               tmpfs      2023860     1068   2022792   1% /run
tmpfs               tmpfs      2023860        0   2023860   0% /sys/fs/cgroup
/dev/sdc1           ext4      15299276  1364828  13134232  10% /run/initramfs/live
tmpfs               tmpfs      2023860       12   2023848   1% /tmp
varcacheyum         tmpfs      2023860        0   2023860   0% /var/cache/yum
vartmp              tmpfs      2023860       12   2023848   1% /var/tmp
/dev/loop5          ext4        999320     8544    921964   1% /home
tmpfs               tmpfs       404776       12    404764   1% /run/user/1000

Comment 14 Todd 2015-06-12 21:48:54 UTC
(In reply to Frederick Grose from comment #13)
> (In reply to Todd from comment #12)
> > How did you get ext4 to work?
> 
> See above. I used the onboard copy of livecd-iso-to-disk in the LiveOS
> directory of the root .iso filesystem.
> 
> > What revision of livecd-tools are you on?  Maybe that is the problem.
> 
> [liveuser@localhost ~]$ extlinux -v
> extlinux 6.03  Copyright 1994-2014 H. Peter Anvin et al


What is "extlinux"?

I was looking for 
    rpm -qa livecd-tools


This is mine:

$ rpm -qa livecd-tools
livecd-tools-13.4.5-1.el6.x86_64

Comment 15 Frederick Grose 2015-06-13 01:57:12 UTC
The livecd-tools package is not installed in Fedora-Live-Xfce-x86_64-21-5.iso.  As I mentioned, I used the copy of livecd-iso-to-disk that was copied to the iso filesystem at /LiveOS/ during the building of the iso.  It was most likely from Fedora 21.

livecd-iso-to-disk can run without an installation of livecd-tools.  livecd-iso-to-disk's preparation of the boot loader does require the syslinux-extlinux package (extlinux for ext234 USB device filesystems).

Comment 16 Todd 2015-06-13 02:15:02 UTC
(In reply to Frederick Grose from comment #15)
> The livecd-tools package is not installed in
> Fedora-Live-Xfce-x86_64-21-5.iso.  As I mentioned, I used the copy of
> livecd-iso-to-disk that was copied to the iso filesystem at /LiveOS/ during
> the building of the iso.  It was most likely from Fedora 21.
> 
> livecd-iso-to-disk can run without an installation of livecd-tools. 
> livecd-iso-to-disk's preparation of the boot loader does require the
> syslinux-extlinux package (extlinux for ext234 USB device filesystems).


How about using the program that I complained about? 

If it works differently than the one you used, reopen this bug and fix the program I complained about.

This bug is SPECIFICALLY about the version found in livecd-tools.

Shouldn't be too difficult to fix and the one you are using can be used as a template.

Reopening.  First analysis was on different version of the program.

Comment 17 Brian Lane 2015-06-15 19:23:12 UTC
Please stop reopening this.

See comment 6.

Use a *matching* (or close) version of livecd-iso-to-disk when creating USB media for Fedora releases. If you did this you wouldn't have all these problems.