Bug 603277 - livecd-iso-to-disk mentions it will take ext2/3 partitions but only accepts vfat?
Summary: livecd-iso-to-disk mentions it will take ext2/3 partitions but only accepts v...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: livecd-tools
Version: 13
Hardware: All
OS: Linux
low
medium
Target Milestone: ---
Assignee: David Huff
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On: 609443
Blocks:
TreeView+ depends on / blocked
 
Reported: 2010-06-12 06:50 UTC by Ed
Modified: 2010-09-16 03:48 UTC (History)
10 users (show)

Fixed In Version: livecd-tools-034-7.fc14
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2010-09-16 03:48:45 UTC


Attachments (Terms of Use)
livecd-iso-to-disk patch for Ext2/3/4 (Ext4 requires syslinux 4.00 or greater) (3.65 KB, application/octet-stream)
2010-08-05 04:51 UTC, Jasper O'neal Hartline
no flags Details

Description Ed 2010-06-12 06:50:05 UTC
Description of problem:

livecd-iso-to-disk apparently does not accept an ext3-formatted partition on a USB thumb drive, despite printing message "USB filesystem must be vfat or ext[23]"

Version-Release number of selected component (if applicable):

No version number found in the shell script/--help invocation. As in Fedora 13 Live CD, whose URL is below.

How reproducible:

Visit http://docs.fedoraproject.org/en-US/Fedora/13/html/Installation_Guide/Making_USB_Media-UNIX_Linux-other.html
and try to follow procedure with F13 i686 Live CD ISO (d/l from
http://fedoramirror.hnsdc.com/releases/13/Live/i686/Fedora-13-i686-Live.iso)
with 1 GB USB thumb drive with data on it (bootable Ubuntu 10.04, created using
Ubuntu's "USB Startup Disk" utility).

Steps to Reproduce:
1. Running Ubuntu 10.04, mount F13 ISO as mentioned in documentation linked above.
2. Insert USB thumb drive with bootable Ubuntu 10.04 on it created using
Ubuntu's "USB Startup Disk" utility. 
3. Note partition device node (in my case, /dev/sdb1)
4. Run [mountpoint]/LiveOS/livecd-iso-to-disk /path/to/Fedora-13-i686-Live.iso /dev/sdb1
5. Observe "Unable to fit live image + overlay on available space" error message.
6. (Unthinkingly) run "mkfs.ext3 /dev/sdb1" to quickly reclaim used space.
7. Re-run livecd-iso-to-disk as in (4).
8. Observe that it fails with error message "USB filesystem must be vfat or ext[23]".
9. Format /dev/sdb1 with vfat.
10. Re-run livecd-iso-to-disk as in (4).
11. Observe that it succeeds, as seen in session transcript below.
  
Actual results:

livecd-iso-to-disk does not seem to recognize an ext3-formatted partition despite the message saying it is acceptable.

Expected results:

livecd-iso-to-disk should accept and work with ext3 partition as advertised.
OR
livecd-iso-to-disk message should state that only vfat partitions are acceptable.

Additional info:
Session transcript (with some common mkfs.ext3 output snipped):

root@edgar-laptop:~# /tmp/mnt/LiveOS/livecd-iso-to-disk /new-storage/downloads/Fedora-13-i686-Live.iso /dev/sdb1
Verifying image...
/tmp/mnt/LiveOS/livecd-iso-to-disk: line 399: checkisomd5: command not found
Are you SURE you want to continue?
Press Enter to continue or ctrl-c to abort

Unable to fit live image + overlay on available space on USB stick
+ Size of live image:  660
---------------------------
= Requested:  660
- Available:  0
---------------------------
= To fit, free or decrease requested size total by:  660
Cleaning up to exit...
root@edgar-laptop:~# mkfs.ext3 /dev/sdb1
mke2fs 1.41.11 (14-Mar-2010)
Filesystem label=
[snip]
This filesystem will be automatically checked every 36 mounts or
180 days, whichever comes first.  Use tune2fs -c or -i to override.
root@edgar-laptop:~# sync
root@edgar-laptop:~# /tmp/mnt/LiveOS/livecd-iso-to-disk /new-storage/downloads/Fedora-13-i686-Live.iso /dev/sdb1
Verifying image...
/tmp/mnt/LiveOS/livecd-iso-to-disk: line 399: checkisomd5: command not found
Are you SURE you want to continue?
Press Enter to continue or ctrl-c to abort

USB filesystem must be vfat or ext[23]
Cleaning up to exit...
root@edgar-laptop:~# mkfs.vfat /dev/sdb1 
mkfs.vfat 3.0.7 (24 Dec 2009)
root@edgar-laptop:~# sync
root@edgar-laptop:~# /tmp/mnt/LiveOS/livecd-iso-to-disk /new-storage/downloads/Fedora-13-i686-Live.iso /dev/sdb1
Verifying image...
/tmp/mnt/LiveOS/livecd-iso-to-disk: line 399: checkisomd5: command not found
Are you SURE you want to continue?
Press Enter to continue or ctrl-c to abort

Copying live image to USB stick
Updating boot config file
Installing boot loader
USB stick set up as live image!

Comment 1 Jasper O'neal Hartline 2010-06-12 07:32:34 UTC
Can you put the Ubuntu disk utility software back on that USB drive.. then run: blkid /dev/sdb1

Provide that output. This would be helpful.
Also, after that run: mkfs.ext3 /dev/sdb1
and then again: blkid /dev/sdb1

Provide the two outputs from that.

Comment 2 Ed 2010-06-13 23:47:03 UTC
> Can you put the Ubuntu disk utility software back on that USB drive.. then run:
> blkid /dev/sdb1

Re-created Ubuntu bootable USB drive via "Startup Disk Creator" and ran the command:

root@edgar-laptop:~# blkid /dev/sdb1
/dev/sdb1: UUID="0D6F-40F3" TYPE="vfat" 

> Also, after that run: mkfs.ext3 /dev/sdb1
> and then again: blkid /dev/sdb1

(after unmounting the partition)
root@edgar-laptop:~# mkfs.ext3 /dev/sdb1
mke2fs 1.41.11 (14-Mar-2010)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
62592 inodes, 250224 blocks
12511 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=260046848
8 block groups
32768 blocks per group, 32768 fragments per group
7824 inodes per group
Superblock backups stored on blocks: 
	32768, 98304, 163840, 229376

Writing inode tables: done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 32 mounts or
180 days, whichever comes first.  Use tune2fs -c or -i to override.
root@edgar-laptop:~# blkid /dev/sdb1
root@edgar-laptop:~# blkid /dev/sdb1
root@edgar-laptop:~# 

Oh, oh - I think I see why I faced this problem - error between chair and keyboard? :-P With my knee-jerk formatting with ext3, the partition apparently still has its type set to "c", W95 FAT32 that Ubuntu's Startup Disk Creator set. :-( although it now contains an ext3 filesystem.

root@edgar-laptop:~# fdisk -l /dev/sdb

Disk /dev/sdb: 1024 MB, 1024966656 bytes
32 heads, 62 sectors/track, 1009 cylinders
Units = cylinders of 1984 * 512 = 1015808 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000b1477

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1   *           1        1009     1000897    c  W95 FAT32 (LBA)
root@edgar-laptop:~# 

Hmm... is this a situation that anyone else might encounter? Possible, I'd say... ashamed to label myself as an ID 10 Tango :-(

Comment 3 Jasper O'neal Hartline 2010-06-14 00:02:19 UTC
Well. It is potentially a problem others will face.
I would expect that it should all function correctly when the USB stick is removed, and reinserted, maybe that is a bad assumption.

I tested mkfs with vfat and mkfs with ext3 on several different partition types and back and forth, I couldn't reproduce the problem myself.

I see though that the blkid is not returning any data.

Comment 4 Ed 2010-06-14 01:27:08 UTC
I thought I'd try and correct my apparent mistake with partition type; however, my several attempts to use ext2/3 all failed. Session log below...


I fired up the GUI [Red Hat] Disk Utility 2.30.1 that Ubuntu 10.04 carries. I changed the partition type of /dev/sdb1 to Linux (83). Then I re-ran livecd-iso-to-disk:

root@edgar-laptop:~# /tmp/mnt/LiveOS/livecd-iso-to-disk
/new-storage/downloads/Fedora-13-i686-Live.iso /dev/sdb1
[snip 'missing checkisomd5' blurb]
USB filesystem must be vfat or ext[23]
Cleaning up to exit...
root@edgar-laptop:~# fdisk -l /dev/sdb

Disk /dev/sdb: 1024 MB, 1024966656 bytes
[snip]
   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1   *           1        1009     1000897   83  Linux
root@edgar-laptop:~# /tmp/mnt/LiveOS/livecd-iso-to-disk /new-storage/downloads/Fedora-13-i686-Live.iso /dev/sdb1
[snip]
USB filesystem must be vfat or ext[23]
Cleaning up to exit...


Ejected and re-plugged the USB drive. 'blkid /dev/sdb1' still returns nothing. livecd-iso-to-disk still complains "USB filesystem must be vfat or ext[23]"

I then used Disk Utility to format /dev/sdb1 with ext3 again. Re-tried; "USB filesystem must be vfat or ext[23]." 

Used Disk Utility to *delete the partition* on the USB drive, and create a new ext2 (two, not three) partition. Ejected and re-plugged the USB drive. livecd-iso-to-disk: "USB filesystem must be vfat or ext[23]." 'blkid /dev/sdb1' still returns nothing.

(BTW - even though blkid does not return any information, these are valid filesystems; Ubuntu auto-mounts and pops up a Nautilus window on plugging in the USB drive.)


Stymied, decided to once again try vfat. Used Disk Utility GUI to delete the ext2 partition and create a new FAT32 partition. Ejected and re-plugged the USB drive.

root@edgar-laptop:~# /tmp/mnt/LiveOS/livecd-iso-to-disk
/new-storage/downloads/Fedora-13-i686-Live.iso /dev/sdb1
[snip]
Partition isn't marked bootable!
You can mark the partition as bootable with 
    # /sbin/parted /dev/sdb
    (parted) toggle N boot
    (parted) quit
Cleaning up to exit...
root@edgar-laptop:~# /sbin/parted /dev/sdb
GNU Parted 2.2
Using /dev/sdb
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) toggle 1 boot                                                   
(parted) quit                                                             
Information: You may need to update /etc/fstab.                           

root@edgar-laptop:~# /tmp/mnt/LiveOS/livecd-iso-to-disk
/new-storage/downloads/Fedora-13-i686-Live.iso /dev/sdb1
[snip]
Copying live image to USB stick
Updating boot config file
Installing boot loader
USB stick set up as live image!
root@edgar-laptop:~# blkid /dev/sdb1
/dev/sdb1: UUID="49C6-663E" TYPE="vfat" LABEL="New Volume" 
root@edgar-laptop:~# 

Conclusion: 
blkid seems to only return info for a vfat-formatted volume (on Ubuntu 10.04, at least) - checkFilesystem() obviously expects info for ext2/3 as well. If there haven't been problems reported by people running livecd-iso-to-disk under Fedora itself, this may be an Ubuntu quirk? Please let me know if you have any more tests I should try on Ubuntu blkid.

Ideas:

- For a drive that has a FAT32 partition, but free space is not enough for the Fedora image: offer to format it (createMSDOSLayout()) to reclaim space. OR, before exiting with "not enough space", inform user that he can run livecd-iso-to-disk with --format to automatically format the drive.

- Could livecd-iso-to-disk have some fallback checks besides blkid to detect valid ext2/3 filesystems on non-RH/Fedora distros? (assuming that this is different behavior of blkid on Ubuntu, which may not be the case).

Thanks!

Comment 5 Ed 2010-06-14 01:41:52 UTC
(In reply to comment #3)
> Well. It is potentially a problem others will face.
> I would expect that it should all function correctly when the USB stick is
> removed, and reinserted, maybe that is a bad assumption.

Had a "Mid-air collision" when I tried to submit previous comment, because you had posted comment #3 :-) Anyway, as I've noted in #4, unplugging and re-plugging the USB drive does not make any difference.

> I tested mkfs with vfat and mkfs with ext3 on several different partition types
> and back and forth, I couldn't reproduce the problem myself.

Hmmm?? That's strange. I assume you're running Fedora...?
 
> I see though that the blkid is not returning any data.    

Wait - that's what I thought was the root cause of livecd-iso-to-disk not working for me. If blkid doesn't return any data for you either, why does livecd-iso-to-disk work for you?

Comment 6 Jasper O'neal Hartline 2010-06-14 02:03:31 UTC
All my tests were done on Fedora with Fedora's blkid from util-linux-ng and mkfs from Fedora. I think this might be an Ubuntu blkid problem.
Can you check which version of util-linux-ng Ubuntu has just for a comparison to the one I'm using. It isn't really important, just trying to know why. 

Originally I thought you were using Fedora tools, and the USB stick had Ubuntu, unfortunately we cannot do anything about Ubuntu to help relieve problems with livecd-iso-to-disk in Fedora as the tools seem to work fine, in my testing anyways.

Yeah so what is the version of util-linux-ng?

Comment 7 Ed 2010-06-14 05:00:37 UTC
dpkg -l util-linux shows:
util-linux 2.17.2-0ubuntu1

$ blkid -v
blkid from util-linux-ng 2.17.2 (libblkid 2.17.0, 22-Mar-2010)

How does that compare with your version?

==============
Well, no, I wasn't asking for you to fix Ubuntu (though I really wish someone would! :) )

However, I don't think that Ubuntu blkid is necessarily broken, because you already noted that blkid on your Fedora also returns nil info on ext3 partitions, which is the same behavior I see in Ubuntu.

From what I can see in the source of livecd-iso-to-disk, that means it should fail in your case too, and not create a bootable Fedora USB. Have you tried  to create a live USB on an ext3/ext2 partition on a USB drive, from Fedora?

Just to make sure we are looking at the same file, I uploaded the one from my LiveCD ISO to http://filebin.ca/kqcfu - could you download it and do a diff against the one you're using, if yours works and this one doesn't?

Thanks,
Ed.

Comment 8 Jasper O'neal Hartline 2010-06-14 05:32:31 UTC
That is the same blkid version I am using.. and no when I said I see blkid is coming up empty, I was noticing it in your output which was requested.

In my testing I cannot get blkid to come up with no print out of information.
I tried, type 82 partition with ext3 fs, I tried type b partition with ext2 and ext3. All times I could not get the system to show no data like yours.

Comment 9 Ed 2010-06-14 05:47:36 UTC
(In reply to comment #8)
> In my testing I cannot get blkid to come up with no print out of information.

Oh. 

Ouch.

Sighh... well, I guess this must be another of the huge pile of bugs in the so-called "Long-Term Support" release :-/ Every day brings a new reason to find a replacement for Ubuntu in my personal usage :-(

Well, thanks for your patience and help, Jasper, I guess this is a dead end but at least it teaches me a lesson - Ubuntu 10.04 is way more broken than I thought!

Thanks,
Ed.

Comment 10 Jasper O'neal Hartline 2010-08-05 04:50:49 UTC
Try this patch, it should work on ext3 and ext2 preformatted filesystems.
If you decide to use --format flag, please have syslinux 4.00 or greater installed as it is required. Otherwise this patch should work for pre-existing ext2 and ext3 filesystems:

Comment 11 Jasper O'neal Hartline 2010-08-05 04:51:51 UTC
Created attachment 436739 [details]
livecd-iso-to-disk patch for Ext2/3/4 (Ext4 requires syslinux 4.00 or greater)

Patches clean against livecd-tools git

Comment 12 Frederick Grose 2010-08-29 16:04:27 UTC
See https://bugzilla.redhat.com/show_bug.cgi?id=587411 livecd-iso-to-disk --format fails with SD/MMC partitions.

The umount ${device}? will fail with an SD/MMC partition that uses a device-partition# separator, such as the 'p' in /dev/mmcblk0p1

umount ${device}* would work, but would also unmount all partitions.

Having the partition number 1 hardcoded as USBDEV=${device}1 could also fail with some SD/MMC drivers.
 
Attachment, https://bugzilla.redhat.com/attachment.cgi?id=413572&action=edit , has some replacement code:

-    USBDEV=${device}1

+    partition=$( awk '$4 ~ /'${device#/dev/}'.*[0-9]$/ {print $4}' /proc/partitions )
+    partnum=${partition##${device#/dev/}}
+    USBDEV=${device}${partnum}

(See mailing list thread, http://lists.fedoraproject.org/pipermail/livecd/2010-August/006186.html )

Comment 13 Bruno Wolff III 2010-09-12 02:29:38 UTC
I have a question about this one.
If it is the case that using umount ${device}* will unmount more than one partition, how are you supposed to know which is the right one to unmount?

For the initial go around I'm going to use umount ${device}*, but we will likely need to do another update soon anyway, since with the number of bug fix patches I'm getting in, there is likely to be at least one regression that will require an update.

Comment 14 Fedora Update System 2010-09-12 04:20:25 UTC
livecd-tools-034-1.fc14 has been submitted as an update for Fedora 14.
https://admin.fedoraproject.org/updates/livecd-tools-034-1.fc14

Comment 15 Fedora Update System 2010-09-13 18:10:15 UTC
livecd-tools-034-2.fc14 has been pushed to the Fedora 14 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update livecd-tools'.  You can provide feedback for this update here: https://admin.fedoraproject.org/updates/livecd-tools-034-2.fc14

Comment 16 Fedora Update System 2010-09-15 03:23:04 UTC
livecd-tools-034-7.fc14 has been submitted as an update for Fedora 14.
https://admin.fedoraproject.org/updates/livecd-tools-034-7.fc14

Comment 17 Fedora Update System 2010-09-15 06:52:23 UTC
livecd-tools-034-7.fc14 has been pushed to the Fedora 14 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update livecd-tools'.  You can provide feedback for this update here: https://admin.fedoraproject.org/updates/livecd-tools-034-7.fc14

Comment 18 Fedora Update System 2010-09-16 03:47:08 UTC
livecd-tools-034-7.fc14 has been pushed to the Fedora 14 stable repository.  If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.