Bug 1328498

Summary: partitions are not unmounted before writing image to a device, can lead to corrupted data written
Product: [Fedora] Fedora Reporter: Kamil Páral <kparal>
Component: liveusb-creatorAssignee: Luke Macken <lmacken>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 23CC: awilliam, bcl, lmacken, lsatenstein, mbriza, pfrields, pschindl, robatino
Target Milestone: ---Keywords: Reopened
Target Release: ---Flags: lsatenstein: needinfo?
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: AcceptedPreviousRelease
Fixed In Version: liveusb-creator-3.93.3-1.fc24 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-04-29 11:40:06 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1230433, 1310542    
Attachments:
Description Flags
My description of problems encountered while testing the Fedora Writer.
none
A better description of the attachement 1. Please set attachment 1 aside. none

Description Kamil Páral 2016-04-19 14:12:52 UTC
Description of problem:
When I insert a flash drive with previously written Fedora Workstation Live image, one of the partitions is automatically mounted (the first partition, with "Fedora-WS-Live-24-*" label).

When liveusb-creator starts writing to the device using dd, the partition is still mounted. That is wrong, because after the new data were written, the mounted filesystem can cause some changes to be written over the new data (either when unmounting, or when performing some changes with the filesystem). Alternatively, it might cause a number of errors with the programs accessing that open but no longer physically existing filesystem at that moment. This is easily reproducible, see steps to reproduce.

Liveusb-creator has to make sure that all partitions on that device are unmounted before writing a raw image to it. When some partitions are mounted, it should try to unmount them, and if that fails, it should not allow the user to continue. This is essential to keep the image correctly written.

Version-Release number of selected component (if applicable):
liveusb-creator-3.92.1-1.fc23.noarch

How reproducible:
always

Steps to Reproduce:
1. have a device with an existing mounted partition, make sure the partition size is smaller than the image you're going to write (to make sure we can easily corrupt new data), and that it is at the start of the device
2. open that partition in nautilus
3. start writing a new image to the device in liveusb-creator
4. see that the partition is still mounted in nautilus
5. after the new image was written, just for fun, pick a file and copy it in nautilus into that mounted filesystem (which no longer exists physically, but the system still sees it)
6. unmount the filesystem
7. try to boot the image and check its consistency
8. in my case, I received a kernel panic, because initramfs unpacking failed (because I partly overwrote it, it seems)

Please note that we triggered this issue willingly and manually and it can seem a bit far-fetched (who would do this, right?). But the point was to demonstrate how easily this can happen. In reality, this will sometimes happen and sometimes not, even without intentional user interaction (copying new files), just based on different processes that run on the background, and some luck factor. 

Actual results:
written image can get corrupted because old filesystems are mounted during writing

Expected results:
written image should not get corrupted, because filesystems get unmounted before writing

Comment 1 Kamil Páral 2016-04-20 10:18:24 UTC
Proposing as a Beta blocker, based on this criterion:
"Release-blocking live and dedicated installer images must boot when written to optical media of an appropriate size (if applicable) and when written to a USB stick with any of the officially supported methods."
https://fedoraproject.org/wiki/Fedora_24_Beta_Release_Criteria#Release-blocking_images_must_boot

When there is a high chance that the image written will be corrupted, it can't be considered as a functional writing tool. I have demonstrated that it is quite easy to end up with an unbootable usb stick. Also, when you look at gnome-disks for comparison (which also does a dd-based copy), gnome-disks unmounts all the partitions before writing the image. If it fails to unmount some of the partitions, it announces the problem and does not allow the user to continue.

Comment 2 Fedora Update System 2016-04-21 16:13:34 UTC
liveusb-creator-3.93.1-1.fc24 has been submitted as an update to Fedora 24. https://bodhi.fedoraproject.org/updates/FEDORA-2016-ff4136b90c

Comment 3 Fedora Update System 2016-04-21 16:14:17 UTC
liveusb-creator-3.93.1-1.fc24 has been submitted as an update to Fedora 24. https://bodhi.fedoraproject.org/updates/FEDORA-2016-ff4136b90c

Comment 4 Fedora Update System 2016-04-21 22:00:57 UTC
liveusb-creator-3.93.1-1.fc24 has been pushed to the Fedora 24 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-ff4136b90c

Comment 5 Kamil Páral 2016-04-22 07:58:27 UTC
This is partially fixed with liveusb-creator-3.93.1-1.fc24. The partitions are now unmounted before writing, which is great. But if the unmounting process fails for some reason (e.g. you have a terminal opened with cwd inside that mounted directory, iow the partition is being used), FMR doesn't mind and starts overwriting it. Instead, it should complain that the device/partitions is being used and can't be overwritten at the moment.

Comment 6 Kamil Páral 2016-04-22 07:59:17 UTC
I see this in terminal, but it might not be related to this issue:

[creator:355] Overwriting device with live image
Traceback (most recent call last):
  File "/usr/lib64/python2.7/logging/__init__.py", line 853, in emit
    msg = self.format(record)
  File "/usr/lib64/python2.7/logging/__init__.py", line 726, in format
    return fmt.format(record)
  File "/usr/lib64/python2.7/logging/__init__.py", line 465, in format
    record.message = record.getMessage()
  File "/usr/lib64/python2.7/logging/__init__.py", line 329, in getMessage
    msg = msg % self.args
TypeError: not all arguments converted during string formatting
Logged from file creator.py, line 368

Comment 7 Fedora Update System 2016-04-22 17:07:31 UTC
liveusb-creator-3.93.2-1.fc24 has been submitted as an update to Fedora 24. https://bodhi.fedoraproject.org/updates/FEDORA-2016-ff4136b90c

Comment 8 Fedora Update System 2016-04-24 01:25:48 UTC
liveusb-creator-3.93.2-1.fc24 has been pushed to the Fedora 24 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-ff4136b90c

Comment 9 Kamil Páral 2016-04-25 07:55:26 UTC
Awesome, fixed in liveusb-creator-3.93.2-1.fc24 :-) LUC now presents an error in gui if the partitions can't be unmounted.

Comment 10 Kamil Páral 2016-04-25 08:46:43 UTC
Well, there is still a minor problem. LUC reports that the "drive is in use" every time some partition is mounted, even if the partition gets unmounted fine. So right now, if you insert a flash drive and is automounted, the first time you click "Write to disk" you'll receive "drive is in use", and the second time you click on it, it starts writing. 100% reproducible.

So either there is a code sequence error:

  if mounted:
    print error
  unmount

instead of

  unmount
  if mounted:
    print error

or there is some race condition between unmounting finishing and LUC refreshing the mounted status.

Comment 11 Kamil Páral 2016-04-25 09:04:55 UTC
(In reply to Kamil Páral from comment #10)
> Well, there is still a minor problem. LUC reports that the "drive is in use"
> every time some partition is mounted, even if the partition gets unmounted
> fine. 

The original problem has been fixed, so I decided to switch this bug report back to VERIFIED and report the new problem separately as bug 1329995.

Comment 12 Petr Schindler 2016-04-25 18:00:14 UTC
Discussed at 2016-04-25 blocker review meeting: [1]. 

This bug was accepted as Beta blocker (solved in previous releases) and Freeze Exception: this violates "All release-blocking images must boot in their supported configurations." (with the footnote about USB media) for F22 and F23. we may also consider it serious enough to be an F24 Beta blocker if not pushed stable by go/no-go, but it is at least freeze exception-worthy for now

[1] http://meetbot.fedoraproject.org/fedora-blocker-review/2016-02-29/f24-blocker-review.2016-04-25-17.02.html

Comment 13 Fedora Update System 2016-04-26 08:14:16 UTC
liveusb-creator-3.93.3-1.fc24 has been submitted as an update to Fedora 24. https://bodhi.fedoraproject.org/updates/FEDORA-2016-ff4136b90c

Comment 14 Adam Williamson 2016-04-26 16:18:27 UTC
this was agreed as AcceptedPreviousRelease, fixing.

Comment 15 Fedora Update System 2016-04-26 16:24:12 UTC
liveusb-creator-3.93.3-1.fc24 has been pushed to the Fedora 24 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-ff4136b90c

Comment 16 Fedora Update System 2016-04-26 16:33:48 UTC
liveusb-creator-3.93.3-1.fc24 has been pushed to the Fedora 24 stable repository. If problems still persist, please make note of it in this bug report.

Comment 17 Adam Williamson 2016-04-26 17:17:26 UTC
re-opening, still needs pushing for f22 and f23.

Comment 18 Leslie Satenstein 2016-04-27 00:12:36 UTC
Created attachment 1151159 [details]
My description of problems encountered while testing the Fedora Writer.

I took snapshots about problems encountered as well as bugs encountered.

In addition areas for improvements were noted.

Cant have a subdirectory within the same directory as the iso file
The wrong home directory is not shown
Text is too small (6 or 7 pts).
Expanding the window does not expand the text.
dd command needs a sync call before "finished" is flashed onto the screen¯

Suitable dd command

dd if=path/to/iso/the.iso of=path/to/flashdrive bs=1M && sync

You may remove the bs=1M if you want to live with 512 byte buffers.

Comment 19 Leslie Satenstein 2016-04-27 00:15:53 UTC
Created attachment 1151162 [details]
A better description of the attachement 1. Please set attachment 1 [details] aside.

The pdf file has improved comments.

Comment 20 Kamil Páral 2016-04-27 08:01:20 UTC
The f22 and f23 builds were pushed, but this needs to stay open until the mirror manager stops offering older metadata, according to:
https://fedoraproject.org/wiki/QA:SOP_blocker_bug_process#Tracking_AcceptedPreviousRelease_blocker_bugs

Currently that is not the case:
$ ./track-previous-release-blocker.py liveusb-creator-3.93.3-1.fc23
INFO    Querying Koji for liveusb-creator-3.93.3-1.fc23 in f23-updates ...
INFO    Build liveusb-creator-3.93.3-1.fc23 was tagged into f23-updates at: 2016-04-26 16:44:31 UTC (1461689071.38)
INFO    Downloading metalink for updates-released-f23 ...
INFO    Metalink contains metadata with these timestamps:
        2016-04-24 20:30:33 UTC (1461529833.0)   ✘ older than pushed package
        2016-04-25 23:32:36 UTC (1461627156.0)   ✘ older than pushed package
        2016-04-26 20:26:11 UTC (1461702371.0)   ✔ sufficiently recent
WARNING ✘ FAILED Some metadata referenced in metalink is still older than the time when liveusb-creator-3.93.3-1.fc23 was tagged into f23-updates. Some users would not receive this update if they chose to update now.

And similar for fc22.

Comment 21 Kamil Páral 2016-04-29 11:40:06 UTC
INFO    ✔ PASSED All metadata referenced in metalink is sufficiently newer than the time when liveusb-creator-3.93.3-1.fc23 was tagged into f23-updates. All users should be able to receive the update now.
INFO    ✔ PASSED All metadata referenced in metalink is sufficiently newer than the time when liveusb-creator-3.93.3-1.fc22 was tagged into f22-updates. All users should be able to receive the update now.

Comment 22 Leslie Satenstein 2016-04-29 13:35:11 UTC
https://bugzilla.redhat.com/attachment.cgi?id=1151162

This document is a print out of the multiple problems encountered with the actual item or it's successor.