Bug 885133

Summary: no notification when unmounting external HDDs and data are still not synced
Product: [Fedora] Fedora Reporter: Kamil Páral <kparal>
Component: nautilusAssignee: Tomáš Bžatek <tbzatek>
Status: CLOSED CURRENTRELEASE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 18CC: alexl, artkun, awilliam, bnocera, ccecchi, davidz, mclasen, mishu, robatino, samuel-rhbugs, tbzatek, tsmetana
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: AcceptedBlocker
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-12-15 03:14:34 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 752661    
Attachments:
Description Flags
gvfs-mount -li
none
udisks --dump
none
udisksctl dump none

Description Kamil Páral 2012-12-07 14:59:11 UTC
Description of problem:
This is a follow-up of bug 819492. It was a Fedora 17 blocker and unfortunately, even though we thought we did, we haven't fixed it properly.

== Brief history ==

In GNOME 3.2 if you tried to eject a flash drive and all data were still not synced to it, there was a pop-up dialog saying "don't eject the drive yet" and then "now you can eject".

Bug 819492 described a regression in GNOME 3.4, the dialog was gone. The code change was intentional, however no code replacement was provided. A proper fix was to be implemented in GNOME 3.6.

The bug was deemed to be critical, because there was a high probability of users losing their data. Also because LiveCDs can't be fixed post-release and are often used for system recovery purposes (thus often interacting with external drives), it was accepted as a blocker for Fedora 17. A simple fix was implemented that makes a gnome-shell-style dialog pop up every time you try to disconnect a drive that is not synced yet.

== Present time ==

In GNOME 3.6 the behavior seems to be completely same as in GNOME 3.4. I don't know if any of the code changed, but visually it seems like the same "quick fix" from GNOME 3.4 remained also in GNOME 3.6.

In bug 819492 comment 72 to 74 we have received some information that this still might be broken. Also I have heard people saying GNOME acts weird sometimes. So I started to investigate and found the issue.

== The problem ==

In bug 819492 the developers have fixed the bug, but just for flash drives. It is not fixed for external HDDs.

If you use a flash drive, it works like this:
* If you click on the unmount icon in Nautilus and all data are synced, the device disappears from the list. It is safe to be ejected.
* If some data are not synced, a gnome-shell popup appears and informs you to wait. After the sync is completely, the device disappears from the list and it is safe to be ejected.

If you use an external HDD drive, it works like this:
* If you click on the unmount icon in Nautilus and all data are synced, the device _doesn't_ disappear from the list, just the unmount icon disappears. It is safe to be ejected.
* If some data are not synced, the unmount icon still disappears and no dialog pops up. There is _no visual difference_ between a safe-to-eject disk and a still-syncing disk (!). And that is what this bug is about.

The question remains why a flash drive behaves differently from an external HDD.

== The difference between a flash drive and an external HDD ==

I think I have managed to find the reason why GNOME acts differently for flash drives and for external HDDs. If you insert a flash drive, a text similar to the following appears in dmesg:

> [33436.521311] sd 6:0:0:0: [sdb] Attached SCSI removable disk

But if you insert an external HDD, the following appears:

> [33436.521311] sd 6:0:0:0: [sdb] Attached SCSI disk

Please notice the missing "removable" word. The difference is the "Removable Media Bit" (google for it). For some reason the flash drives have it and the external HDDs don't.

== Affected systems ==

From my experience anything with GNOME 3.2 or newer is affected. That means Fedora 17 and Fedora 18, I have tested them both.

The most recent builds I have tested with are:
gvfs-1.14.2-1.fc18.x86_64
gnome-shell-3.6.2-2.fc18.x86_64
nautilus-3.6.3-1.fc18.x86_64

With regards to the removable media bit (RMB):
* I have tested 5 different brands of flash drives, all have RMB set (and therefore are safe to use).
* I have tested 3 different brands of external HDDs (Western Digital, Seagate and Transcend), none of them has RMB set (and therefore are unsafe to use). In bug 819492 comment 72 - 74 we also have some further mediated confirmation from other people.
* I have tested an internal SATA drive connected to a PC using a SATA<->USB converter. It doesn't have RMB set (and therefore is unsafe to use).

That makes me think that the numbers are probably very simple - we can say that majority (if not all) of flash drives are safe to use and majority (if not all) of external HDDs are unsafe to use.

== Reproducer ==

1. Open a terminal and a Nautilus
2. Create a 1 GB file
   $ dd if=/dev/zero of=file bs=10M count=100
3. Insert a device you want to test (a flash drive or an external HDD)
4. Look into dmesg and confirm whether it is regarded as removable or not
   $ dmesg | grep "Attached SCSI"  # search for "removable" word
5. In the nautilus copy the file to the device
6. While it is copying, run "sync" in the terminal. You will have it as an indicator that all data are not synced yet (once the command finishes, all data are synced).
   $ sync
7. Immediately once the copy dialog in Nautilus disappears (i.e. the user assumes the copy process is over), but before the "sync" command finishes, click the unmount icon on the device in Nautilus.
8. If a gnome-shell dialog pops up and informs you that you have to wait, congratulations, you're safe. If no dialog appears, congratulation, you have reproduced the bug. Plug the device from the computer. In this case you should see huge amounts of write errors in dmesg afterwards and your file on the device should differ from the file on your computer.

You can also watch my older (very long) video in attachment 582641 [details] where I showcase this with different approaches and filesystems. That was shot in the time where this functionality was broken even for flash drives.

== Proposed solution ==

I have no knowledge of the code in gvfs, so my views can be completely off. But I assume the system should behave the same for both types of media.
* It doesn't matter whether it is a flash drive, external HDD or internal HDD, if I try to unmount/eject it, I should be informed when the data are still being written.
* The unmount icon in Nautilus shouldn't disappear until the sync is done, or it should be replaced with a spinner in the mean time.

== Blocker status ==

Consistently with previous decision in bug 819492, I'm proposing this as a Final blocker for Fedora 18. While this is fixed for flash drives, external HDDs are also used heavily with LiveCDs. Because they are faster, the risk of losing data is a bit lower, but it still seems to be high enough. A loss of user data is always considered as the most critical type of bugs.

I'm very sorry I discovered this so late in the cycle. But I believe we would strive for a very fast fix in all cases, because it also affects users of our stable release (Fedora 17).

Comment 1 Adam Williamson 2012-12-10 19:22:50 UTC
Discussed at 2012-12-10 blocker review meeting: http://meetbot.fedoraproject.org/fedora-bugzappers/2012-12-10/f18final-blocker-review-3.2012-12-10-17.13.log.txt . Accepted as a blocker per criterion "All known bugs that can cause corruption of user data must be fixed or documented at Common F18 bugs". Desktop team, please advise us ASAP of your position on this - is it something you're happy to fix along the lines used for flash drives, or do we need to have a big debate about this again? :) Thanks.

Comment 2 Tomáš Bžatek 2012-12-11 11:15:20 UTC
(In reply to comment #0)
> I'm very sorry I discovered this so late in the cycle.
Nice Christmas present!

> With regards to the removable media bit (RMB):
> * I have tested 5 different brands of flash drives, all have RMB set (and
> therefore are safe to use).
> * I have tested 3 different brands of external HDDs (Western Digital,
> Seagate and Transcend), none of them has RMB set (and therefore are unsafe
> to use). In bug 819492 comment 72 - 74 we also have some further mediated
> confirmation from other people.
> * I have tested an internal SATA drive connected to a PC using a SATA<->USB
> converter. It doesn't have RMB set (and therefore is unsafe to use).

I believe it's all about removable media detection and classification. Could you please attach outputs of `gvfs-mount -li` and `udisks --dump` for both types of devices? (can be connected simultaneously) That would show us more.

Comment 3 Kamil Páral 2012-12-11 13:46:29 UTC
Created attachment 661459 [details]
gvfs-mount -li

Here we go. /dev/sdb is the flash drive, /dev/sdc is the external HDD

Comment 4 Kamil Páral 2012-12-11 13:46:49 UTC
Created attachment 661460 [details]
udisks --dump

Comment 5 David Zeuthen 2012-12-11 14:17:55 UTC
(In reply to comment #4)
> Created attachment 661460 [details]
> udisks --dump

No, that's from udisks 1.x (which ideally wouldn't even be installed but some dependent projects are too slow to port to 2.x unfortunately). These days GNOME uses udisks 2.x (RPM: udisks2) so that's what you should be trying to debug. This page

 http://www.freedesktop.org/wiki/Software/udisks

shows you what stuff information to gather (output of 'udisksctl dump', basically).

Comment 6 David Zeuthen 2012-12-11 14:19:44 UTC
As for the problem at hand, please check if "umount /dev/sdc" does what you expect, e.g. block until all data is written to disk (if it doesn't the problem isn't a gvfs or udisks problem.)

Comment 7 David Zeuthen 2012-12-11 14:34:41 UTC
I just tested this. I get the notification if I unmount from gnome-shell (from the message tray), but I don't get it with Nautilus. Reporter: please see if you get the expected behavior when unmounting from gnome-shell.

So I think GVfs is fine and this is a problem with Nautilus (or maybe the GtkMountOperation proxying to the shell). Either way, you want Cosimo involved. Adding him as Cc.

Comment 8 Kamil Páral 2012-12-11 15:11:38 UTC
Created attachment 661504 [details]
udisksctl dump

Comment 9 Kamil Páral 2012-12-11 15:21:11 UTC
(In reply to comment #6)
> As for the problem at hand, please check if "umount /dev/sdc" does what you
> expect, e.g. block until all data is written to disk (if it doesn't the
> problem isn't a gvfs or udisks problem.)

umount waits until the data is synced. But the unmount icon disappears from Nautilus immediately after running the command.

(In reply to comment #7)
> I just tested this. I get the notification if I unmount from gnome-shell
> (from the message tray), but I don't get it with Nautilus. Reporter: please
> see if you get the expected behavior when unmounting from gnome-shell.
> 
> So I think GVfs is fine and this is a problem with Nautilus (or maybe the
> GtkMountOperation proxying to the shell). Either way, you want Cosimo
> involved. Adding him as Cc.

Yes you are right. Unmounting from gnome-shell message tray triggers the warning "still writing data". Reassigning to Nautilus.

Comment 10 David Zeuthen 2012-12-11 15:23:29 UTC
(In reply to comment #9)
> Unmounting from gnome-shell message tray triggers the
> warning "still writing data". Reassigning to Nautilus.

OK, so, basically, from the gnome-shell you get the same behavior no matter if it's a HDD or a USB stick, right? (e.g. the value of 'removable' in sysfs doesn't matter)

Comment 11 Kamil Páral 2012-12-11 15:31:17 UTC
(In reply to comment #10)
Yes, gnome-shell systray warns me always regardless of media type (flash drive or external HDD) or RMB. Nautilus warns me just for flash drives (RMB=1).

Comment 12 David Zeuthen 2012-12-11 15:37:07 UTC
(In reply to comment #11)
> (In reply to comment #10)
> Yes, gnome-shell systray warns me always regardless of media type (flash
> drive or external HDD) or RMB. Nautilus warns me just for flash drives
> (RMB=1).

OK. The difference in behavior probably has to do with timing of the uevents, e.g. the underlying GMount is removed while the operation is pending if RMB=1, and after otherwise.

Either way, looks like it's possible to fix Nautilus so it works the same way as the Shell.

Comment 13 Cosimo Cecchi 2012-12-11 16:39:23 UTC
This is indeed a Nautilus bug; I fixed it upstream now [1]. A Fedora package update including the patch is now building.

[1] http://git.gnome.org/browse/nautilus/commit/?id=463e8d1b1922474e4cf591a3029ea813804bd3e7

Comment 14 Fedora Update System 2012-12-11 17:09:10 UTC
nautilus-3.6.3-3.fc18 has been submitted as an update for Fedora 18.
https://admin.fedoraproject.org/updates/nautilus-3.6.3-3.fc18

Comment 15 Fedora Update System 2012-12-11 20:06:15 UTC
Package nautilus-3.6.3-3.fc18:
* should fix your issue,
* was pushed to the Fedora 18 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing nautilus-3.6.3-3.fc18'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2012-20187/nautilus-3.6.3-3.fc18
then log in and leave karma (feedback).

Comment 16 Kamil Páral 2012-12-12 09:49:08 UTC
Great job, Cosimo, it is fixed.

Comment 17 Kamil Páral 2012-12-12 09:57:13 UTC
Cosimo, I have created bug 886435 to cover this issue also in Fedora 17. Could you please push the patch also the our current stable release? It is affected as well and people will continue to use it for a long time. Thanks a lot.

Comment 18 Artyom Kunyov 2012-12-12 14:59:54 UTC
Is anything can be done to have the same notification in Cinnamon? In F17 we don't have any kind of notification at all, no matter if we are using Nautilus or Cinnamon applet to eject the drive. Same thing in F18, which is very confusing especially for new users.
Sorry for my English.

Comment 19 Fedora Update System 2012-12-15 03:14:37 UTC
nautilus-3.6.3-3.fc18 has been pushed to the Fedora 18 stable repository.  If problems still persist, please make note of it in this bug report.