Bug 1154840

Description of problem:

Since the update to Firefox 31 ESR, users have been complaining that
downloads of PDFs hang indefinitely. Oddly, they hang at the point where
progress shows the total size has been received. The file has been
completely downloaded to /tmp/randomstring.pdf.part but the copy from there
to the final file destination hangs.

If the preferred action is to render in the builtin PDF viewer,
that works - but using the 'save' button in the builtin viewer
still results in a hang.

It is very curious to see a hang for downloads of a specific file type only.

Version-Release number of selected component (if applicable):

firefox-31.1.0-6.el5

How reproducible:

Quite.

Steps to Reproduce:
1. Browse to a site with a PDF file to download, and choose to save it.
2. Choose a location that is NOT the same filesystem as /tmp
3. Observe hang AFTER complete file contents are in /tmp/something.pdf.part

Actual results:

1. The chosen final filename gets created with 0 length in the chosen
   final location.
2. The complete contents of the file are successfully downloaded
   to a /tmp/somerandomstring.pdf.part file.
3. As seen with strace, a rename() is attempted to the final file name,
   which fails because the filesystems are different. Fallback is to
   copy the data.
4. The download progress indicator never progresses further. The X by
   the progress bar can only "pause" the download, nothing else is
   possible. The final destination file remains 0 length.

Telling Firefox to exit results in a dialog saying that 1 download
will be cancelled. If told to exit anyway, Firefox will close its
windows but not immediately exit. After several seconds, these
messages appear:

WARNING: A phase completion condition is taking too long to complete. Condition: OS.File: flush I/O queued before profile-before-change Phase: profile-before-change State: (none)
WARNING: A phase completion condition is taking too long to complete. Condition: CrashMonitor: Writing notifications to file after receiving profile-before-change Phase: profile-before-change State: (none)


After another minute or two, Firefox does in fact exit, with these further messages:

ERROR: At least one completion condition failed to complete within a reasonable amount of time. Causing a crash to ensure that we do not leave the user with an unresponsive process draining resources. Conditions: [{"name":"OS.File: flush I/O queued before profile-before-change","state":{"launched":true,"shutdown":false,"worker":true,"pendingReset":false,"latestSent":["Wed Oct 08 2014 13:12:53 GMT-0400 (EDT)","move",[{"string":"/tmp/tYlACT6A.pdf.part"},{"string":"/homes/foo/Downloads/bar.pdf"},null]],"latestReceived":null,"messagesSent":0,"messagesReceived":34,"messagesQueued":43,"DEBUG":false}},{"name":"CrashMonitor: Writing notifications to file after receiving profile-before-change","state":"(none)"}] Phase: profile-before-change
WARNING: No crash reporter available
[Parent 13094] ###!!! ABORT: file resource://gre/modules/AsyncShutdown.jsm, line 431
[Parent 13094] ###!!! ABORT: file resource://gre/modules/AsyncShutdown.jsm, line 431
Segmentation fault

If Firefox is restarted and the /tmp/...pdf.part file is still there, Firefox
seems to detect it and finish moving it to the desired location - successfully!
... sometimes.  I've also seen it not happen.

The hang does not occur if the chosen destination is the same filesystem
as /tmp - the case where the rename() succeeds. The "move" worker seems to
be hanging if it has to actually copy data.

Expected results:

The download should complete without hanging.

Additional info:

At first I thought nfs might be involved, because it didn't hang downloading
to /tmp and did hang downloading to my nfs-mounted desktop. But then I tried
downloading to another local filesystem different from /tmp and it hangs
there too, so it's not nfs, it's just the cross-filesystem move even when
both filesystems are local.