Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1365237

Summary: Upload image doesn't notice that the disk image was removed, it finalizes the upload and marks it as OK
Product: [oVirt] ovirt-engine Reporter: Natalie Gavrielov <ngavrilo>
Component: BLL.StorageAssignee: Daniel Erez <derez>
Status: CLOSED CURRENTRELEASE QA Contact: Natalie Gavrielov <ngavrilo>
Severity: medium Docs Contact:
Priority: medium    
Version: 4.0.2.4CC: amureini, bugs, lveyde, ngavrilo, ratamir, stirabos, tnisan, ylavi
Target Milestone: ovirt-4.1.3Flags: rule-engine: ovirt-4.1+
rule-engine: planning_ack+
rule-engine: devel_ack+
ratamir: testing_ack+
Target Release: 4.1.3   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-07-06 13:42:39 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Storage RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
logs
none
logs: engine, imageio-proxy, imageio-daemon
none
snapshot none

Description Natalie Gavrielov 2016-08-08 17:00:56 UTC
Created attachment 1188862 [details]
logs

Description of problem:
Upload image should have a way of telling if it uploaded the whole disk image or just a part of it (in case something went wrong, and somehow the disk image disappeared from it's location.

Version-Release number of selected component:
rhevm-4.0.2.4-0.1.el7ev.noarch
ovirt-imageio-common-0.3.0-0.el7ev.noarch
ovirt-imageio-proxy-0.3.0-0.el7ev.noarch
vdsm-4.18.10-1.el7ev.x86_64
ovirt-imageio-daemon-0.3.0-0.el7ev.noarch

How reproducible:
100%

Steps to Reproduce:
1. Upload a disk image.
2. While upload image takes place (is active) delete the disk image from its location.

Actual results:
The moment file was deleted, status bar changes to Finalizing then Complete and eventually OK.

Expected results:

A way for the upload to notice that something went wrong - even pause the upload if it can't alert the user of the strange situation.

Additional info:
Happens both for qcow2 disk images and raw.

Comment 1 Allon Mureinik 2016-09-12 07:58:06 UTC
Does this happen on both file and block based domains?

Comment 2 Natalie Gavrielov 2016-09-15 12:03:29 UTC
(In reply to Allon Mureinik from comment #1)
> Does this happen on both file and block based domains?

I couldn't remember whether I tested this on both block and file or just one of them, so I retested it now:
I used the same scenario described in comment #0, but with 4.0.4 builds.
Got completely different results this time:
Seems it doesn't have any timeout now.
for example, I deleted the file being uploaded when the UI showed:
"Sent 280 of 3072 MB"
And it stays that way for more than half an hour now (I have a feeling it will stay that way..)

Comment 3 Natalie Gavrielov 2017-04-30 15:09:12 UTC
Created attachment 1275290 [details]
logs: engine, imageio-proxy, imageio-daemon

Performed scenario described in comment 0, for both block and file storage types, in both cases the upload remains as if it is still active (does not change state in any point, shows: Sent x of y MB in the progress bar).

Now, the first time I tested this, apparently the engine's date and time wasn't in sync, so I fixed that and once the time changed the two uploads switched state to failed.
*But* when I performed the test again with engine showing the correct time, the uploads just won't switch to failed - and are shown as active.

Comment 4 Natalie Gavrielov 2017-04-30 15:10:41 UTC
Version tested:
rhevm-4.1.2-0.1.el7.noarch
ovirt-imageio-proxy-1.0.0-0.el7ev.noarch

Comment 5 Red Hat Bugzilla Rules Engine 2017-04-30 15:10:46 UTC
Target release should be placed once a package build is known to fix a issue. Since this bug is not modified, the target version has been reset. Please use target milestone to plan a fix for a oVirt release.

Comment 6 Natalie Gavrielov 2017-04-30 15:13:22 UTC
Created attachment 1275292 [details]
snapshot

Comment 7 Daniel Erez 2017-05-07 09:17:03 UTC
(In reply to Natalie Gavrielov from comment #3)
> Created attachment 1275290 [details]
> logs: engine, imageio-proxy, imageio-daemon
> 
> Performed scenario described in comment 0, for both block and file storage
> types, in both cases the upload remains as if it is still active (does not
> change state in any point, shows: Sent x of y MB in the progress bar).
> 
> Now, the first time I tested this, apparently the engine's date and time
> wasn't in sync, so I fixed that and once the time changed the two uploads
> switched state to failed.
> *But* when I performed the test again with engine showing the correct time,
> the uploads just won't switch to failed - and are shown as active.

@Natalie:
* Can you please attach the ui.log as well? 
* In which browsers have you tried it (and which versions)?

Comment 8 Daniel Erez 2017-05-14 09:16:12 UTC
Seems that the current fix is functioning in Chrome but not FireFox.
Thus, added another solution for FireFox: handling the timeout error raised by file removal. I.e. removing the file during upload will raise an XHR timeout error configurable by UploadImageXhrTimeoutInSeconds value - 2 minutes by default.

Comment 9 Natalie Gavrielov 2017-06-07 15:21:06 UTC
Partially verified:

Firefox 52.1.1 (64-bit) - uploaded both qcow2 v3 and raw to both nfs and iscsi
While upload is in progress, I moved the files to a different location and the upload switched state to "Paused by the system", then resumed the upload, and performed the move/resume a few more times.
Works just fine.

I was unable to test this on my chrome browser, because of bug 1459610.

Comment 10 Natalie Gavrielov 2017-06-07 15:22:35 UTC
(In reply to Natalie Gavrielov from comment #9)
> Partially verified:
> 
> Firefox 52.1.1 (64-bit) - uploaded both qcow2 v3 and raw to both nfs and
> iscsi
> While upload is in progress, I moved the files to a different location and
> the upload switched state to "Paused by the system", then resumed the
> upload, and performed the move/resume a few more times.
> Works just fine.
> 
> I was unable to test this on my chrome browser, because of bug 1459610.

Builds used:
ovirt-engine-4.1.3.1-0.1.el7.noarch
ovirt-imageio-proxy-1.0.0-0.el7ev.noarch
ovirt-imageio-common-1.0.0-0.el7ev.noarch

Comment 11 Natalie Gavrielov 2017-06-11 12:31:36 UTC
Verified also using Chrome 55.0.2883.87 (64-bit)
and builds: 
ovirt-engine-4.1.3.2-0.1.el7.noarch
ovirt-imageio-common-1.0.0-0.el7ev.noarch
ovirt-imageio-proxy-1.0.0-0.el7ev.noarch
vdsm-4.19.18-1.el7ev.x86_64
ovirt-imageio-daemon-1.0.0-0.el7ev.noarch