Bug 1677500

Summary:	[RFE] Concurrent read of template VM disks fails due to disk lock
Product:	[oVirt] ovirt-engine	Reporter:	AbhayM <abhay.marode>
Component:	Backend.Core	Assignee:	Benny Zlotnik <bzlotnik>
Status:	CLOSED CURRENTRELEASE	QA Contact:	sshmulev
Severity:	high	Docs Contact:
Priority:	high
Version:	4.2.7	CC:	ahadas, bugs, bzlotnik, derez, dfodor, ebenahar, jortialc, mjankula, mtessun, nsoffer, sfishbai, tnisan, ybenshim
Target Milestone:	ovirt-4.5.2	Keywords:	FutureFeature, Improvement
Target Release:	---	Flags:	pm-rhel: ovirt-4.5? mtessun: planning_ack+ pm-rhel: devel_ack+ pm-rhel: testing_ack+
Hardware:	x86_64
OS:	Unspecified
Whiteboard:
Fixed In Version:	ovirt-engine-4.5.2	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:
Clones:	1879391 (view as bug list)		Environment:
Last Closed:	2022-08-30 08:47:42 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	Storage	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1691340, 1707372, 1709496, 1879391

Description AbhayM 2019-02-15 06:00:22 UTC

Description of problem:
In our backup workload, we are trying to read template disks concurrently as we are taking concurrent backup of multiple VMs sharing the same template.
If VMs are dependent clone, while reading template disk using IMAGE IO API, one VM locks the disk and till it is read is over, another process backing up another VM cannot access it.

Version-Release number of selected component (if applicable):


How reproducible:
Create two VMs (VM1, VM2) from template as dependent clones.
Download the all virtual disks for VM1 and start download for VM2 simultaneously. 
Downloading template VM disks for VM2 will fail (initiate transfer API) will fail as disk is already locked.

Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:

We want to initiate transfer for template disks as these are read only transfer.

Additional info:

Comment 1 Daniel Erez 2019-04-07 10:04:24 UTC

@Tal - As the request is to read the same image concurrently, to allow this behaviour,
it would require the introduce some form of a global manager for prepare image 
and teardown image. Thus, as this a major change, I think this should be considered
for a future release. As an alternative, the workaround for this behaviour
should be to download all templates and image layers separately.
Could be done using the 'templates' entity in the API.
I.e. /ovirt-engine/api/templates/<id>/diskattachments

Comment 2 Tal Nisan 2019-04-07 12:46:26 UTC

Fair enough, moving to 4.4 and setting as an RFE

Comment 3 Eyal Shenitzky 2020-06-16 08:40:50 UTC

*** Bug 1728693 has been marked as a duplicate of this bug. ***

Comment 4 Avihai 2020-07-13 14:09:54 UTC

According to Daniel E. comment this is a major change

In order to QE_ACK this we need a clear verification scenario.
Can you please provide one?

Is it only this one or is more required ?

1) Create two VMs (VM1, VM2) from template as dependent clones.
2) Download the all virtual disks for VM1 and start download for VM2 simultaneously. 
3) Downloading template VM disks for VM2 will fail (initiate transfer API) will fail as disk is already locked.

Comment 5 Tal Nisan 2020-08-04 11:25:33 UTC

*** Bug 1428395 has been marked as a duplicate of this bug. ***

Comment 7 Michal Skrivanek 2021-08-20 08:27:27 UTC

This bug/RFE is more than 2 years old and it didn't get enough attention so far, and is now flagged as pending close. 
Please review if it is still relevant and provide additional details/justification/patches if you believe it should get more attention for the next oVirt release.

Comment 8 Eyal Shenitzky 2021-08-23 10:49:42 UTC

This is the upstream bug for bug 1879391.

We should fix it both upstream and downstream.

Comment 12 Shir Fishbain 2022-05-30 20:17:01 UTC

QE doesn't have the capacity to verify this bug during 4.5.1.

Comment 15 sshmulev 2022-07-12 10:48:28 UTC

Following the given steps to reproduce the issue, I couldn't reproduce it on a version without the fix:

Versions:
ovirt-engine-4.5.1-0.62.el8ev
vdsm-4.50.1.1-1.el8ev

Given steps to reproduce:
"
1) Create two VMs (VM1, VM2) from template as dependent clones.
2) Download the all virtual disks for VM1 and start download for VM2 simultaneously. 
3) Downloading template VM disks for VM2 will fail (initiate transfer API) will fail as disk is already locked.
"

I tried to do these steps on a non-fixed version just to see the different behavior on a fixed version.
Maybe something in the steps is missing here?

Comment 17 Benny Zlotnik 2022-07-17 08:20:29 UTC

(In reply to sshmulev from comment #15)
> Following the given steps to reproduce the issue, I couldn't reproduce it on
> a version without the fix:
> 
> Versions:
> ovirt-engine-4.5.1-0.62.el8ev
> vdsm-4.50.1.1-1.el8ev
> 
> Given steps to reproduce:
> "
> 1) Create two VMs (VM1, VM2) from template as dependent clones.
> 2) Download the all virtual disks for VM1 and start download for VM2
> simultaneously. 
> 3) Downloading template VM disks for VM2 will fail (initiate transfer API)
> will fail as disk is already locked.
> "
> 
> I tried to do these steps on a non-fixed version just to see the different
> behavior on a fixed version.
> Maybe something in the steps is missing here?

As discussed, will be verified based on regression automation

Comment 18 sshmulev 2022-08-01 11:26:02 UTC

Verified based on automation regression tests related to download disks and also the rest of the tests of all tiers.

Versions:
ovirt-engine-4.5.2-0.3.el8ev.noarch
vdsm-4.50.2.1-1.el8ev.x86_64

Comment 20 Sandro Bonazzola 2022-08-30 08:47:42 UTC

This bugzilla is included in oVirt 4.5.2 release, published on August 10th 2022.
Since the problem described in this bug report should be resolved in oVirt 4.5.2 release, it has been closed with a resolution of CURRENT RELEASE.
If the solution does not work for you, please open a new bug report.