Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1581158

Summary: Live Storage Migration releases lock twice
Product: [oVirt] ovirt-engine Reporter: Benny Zlotnik <bzlotnik>
Component: BLL.StorageAssignee: Benny Zlotnik <bzlotnik>
Status: CLOSED CURRENTRELEASE QA Contact: Kevin Alon Goldblatt <kgoldbla>
Severity: medium Docs Contact:
Priority: unspecified    
Version: futureCC: bugs, bzlotnik, lveyde, rnori, tnisan
Target Milestone: ovirt-4.2.4Flags: rule-engine: ovirt-4.2+
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: ovirt-engine-4.2.4.1 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-06-26 08:39:03 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Storage RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Benny Zlotnik 2018-05-22 09:15:42 UTC
Description of problem:

The engine lock is being released twice when LSM is run

Version-Release number of selected component (if applicable):


How reproducible:
100%

Steps to Reproduce:
1. Run LSM
2.
3.

Actual results:


Expected results:


Additional info:
Lock is first released here at the end of live merge

2018-05-22 12:07:28,409+03 INFO  [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-35) [6d710eea-9bb9-4998-acbb-f2c8a45ddd59] Ending command 'org.ovirt.engine.core.bll.snapshots.RemoveSnapshotCommand' successfully.
2018-05-22 12:07:28,430+03 INFO  [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-35) [6d710eea-9bb9-4998-acbb-f2c8a45ddd59] Lock freed to object 'EngineLock:{exclusiveLocks='[9e56302a-81b3-4d27-b5cb-d151ef707504=LIVE_STORAGE_MIGRATION]', sharedLocks=''}'


And then again at the end of LSM
2018-05-22 12:07:30,696+03 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.GetVolumeInfoVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-62) [6d710eea-9bb9-4998-acbb-f2c8a45ddd59] FINISH, GetVolumeInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.storage.DiskImage@807e58f3, log id: 1b79351b
2018-05-22 12:07:30,696+03 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-62) [6d710eea-9bb9-4998-acbb-f2c8a45ddd59] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.storage.DiskImage@807e58f3, log id: 6f8241a9
2018-05-22 12:07:30,704+03 WARN  [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (EE-ManagedThreadFactory-engineScheduled-Thread-62) [6d710eea-9bb9-4998-acbb-f2c8a45ddd59] Trying to release exclusive lock which does not exist, lock key: '9e56302a-81b3-4d27-b5cb-d151ef707504LIVE_STORAGE_MIGRATION'
2018-05-22 12:07:30,704+03 INFO  [org.ovirt.engine.core.bll.storage.lsm.LiveMigrateDiskCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-62) [6d710eea-9bb9-4998-acbb-f2c8a45ddd59] Lock freed to object 'EngineLock:{exclusiveLocks='[9e56302a-81b3-4d27-b5cb-d151ef707504=LIVE_STORAGE_MIGRATION]', sharedLocks=''}'
2018-05-22 12:07:30,719+03 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-62) [6d710eea-9bb9-4998-acbb-f2c8a45ddd59] EVENT_ID: USER_MOVED_DISK_FINISHED_SUCCESS(2,010), User admin@internal-authz finished moving disk vm_Disk1 to domain sd1.

Comment 1 Tal Nisan 2018-05-22 14:29:35 UTC
Please state the effect of this so I'll know how to target, is the second release ignored?

Comment 2 Benny Zlotnik 2018-05-22 14:39:12 UTC
It can cause a potential race since the lock is released prematurely by the child command. There is no actual second release since the lock is already free, the attempt fails with a warning

https://www.mail-archive.com/devel@ovirt.org/msg11683.html

Comment 3 Tal Nisan 2018-05-22 15:01:06 UTC
OK, fix seems quite trivial and safe, let's have it 4.2.4

Comment 4 Yaniv Kaul 2018-05-23 06:50:45 UTC
Severity... ?

Comment 5 Benny Zlotnik 2018-06-12 13:52:35 UTC
For verification: This warning should not appear in the logs:

2018-05-22 12:07:30,704+03 WARN  [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (EE-ManagedThreadFactory-engineScheduled-Thread-62) [6d710eea-9bb9-4998-acbb-f2c8a45ddd59] Trying to release exclusive lock which does not exist, lock key: '9e56302a-81b3-4d27-b5cb-d151ef707504LIVE_STORAGE_MIGRATION'

Comment 8 Kevin Alon Goldblatt 2018-06-13 15:11:28 UTC
Verified with the following code:
-------------------------------------
ovirt-engine-4.2.4.2-0.1.el7_3.noarch
vdsm-4.20.30-1.el7ev.x86_64

Verified with the following scenario:
-------------------------------------
1. Run LSM of 3 different disks >>>>> The InMemoryLockManager warning is no longer reported.


Moving to VERIFIED

Comment 9 Sandro Bonazzola 2018-06-26 08:39:03 UTC
This bugzilla is included in oVirt 4.2.4 release, published on June 26th 2018.

Since the problem described in this bug report should be
resolved in oVirt 4.2.4 release, it has been closed with a resolution of CURRENT RELEASE.

If the solution does not work for you, please open a new bug report.

Comment 10 Ravi Nori 2018-08-01 19:03:14 UTC
*** Bug 1568556 has been marked as a duplicate of this bug. ***