Bug 1581158 - Live Storage Migration releases lock twice
Summary: Live Storage Migration releases lock twice
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: ovirt-engine
Classification: oVirt
Component: BLL.Storage
Version: future
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: ovirt-4.2.4
: ---
Assignee: Benny Zlotnik
QA Contact: Kevin Alon Goldblatt
URL:
Whiteboard:
: 1568556 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-05-22 09:15 UTC by Benny Zlotnik
Modified: 2018-08-01 19:03 UTC (History)
5 users (show)

Fixed In Version: ovirt-engine-4.2.4.1
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-06-26 08:39:03 UTC
oVirt Team: Storage
rule-engine: ovirt-4.2+


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
oVirt gerrit 90625 0 master MERGED core: fix double lock release for LSM 2018-05-27 16:20:04 UTC
oVirt gerrit 91681 0 ovirt-engine-4.2 MERGED core: fix double lock release for LSM 2018-05-28 14:41:03 UTC

Description Benny Zlotnik 2018-05-22 09:15:42 UTC
Description of problem:

The engine lock is being released twice when LSM is run

Version-Release number of selected component (if applicable):


How reproducible:
100%

Steps to Reproduce:
1. Run LSM
2.
3.

Actual results:


Expected results:


Additional info:
Lock is first released here at the end of live merge

2018-05-22 12:07:28,409+03 INFO  [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-35) [6d710eea-9bb9-4998-acbb-f2c8a45ddd59] Ending command 'org.ovirt.engine.core.bll.snapshots.RemoveSnapshotCommand' successfully.
2018-05-22 12:07:28,430+03 INFO  [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-35) [6d710eea-9bb9-4998-acbb-f2c8a45ddd59] Lock freed to object 'EngineLock:{exclusiveLocks='[9e56302a-81b3-4d27-b5cb-d151ef707504=LIVE_STORAGE_MIGRATION]', sharedLocks=''}'


And then again at the end of LSM
2018-05-22 12:07:30,696+03 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.GetVolumeInfoVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-62) [6d710eea-9bb9-4998-acbb-f2c8a45ddd59] FINISH, GetVolumeInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.storage.DiskImage@807e58f3, log id: 1b79351b
2018-05-22 12:07:30,696+03 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-62) [6d710eea-9bb9-4998-acbb-f2c8a45ddd59] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.storage.DiskImage@807e58f3, log id: 6f8241a9
2018-05-22 12:07:30,704+03 WARN  [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (EE-ManagedThreadFactory-engineScheduled-Thread-62) [6d710eea-9bb9-4998-acbb-f2c8a45ddd59] Trying to release exclusive lock which does not exist, lock key: '9e56302a-81b3-4d27-b5cb-d151ef707504LIVE_STORAGE_MIGRATION'
2018-05-22 12:07:30,704+03 INFO  [org.ovirt.engine.core.bll.storage.lsm.LiveMigrateDiskCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-62) [6d710eea-9bb9-4998-acbb-f2c8a45ddd59] Lock freed to object 'EngineLock:{exclusiveLocks='[9e56302a-81b3-4d27-b5cb-d151ef707504=LIVE_STORAGE_MIGRATION]', sharedLocks=''}'
2018-05-22 12:07:30,719+03 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-62) [6d710eea-9bb9-4998-acbb-f2c8a45ddd59] EVENT_ID: USER_MOVED_DISK_FINISHED_SUCCESS(2,010), User admin@internal-authz finished moving disk vm_Disk1 to domain sd1.

Comment 1 Tal Nisan 2018-05-22 14:29:35 UTC
Please state the effect of this so I'll know how to target, is the second release ignored?

Comment 2 Benny Zlotnik 2018-05-22 14:39:12 UTC
It can cause a potential race since the lock is released prematurely by the child command. There is no actual second release since the lock is already free, the attempt fails with a warning

https://www.mail-archive.com/devel@ovirt.org/msg11683.html

Comment 3 Tal Nisan 2018-05-22 15:01:06 UTC
OK, fix seems quite trivial and safe, let's have it 4.2.4

Comment 4 Yaniv Kaul 2018-05-23 06:50:45 UTC
Severity... ?

Comment 5 Benny Zlotnik 2018-06-12 13:52:35 UTC
For verification: This warning should not appear in the logs:

2018-05-22 12:07:30,704+03 WARN  [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (EE-ManagedThreadFactory-engineScheduled-Thread-62) [6d710eea-9bb9-4998-acbb-f2c8a45ddd59] Trying to release exclusive lock which does not exist, lock key: '9e56302a-81b3-4d27-b5cb-d151ef707504LIVE_STORAGE_MIGRATION'

Comment 8 Kevin Alon Goldblatt 2018-06-13 15:11:28 UTC
Verified with the following code:
-------------------------------------
ovirt-engine-4.2.4.2-0.1.el7_3.noarch
vdsm-4.20.30-1.el7ev.x86_64

Verified with the following scenario:
-------------------------------------
1. Run LSM of 3 different disks >>>>> The InMemoryLockManager warning is no longer reported.


Moving to VERIFIED

Comment 9 Sandro Bonazzola 2018-06-26 08:39:03 UTC
This bugzilla is included in oVirt 4.2.4 release, published on June 26th 2018.

Since the problem described in this bug report should be
resolved in oVirt 4.2.4 release, it has been closed with a resolution of CURRENT RELEASE.

If the solution does not work for you, please open a new bug report.

Comment 10 Ravi Nori 2018-08-01 19:03:14 UTC
*** Bug 1568556 has been marked as a duplicate of this bug. ***


Note You need to log in before you can comment on or make changes to this bug.