Bug 1524904

Summary: Repeated "Failed to acquire lock: Lease is held by another host." after deployment
Product: [oVirt] ovirt-hosted-engine-ha Reporter: Bernhard Seidl <info>
Component: GeneralAssignee: bugs <bugs>
Status: CLOSED DUPLICATE QA Contact: meital avital <mavital>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 2.2.0CC: bugs, dfediuck, info
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-01-03 11:08:25 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: SLA RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
agent and broker logs from /var/log/ovirt-hosted-engine-ha none

Description Bernhard Seidl 2017-12-12 09:51:35 UTC 
Description of problem:
After installing a three node gluster with self hosted engine and deploying the engine on all cluster members, the members not running the engine repeatedly logging 

VM HostedEngine is down with error. Exit message: resource busy: Failed to acquire lock: Lease is held by another host. 

Version-Release number of selected component (if applicable):
ovirt-node-ng-installer-master-2017121109
4.2.1-0.0.master.20171210113630.git504d08f.el7.centos

How reproducible:
100%

Steps to Reproduce:
1. Install three node cluster using gluster and install self hosted engine
2. Complete cluster setup
3. Complete storage domain setup
4. Set member into local maintenance and trigger reinstall with engine deployment for members not running the engine

Actual results:
Repeated lock errors

Expected results:
No errors

Additional info:
Comment 1 Yaniv Kaul 2017-12-12 15:30:09 UTC 
Can you please attach logs?
Comment 2 Bernhard Seidl 2017-12-18 06:59:23 UTC 
Please excuse me: What kind of logs do you need?
Comment 3 Yaniv Kaul 2017-12-18 07:05:59 UTC 
(In reply to Bernhard Seidl from comment #2)
> Please excuse me: What kind of logs do you need?

Usually, the ovirt-hosted-engine HA logs, which are somewhere under /var/log/ovirt-hosted-engine-*
Comment 4 Bernhard Seidl 2017-12-18 09:05:32 UTC 
Created attachment 1369323 [details]
agent and broker logs from /var/log/ovirt-hosted-engine-ha
Comment 5 Martin Sivák 2017-12-20 11:35:24 UTC 
Can you please add the logs from the other nodes? This one looks like it comes from the host that currently runs the VM.

You can also attach the output of hosted-engine --vm-status from all the nodes.

This is a potential duplicate of https://bugzilla.redhat.com/show_bug.cgi?id=1527394
Comment 6 Martin Sivák 2017-12-20 11:36:12 UTC 
And if it is the same issue then simple systemctl restart ovirt-ha-broker on all nodes might fix it.
Comment 7 Bernhard Seidl 2017-12-22 16:06:39 UTC 
Unfortunately I already reinstalled the setup. I was unable to reproduce this with 4.2.0. I will try again using master branch first week next year.
Comment 8 Bernhard Seidl 2018-01-02 13:16:30 UTC 
I just tested and this seems to be a duplicate of https://bugzilla.redhat.com/show_bug.cgi?id=1527394

Restarting ovirt-ha-broker on all nodes fixed it.

Test Version: ovirt-node-ng-installer-master-2018010109.iso
Comment 9 Martin Sivák 2018-01-03 11:08:25 UTC 
Thanks for checking. We will handle it as duplicate then.

*** This bug has been marked as a duplicate of bug 1527394 ***