Bug 1522832
| Summary: | Hot-plug of a VM lease task hang forever when power-off the VM before lease creation finished | ||
|---|---|---|---|
| Product: | [oVirt] ovirt-engine | Reporter: | Eyal Shenitzky <eshenitz> |
| Component: | BLL.Storage | Assignee: | Eyal Shenitzky <eshenitz> |
| Status: | CLOSED CURRENTRELEASE | QA Contact: | Elad <ebenahar> |
| Severity: | medium | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 4.2.0 | CC: | amureini, aperotti, bugs, ebenahar, eshenitz, lveyde, ratamir |
| Target Milestone: | ovirt-4.1.9 | Flags: | rule-engine:
ovirt-4.1+
rule-engine: ovirt-4.2+ |
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | ovirt-engine-4.1.9 | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2018-01-24 10:41:03 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | Storage | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
(In reply to Eyal Shenitzky from comment #0) > Version-Release number of selected component (if applicable): > 4.2.1 - from commit 098fe088d7b89aa9fe5f9a4b24d17d2705fc5d5e So this is a regression compared to 4.1.z, right? Actually no, this is just another scenario that I found while working on the other VM leases bugs. This is not a regression. Powered off a VM while lease hotplug. After few seconds the task cleared and VM start succeeded. Used: rhevm-4.1.9.1-0.1.el7.noarch vdsm-4.19.45-1.el7ev.x86_64 This bugzilla is included in oVirt 4.1.9 release, published on Jan 24th 2018. Since the problem described in this bug report should be resolved in oVirt 4.1.9 release, published on Jan 24th 2018, it has been closed with a resolution of CURRENT RELEASE. If the solution does not work for you, please open a new bug report. |
Description of problem: When VM is powered off during update that includes addition of a VM lease, The hot-plug action will fail because the VM is in state 'Down'. This occurs the VM powered off before that the lease created successfully, therefore, the hot-plug which take place at the end of the lease creation will fail and the task will hang forever because there the engine will retry to hotplug the lease endlessly. Engine Log: 2017-12-06 15:08:40,214+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HotPlugLeaseVDSCommand] (EE-ManagedThreadFactory-engine-Thread-291) [2b47f957] Failed in 'HotPlugLeaseVDS' method 2017-12-06 15:08:40,223+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-291) [2b47f957] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM h1 command HotPlugLeaseVDS failed: Virtual machine does not exist: {'vmId': u'e6a2c125-209d-4c9c-ac23-a94c6f4ad895'} 2017-12-06 15:08:40,224+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HotPlugLeaseVDSCommand] (EE-ManagedThreadFactory-engine-Thread-291) [2b47f957] Command 'org.ovirt.engine.core.vdsbroker.vdsbroker.HotPlugLeaseVDSCommand' return value 'StatusOnlyReturn [status=Status [code=1, message=Virtual machine does not exist: {'vmId': u'e6a2c125-209d-4c9c-ac23-a94c6f4ad895'}]]' 2017-12-06 15:08:40,224+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HotPlugLeaseVDSCommand] (EE-ManagedThreadFactory-engine-Thread-291) [2b47f957] HostName = h1 2017-12-06 15:08:40,224+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HotPlugLeaseVDSCommand] (EE-ManagedThreadFactory-engine-Thread-291) [2b47f957] Command 'HotPlugLeaseVDSCommand(HostName = h1, LeaseVDSParameters:{hostId='1223b76b-d48c-4de2-863b-5aa273addf06', vmId='e6a2c125-209d-4c9c-ac23-a94c6f4ad895'})' execution failed: VDSGenericException: VDSErrorException: Failed to HotPlugLeaseVDS, error = Virtual machine does not exist: {'vmId': u'e6a2c125-209d-4c9c-ac23-a94c6f4ad895'}, code = 1 2017-12-06 15:08:40,224+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HotPlugLeaseVDSCommand] (EE-ManagedThreadFactory-engine-Thread-291) [2b47f957] FINISH, HotPlugLeaseVDSCommand, log id: 40050e20 2017-12-06 15:08:40,224+02 ERROR [org.ovirt.engine.core.bll.storage.lease.AddVmLeaseCommand] (EE-ManagedThreadFactory-engine-Thread-291) [2b47f957] Failure in hot plugging a lease to VM e6a2c125-209d-4c9c-ac23-a94c6f4ad895, message: EngineException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException: VDSGenericException: VDSErrorException: Failed to HotPlugLeaseVDS, error = Virtual machine does not exist: {'vmId': u'e6a2c125-209d-4c9c-ac23-a94c6f4ad895'}, code = 1 (Failed with error noVM and code 1) 2017-12-06 15:08:40,277+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-291) [2b47f957] EVENT_ID: HOT_PLUG_LEASE_FAILED(1,220), Failed to hot plug lease to the VM vm1. The VM is running without a VM lease. Version-Release number of selected component (if applicable): 4.2.1 - from commit 098fe088d7b89aa9fe5f9a4b24d17d2705fc5d5e How reproducible: 100% Steps to Reproduce: 1. Create a VM with disk and set the resume behavior to Kill 2. Run the VM 3. Add a lease to the VM and power-off the VM immediately after Actual results: The task for hotplug the lease remains forever. Expected results: The hotplug should failed and the task should end nicely Additional info: