Bug 1967715 - "Cannot acquire state change lock" after a VM is destroyed during migration
Summary: "Cannot acquire state change lock" after a VM is destroyed during migration
Keywords:
Status: CLOSED DUPLICATE of bug 1949869
Alias: None
Product: Red Hat Enterprise Linux Advanced Virtualization
Classification: Red Hat
Component: libvirt
Version: 8.4
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: rc
: 8.5
Assignee: Jiri Denemark
QA Contact: Fangge Jin
URL:
Whiteboard:
Depends On:
Blocks: 1966121
TreeView+ depends on / blocked
 
Reported: 2021-06-03 17:01 UTC by Milan Zamazal
Modified: 2021-07-16 15:12 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-07-16 15:12:00 UTC
Type: Bug
Target Upstream Version:
Embargoed:
pm-rhel: mirror+


Attachments (Terms of Use)
libvirtd.log (444.46 KB, application/x-xz)
2021-06-03 17:01 UTC, Milan Zamazal
no flags Details

Description Milan Zamazal 2021-06-03 17:01:41 UTC
Created attachment 1788873 [details]
libvirtd.log

Description of problem:

When a VM is migrating to another host and the VM is destroyed on the source host during the migration, some libvirt calls on the given VM start failing with 

  Timed out during operation: cannot acquire state change lock (held by monitor=remoteDispatchDomainMigratePerform3Params)

This continues until libvirtd is restarted, even if the VM is no longer present on the host. It prevents the given VM from starting on the host again.

Version-Release number of selected component (if applicable):

libvirt-daemon-kvm-7.0.0-14.module+el8.4.0+10886+79296686.x86_64
qemu-kvm-5.2.0-16.module+el8.4.0+10806+b7d97207.x86_64

How reproducible:

100% on my laptop (in RHV running in a nested virtualization).

Steps to Reproduce:
1. Migrate VM to another host.
2. While the migration is still running, run `virsh destroy' on the VM on the source host.
3. Try to start the same VM on the given host, or start it on another host and try to migrate it to the original host, or try running `virsh undefine' on the VM, even if it is not present on the host anymore.

Actual results:

The operations fail with "Timed out during operation: cannot acquire state change lock (held by monitor=remoteDispatchDomainMigratePerform3Params)".

Expected results:

The VM can be wiped out from the host (if needed) and started there normally again.

Additional info:

Attaching libvirtd.log demonstrating the problem.

A very similar problem has been observed in BZ 1966121 when migrating VMs (without destroying them during the migrations) on POWER. This one is not easily reproducible and has been observed only about twice. From the logs, it looks the same, but I don't have any idea whether it can have the same internal cause. Please see BZ 1966121 for more information and logs.

Comment 1 Jaroslav Suchanek 2021-06-04 09:57:12 UTC
Jiri, have a look please. Thanks.

Comment 2 Fangge Jin 2021-06-07 04:40:41 UTC
It seems same as Bug 1949869 - Migration hangs if vm is shutdown during live migration

Comment 3 Arik 2021-06-20 13:23:39 UTC
(In reply to Fangge Jin from comment #2)
> It seems same as Bug 1949869 - Migration hangs if vm is shutdown during live
> migration

Yes, same flow but it seems that bz 1949869 focuses on the migration failure which is not that important and thus the bug is set with low severity and priority while here the focus is on the lock that prevents the VM from running again on that host, thus the high severity of this bz. If shut down from within the guest results in the same situation then yeah but then bz 1949869 should be set with high severity

Comment 4 Jiri Denemark 2021-07-16 15:12:00 UTC

*** This bug has been marked as a duplicate of bug 1949869 ***


Note You need to log in before you can comment on or make changes to this bug.