Hide Forgot
In RHEV 3.3 there seems to be a race condition in which a VM remove command is executed when in fact the guest is migrating between hosts. This results in the guest being removed from the source host but coming up on destination. Disks are also removed so the guest is actually unusable and may hang, appearing on the manager with "external-" prefix. The VM creation/removal is done via the API.
exact race flow, for documentation: migration ends, and vm moves to up on the destination. in the engine, there are 2 monitoring threads that run in parallel (src,dst) dst finish first, and move the vm to up, and release the migration lock. then src finish, but override the status to migratingTo (hand-over), because it is not aware to the other update. now we have vm in migratingTo, without a lock. resolution is to mark this status as a 'running' status, like migratingFrom, and user will be blocked from removing based on vm status, when this case happens. i was able to reproduce this (using debugger).
The fix for bug 947965 also fix this issue, as it will not allow removing a vm in migrating-to, and that fix is much safer than the fix proposed in this bug.
Hi, I am not sure if the following should be part of this bug OR a new bug. Suddenly some virtual machines appeared in Manager, prefixed with 'external'. It seems they were removed from Export Domain but now imported again. All the VMs/Windows guests are from storage_domain="FC-NetAPP-Windows0" engine=> SELECT vm_id,vm_name,log_time,log_type_name,message from audit_log where message like '%external%'; vm_id | vm_name | log_time | log_type_name | message --------------------------------------+------------------------------+----------------------------+---------------+----------------------------------------------------------- 5ef17f98-6ac4-4cd7-ba14-4f52dfac8bec | external-eevrt-002-big-local | 2015-05-26 16:39:10.603-04 | USER_ADD_VM | VM external-eevrt-002-big-local was created by <UNKNOWN>. bd1e8f7f-db44-4207-8215-6394a239ee3a | external-skoor0-big-local | 2015-05-26 16:39:10.453-04 | USER_ADD_VM | VM external-skoor0-big-local was created by <UNKNOWN>. cb93d661-3fc7-4d13-beb4-765a2867febc | external-2008R2TEST | 2015-05-27 14:26:47.257-04 | USER_ADD_VM | VM external-2008R2TEST was created by <UNKNOWN>. 474a70bb-8ba2-4b85-8aa8-8a79fbf18a15 | external-win2008R2-temp | 2015-05-27 14:28:37.111-04 | USER_ADD_VM | VM external-win2008R2-temp was created by <UNKNOWN>. 4b76def7-74b6-4a0f-8a69-8a339df82144 | external-es3-test-big-local | 2015-05-27 14:28:36.776-04 | USER_ADD_VM | VM external-es3-test-big-local was created by <UNKNOWN>. 908047a3-8d05-4273-8314-61023996e081 | external-es1.cloud.big.local | 2015-05-27 14:28:36.93-04 | USER_ADD_VM | VM external-es1.cloud.big.local was created by <UNKNOWN>. 962eb781-4dfa-46a0-a43a-957c54dabc11 | external-sql-dev-big-local | 2015-05-27 14:28:36.588-04 | USER_ADD_VM | VM external-sql-dev-big-local was created by <UNKNOWN>. (7 rows) engine.log ----------- 2015-05-27 21:28:33,157 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-97) [3c0d402c] Importing VM sql-dev-big-local as external-sql-dev-big-local, as it is running on the on Host, but does not exist in the engine. 2015-05-27 21:28:33,157 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-97) [3c0d402c] Importing VM es3-test-big-local as external-es3-test-big-local, as it is running on the on Host, but does not exist in the engine. 2015-05-27 21:28:33,157 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-97) [3c0d402c] Importing VM es1.cloud.big.local as external-es1.cloud.big.local, as it is running on the on Host, but does not exist in the engine. 2015-05-27 21:28:33,157 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-97) [3c0d402c] Importing VM win2008R2-temp as external-win2008R2-temp, as it is running on the on Host, but does not exist in the engine. ... 2015-05-27 21:28:36,594 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-97) [487642b6] Correlation ID: 487642b6, Call Stack: null, Custom Event ID: -1, Message: VM external-sql-dev-big-local was created by <UNKNOWN>
it's not a bug, the engine imports vms detected running on the host and allows you to see them and manage them (partially). these are probably vms that were run by some kind of previous setup and were not cleared well (usually when force removing host or storage, or engine-reinstall) please mind the difference between this case of automatically importing vms that are RUNNING on the host and regular import from export domain.
Verify with: RHEVM: rhevm-tools-3.6.0.3-0.1 vdsm:vdsm-4.17.10.1-0.el7ev libvirt: libvirt-1.2.17-13.el7 Scenario: 1. Migration VM 2. While VM being migrate send request via rest to delete VM. Results: Delete action failed. <fault> <reason>Operation Failed</reason> <detail>[Cannot remove VM. VM golden_env_mixed_virtio_0 is being migrated.]</detail> </fault>
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHEA-2016-0376.html