Bug 1957791 - [MTV][Warm] The migrated VM on target side should be powered off/on accordingly to the source VM's last power state during warm migration
Summary: [MTV][Warm] The migrated VM on target side should be powered off/on according...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Container Native Virtualization (CNV)
Classification: Red Hat
Component: V2V
Version: 4.8.1
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: 4.8.2
Assignee: Sam Lucidi
QA Contact: Maayan Hadasi
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-05-06 13:41 UTC by Maayan Hadasi
Modified: 2021-09-21 11:07 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-09-21 11:06:44 UTC
Target Upstream Version:
Embargoed:
amastbau: needinfo-


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github kubevirt vm-import-operator pull 492 0 None closed Defer storing the VM power status until immediately before stopping it 2021-07-12 14:19:26 UTC
Github kubevirt vm-import-operator pull 507 0 None None None 2021-08-27 06:22:53 UTC
Red Hat Product Errata RHSA-2021:3598 0 None None None 2021-09-21 11:07:53 UTC

Description Maayan Hadasi 2021-05-06 13:41:07 UTC
Description of problem:
The migrated VM is powered off on the target side once the source VM was off when migration started. Though the source VM turned on during warm migration, before the cutover time


Version-Release number of selected component (if applicable):
mtv-operator-bundle-container-2.0.0-8 iib 72981


How reproducible:
100%


Steps to Reproduce:
1. Power off the source VM
2. Warm migrate it
3. During migration, before the cutover time, turn on the source VM


Actual results:
When warm migration has succeeded the target VM is powered off


Expected results:
When warm migration has succeeded the target VM is powered on


Additional info:

Comment 1 Fabien Dupont 2021-05-08 21:35:21 UTC
@slucidi do you think you have enough time to fix this in MTV 2.0.0.

Comment 2 Fabien Dupont 2021-05-10 12:16:56 UTC
The problem is in VMIO, not MTV. Adjusting the product/component.

Comment 3 Fabien Dupont 2021-05-11 19:36:02 UTC
The fix should be in build hco-bundle-registry-container-v2.6.3-9.

Comment 4 Amos Mastbaum 2021-05-19 09:45:02 UTC
v2.6.3-16 - failed
The vm was turned off.

Comment 5 Amos Mastbaum 2021-05-19 09:47:02 UTC
The source vm was turned off when the plan started and was turned on after a few pre-copies before cutover.
The destination vm was not turned on.

Comment 6 Sam Lucidi 2021-05-19 14:58:31 UTC
Amos, could you please share the full yaml of the VirtualMachineImport resource that was created for that VM? Thank you!

Comment 7 Fabien Dupont 2021-05-19 15:54:43 UTC
We're too late for 2.6.3. Re-targeting to 2.6.4.

Comment 8 Fabien Dupont 2021-06-07 14:23:42 UTC
It's probable that the fix was shipped in 2.6.4. Would you mind trying to reproduce it with 2.6.4?

Comment 9 Ric Garcia 2021-06-07 16:57:36 UTC
2.6.4 doesn't include any bug fixes, just some annotations to allow direct version upgrades from certain 2.5.z releases to 2.6.4. I am changing the target release to 2.6.5 so that you can check if it has been fixed in that version, if not the next one will be 2.6.6.

Comment 11 Maayan Hadasi 2021-08-11 14:18:34 UTC
The issue is reproducible with CVN 4.8.1-17, MTV 2.1.0-44

The destination VM is not powered on at the end of warm migration in both cases:
1. Source VM was on during the whole migration
2. Source VM was off and turned on before cutover


Moving to ASSIGNED

Comment 13 Fabien Dupont 2021-08-31 14:26:48 UTC
Can you please verify with hco-bundle-registry-container-v4.8.2-10 ?

Comment 14 Maayan Hadasi 2021-09-01 13:04:01 UTC
Hi @slucidi, 

Though this issue looks like it was fixed, there are 2 strange behaviors in warm migration:
1. Once a warm migration is started - I can see under VMware tasks: "Power On virtual machine"/"Power Off virtual machine",
depends on the source VM's power state, On/Off accordingly.

2. Once the migration is done and I delete the vmimport CR - if the source VM was On during migration --> the source VM is powered on and only at this point the snapshots are removed.
For source VM that was Off during migration - the snapshots are removed as expected, at the end of the warm migration


Tested with:
CNV-4.8.2-12
MTV-2.1.0-53

Comment 15 Maayan Hadasi 2021-09-01 14:32:57 UTC
(In reply to Maayan Hadasi from comment #14)
> Hi @slucidi, 
> 
> 1. Once a warm migration is started - I can see under VMware tasks: "Power
> On virtual machine"/"Power Off virtual machine",
> depends on the source VM's power state, On/Off accordingly.
I don't know what was changed in the meantime, but I cannot reproduce this one, after seeing it so many times.

Comment 16 Amos Mastbaum 2021-09-02 11:05:56 UTC
I believe this confirms #2 in Comment 14  was introduced in 4.8.2-12:

- in 4.8.2-5  the snapshots are deleted once the VMIO import is completed.
- in 4.8.2-12 the snapshots are only removed after deleting VMIO CRs (I also turned on the source VM before deleting the CRs, but the snapshots were only removed after deleting the VMIO CRs).

Comment 17 Amos Mastbaum 2021-09-02 12:05:50 UTC
(In reply to Amos Mastbaum from comment #16)
> I believe this confirms #2 in Comment 14  was introduced in 4.8.2-12:
> 
> - in 4.8.2-5  the snapshots are deleted once the VMIO import is completed.
> - in 4.8.2-12 the snapshots are only removed after deleting VMIO CRs (I also
> turned on the source VM before deleting the CRs, but the snapshots were only
> removed after deleting the VMIO CRs).

same behaviour when the source VM remains turned off.not related to the source VM state

Comment 18 Maayan Hadasi 2021-09-05 07:06:54 UTC
Two regression BZs have been filed:
Bug 2001270 - [VMIO] [Warm from Vmware] Snapshot files are not deleted after Succesfull Import
Bug 2001281 - [VMIO] [Warm from VMware] Source VM should not be turned ON if vmio import is removed

Thanks Amos.

Moving this issue to VERIFIED

Comment 19 Maayan Hadasi 2021-09-05 07:13:26 UTC
Verified as fixed.

Versions:
CNV 4.8.2-12 iib 104410
MTV 2.1.0-53 iib 99664

Comment 24 errata-xmlrpc 2021-09-21 11:06:44 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Virtualization 4.8.2 Images security and bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:3598


Note You need to log in before you can comment on or make changes to this bug.