Bug 1917956 - After restore, vm has warning Pending virtual machine changes : cluster cpu type
Summary: After restore, vm has warning Pending virtual machine changes : cluster cpu type
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: ovirt-engine
Classification: oVirt
Component: BLL.Virt
Version: 4.4.3.12
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: ovirt-4.4.6
: 4.4.6.4
Assignee: Arik
QA Contact: sshmulev
bugs@ovirt.org
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-01-19 18:04 UTC by Yury.Panchenko
Modified: 2021-06-14 06:02 UTC (History)
9 users (show)

Fixed In Version: ovirt-engine-4.4.6.4
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-05-05 05:35:52 UTC
oVirt Team: Virt
Embargoed:
pm-rhel: ovirt-4.4+


Attachments (Terms of Use)
vm xml config before and after restore (17.04 KB, application/zip)
2021-01-19 18:04 UTC, Yury.Panchenko
no flags Details
logs and configuration screen (1003.72 KB, application/zip)
2021-01-20 19:48 UTC, Yury.Panchenko
no flags Details
vm query output (10.09 KB, text/plain)
2021-01-22 09:45 UTC, Yury.Panchenko
no flags Details


Links
System ID Private Priority Status Summary Last Updated
oVirt gerrit 114210 0 master MERGED core: command-level lock for import-vm-from-configuration 2021-04-12 07:44:51 UTC

Description Yury.Panchenko 2021-01-19 18:04:07 UTC
Created attachment 1748797 [details]
vm xml config before and after restore

Description of problem:
After restore, vm has warning Pending virtual machine changes : cluster cpu type

Version-Release number of selected component (if applicable):
ovirt 4.4.3


Red Hat Virtualization Host 4.4.3 (el8.3) 4.18.0 - 240.1.1.el8_3.x86_64
KVM Version:5.1.0 - 14.module+el8.3.0+8438+644aff69
LIBVIRT Version:libvirt-6.6.0-7.module+el8.3.0+8424+5ea525c5
VDSM Version:vdsm-4.40.35.1-1.el8ev


How reproducible:


Steps to Reproduce:
1.Backup vm and delete it.
2.Create new vm from backup OVF
3.Restore all required data

Actual results:
Chipset and CPU showed as cluster default, but vm has warning about incorrect CPU type

Expected results:
Vm without warning

Additional info:
if manually change chipset or CPU type warning is gone

Comment 1 Arik 2021-01-20 08:45:51 UTC
It doesn't mean the CPU type is incorrect, it means that the cluster CPU type has changed and the VM is running with the previous setting.

Comment 2 Arik 2021-01-20 11:10:41 UTC
I closed it since that's the expected behaviour but of course, if you haven't made any change to the cluster and don't see why its CPU type has changed, feel free to open.
In that case, I'd suggest first to restore the VM from the backup and run it - to see if this message appears when there's no change to the cluster.

Comment 3 Yury.Panchenko 2021-01-20 12:12:35 UTC
Hello, Arik.
> I closed it since that's the expected behaviour but of course, if you haven't made any change to the cluster and don't see why its CPU type has changed, feel free to open.
> In that case, I'd suggest first to restore the VM from the backup and run it - to see if this message appears when there's no change to the cluster.
Yes, i don't made any changes. And after vm reboot this warning still present.

Comment 4 Arik 2021-01-20 16:30:50 UTC
OK, so please provide engine.log and either database dump or the output of
1. select name, cpu_name, cpu_flags from cluster;
2. select vm_guid, vm_name, cpu_name from vms;
Thanks!

Comment 5 Yury.Panchenko 2021-01-20 19:48:48 UTC
Created attachment 1749154 [details]
logs and configuration screen

Comment 6 Arik 2021-01-20 22:45:03 UTC
Thanks, and can you please provide the output of (assuming the vm is def-cent82):
select cpu_name, cluster_cpu_verb from vms where vm_name='def-cent82';

Comment 7 Yury.Panchenko 2021-01-21 14:52:49 UTC
Hello, Arik.

[root@pan-rhv44man ~]# psql engine -U engine -c "select cpu_name, cluster_cpu_verb from vms where vm_name='def-cent82'"

 cpu_name | cluster_cpu_verb
----------+------------------
          | SandyBridge

Comment 8 Arik 2021-01-21 17:13:44 UTC
Hi Yury,
Was that output taken while the VM is running?

Comment 9 Yury.Panchenko 2021-01-21 17:19:15 UTC
yes

Comment 10 Arik 2021-01-21 22:26:38 UTC
OK, so that explains it but I can't reproduce that.
Can you please provide the whole configuration of that VM using:
select * from vms where vm_name='def-cent82'

Comment 11 Yury.Panchenko 2021-01-22 09:45:57 UTC
Created attachment 1749683 [details]
vm query output

Hello, Arik.
query output in attach

Comment 12 Arik 2021-01-22 14:44:22 UTC
Thanks Yury, now it's clear.
Looking at the configuration of the running VM, it's not only the CPU name that is missing but also other fields like boot_time and runtime_name.
This can be explained by the log:
2021-01-18 12:22:32,268+01 INFO  [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-25) [5b62fe8a] VM 'ff8e01d6-f781-4bee-9bfa-1ec3ec87713f'(def-cent82) was unexpectedly detected as 'PoweringUp' on VDS '9d30eb4c-c4d9-4eec-bc1f-d49cd225e200'(pan-rhv44rel2) (expected on 'null')

When we run the VM, the engine changes few fields like cpu_name, boot_time, runtime_name, then calls VDSM to run the VM and if the call succeeds, it stores the updated configuration in the database. But in this case, those updates are overridden by the monitoring.
1. The monitoring reads the VM from the database before the above mentioned changes are persisted
2. The engine stores those changes to the database
3. The monitoring makes changes to the configuration that was read in #1 according to the information we get from the host and persist it

It's doesn't seem to be something new, but now it's visible because when cpu_name is empty in the database, it leads to showing that message (that the CPU type doesn't comply with the cluster's CPU type) in the webadmin.

Comment 13 Pavan Chavva 2021-02-01 12:43:33 UTC
Need feedback from Yash

Comment 14 Arik 2021-02-01 13:38:28 UTC
That's a RHV issue, nothing needed from the platform side for this.

Comment 15 Arik 2021-04-11 19:33:04 UTC
OK, so what happened here is that ImportVmFromConfiguration was called and then RunVm.
The problem is that ovirt-engine allowed RunVm to be executed before ImportVmFromConfiguration is finished (while the latter is in its async execution phase) and so when ImportVmFromConfiguration finishes it changes the VM to Down and also clears its CPU type and other information that is set right after calling to start the VM, although the VM is running.

Comment 19 sshmulev 2021-04-27 11:04:25 UTC
The bug has been verified with success according to the following steps of Full backup API flow:

1. Attach snapshots disk of source VM to backup VM
2. Start source and backup VMs
3. Make sure tempfile (i.e. /var/lib/vdsm/transient/) is created on the
   host that is running the backup VM
4. Backup disk with backup software on backup VM
5. Backup OVF of source VM on backup VM
6. Detach snapshot disk of source VM from backup VM
7. Remove source VM
8. Create new VM (to be restored from OVF)
9. Restore source VM that has backup to newly created VMs

Version :
ovirt-engine-4.4.6.5-447.gd80dda7.9.el8ev.noarch

Expected result:
VM should be running without the warning "Pending virtual machine changes : cluster cpu type" after starting to restore it from the backup.

Actual result:
No warnings or error log during and after the restore from backup.

Comment 28 Arik 2021-04-29 14:50:38 UTC
Yury, would you be able to verify this bug with the automation scripts you use?

Comment 30 Sandro Bonazzola 2021-05-05 05:35:52 UTC
This bugzilla is included in oVirt 4.4.6 release, published on May 4th 2021.

Since the problem described in this bug report should be resolved in oVirt 4.4.6 release, it has been closed with a resolution of CURRENT RELEASE.

If the solution does not work for you, please open a new bug report.

Comment 31 Yury.Panchenko 2021-05-05 12:39:08 UTC
Hello, Arik

> Yury, would you be able to verify this bug with the automation scripts you use?
I didn't face with that bug in the latest version

Comment 32 Arik 2021-05-05 12:43:34 UTC
(In reply to Yury.Panchenko from comment #31)
> Hello, Arik
> 
> > Yury, would you be able to verify this bug with the automation scripts you use?
> I didn't face with that bug in the latest version

Thanks!


Note You need to log in before you can comment on or make changes to this bug.