Created attachment 1772510 [details] host logs Description of problem: I can’t start vm backup after upgrade to 4.4.5 Backup failed when trying to create scratch disk Version-Release number of selected component (if applicable): Ovirt-4.4.5.11 vdsm-4.40.50.10-1.el8ev.x86_64 ovirt-imageio-daemon-2.1.1-1.el8ev.x86_64 libvirt-daemon-6.6.0-13.2.module+el8.3.1+10483+85317cf0.x86_64 qemu-kvm-5.1.0-21.module+el8.3.1+10464+8ad18d1a.x86_64 How reproducible: I have two RHV nodes cluster with shared NFS storage domain ugraded from 4.4.4 to 4.4.5 Steps to Reproduce: 1. Start vm backup Actual results: Backup failed Expected results: Successful backup Additional info: 2021-04-16 15:38:15,802+0200 ERROR (jsonrpc/3) [storage.TaskManager.Task] (Task='0cad3f0c-4fa3-4ac5-84bb-12d8f28ce9f5') Unexpected error (task:880) Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/vdsm/storage/task.py", line 887, in _run return fn(*args, **kargs) File "<decorator-gen-61>", line 2, in createVolume File "/usr/lib/python3.6/site-packages/vdsm/common/api.py", line 50, in method ret = func(*args, **kwargs) File "/usr/lib/python3.6/site-packages/vdsm/storage/hsm.py", line 1510, in createVolume add_bitmaps=addBitmaps) File "/usr/lib/python3.6/site-packages/vdsm/storage/sd.py", line 1002, in validateCreateVolumeParams add_bitmaps=add_bitmaps) File "/usr/lib/python3.6/site-packages/vdsm/storage/sd.py", line 649, in validateCreateVolumeParams raise se.InvalidParameterException("DiskType", diskType) vdsm.storage.exception.InvalidParameterException: Invalid parameter: 'DiskType=SCRD'
(In reply to Yury.Panchenko from comment #0) ... > vdsm.storage.exception.InvalidParameterException: Invalid parameter: > 'DiskType=SCRD' The SCRD disk type was added in: commit 7c81c72af24d1455e773569294a14896a2850104 constants.py: add backup scratch disk type to content type $ git describe 7c81c72af24d1455e773569294a14896a2850104 v4.40.50.2-9-g7c81c72af $ xzgrep 'I am' vdsm.log* vdsm.log.3.xz:2021-04-14 09:34:02,410+0200 INFO (MainThread) [vds] (PID: 2592558) I am the actual vdsm 4.40.40.1 pan-rhv44rel2 (4.18.0-240.10.1.el8_3.x86_64) (vdsmd:155) vdsm.log.3.xz:2021-04-14 09:33:34,893+0200 INFO (MainThread) [vds] (PID: 6220) I am the actual vdsm 4.40.40.1 pan-rhv44rel2 (4.18.0-240.15.1.el8_3.x86_64) (vdsmd:155) You are not running vdsm 4.40.50.11. In oVirt 4.4.5, backup requires hosts running 4.4.5 (vdsm 4.40.50). The issue is is not failing earlier when detecting that the host is too old, but since incremental backup is still tech preview I think this is ok.
Yury, Please update your hosts to 4.4.5 (vdsm 4.40.50) and close the bug if all works well.
> Yury, Please update your hosts to 4.4.5 (vdsm 4.40.50) and close the bug if all works well. Yes, i found that upgrade of second via UI host wans't success. > The issue is is not failing earlier when detecting that the host > is too old, but since incremental backup is still tech preview I think > this is ok. So, can you check required versions before starting incremental backup? thanks
I confirm that backup works normally after upgrate host to 4.4.5
There is nothing to verify here in my understanding. This should be closed as 'not a bug' due to not correct VDSM version usage.
I've added a validation before the live backup starts. If the engine version is 4.4.6 and the host VDSM version < 4.4.5 the validation will fail. So the steps to reproduce are: Having a 4.4.6 environment and a host with VDSM version < 4.4.5. 1) Create a VM with disk enabled for incremental backup. 2) Run the VM and make sure it runs on the host with the outdated VDSM version. 3) Start a backup. Backup should fail with a proper error.
Verified on a special setup with newest engine, and outdated host. engine: rhv-release-4.4.5-11-001.noarch Host with an older vdsm version: vdsm-4.40.38-1.el8ev.x86_64 The host resides on a 4.5 cluster level compatibility. When attempting to make a full backup on a disk of a VM which resides on this host, we are getting the following ERROR message on the engine side: 2021-05-05 13:17:07,545+03 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default task-8) [] Operation Failed: [Cannot backup VM. Host (storage-ge15-vdsm1.lab.eng.tlv2.redhat.com) is running on a version lower than 4.4.5. Please upgrade the host in order to use incremental backup.]
This bugzilla is included in oVirt 4.4.6 release, published on May 4th 2021. Since the problem described in this bug report should be resolved in oVirt 4.4.6 release, it has been closed with a resolution of CURRENT RELEASE. If the solution does not work for you, please open a new bug report.