Created attachment 1517935 [details] logs Description of problem: For a RAW shareable thin disk on file based domain, copy to block based domain passes validation on engine. Starting a VM with this copied disk (COW shareable) passes validation too and fails on vdsm. Version-Release number of selected component (if applicable): ovirt-engine-4.3.0-0.6.alpha2.el7.noarch vdsm-4.30.4-1.el7ev.x86_64 How reproducible: Always Steps to Reproduce: 1. Create a shareable thin disk on a file based domain 2. Copy the disk to a block based domain Actual results: the copied disk is COW shareable: 2019-01-02 13:43:34,015+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (EE-ManagedThreadFactory-engine-Thread-68532) [024df732-e22f-4466-aa3d-83f7669de156] START, CopyImageVDSCommand( C opyImageVDSCommandParameters:{storagePoolId='2288b8b4-06e6-44cc-8294-3f6ceec565f5', ignoreFailoverLimit='false', storageDomainId='a85c9e09-8ace-472e-8e10-17d086a4db84', imageGroupId='cd63a025-1f7a-42b7-8413-1f0d e29f0385', imageId='e7acb617-6e72-4554-b26f-169fb1523808', dstImageGroupId='f450310a-7d39-40f2-a60b-29958851c101', vmId='00000000-0000-0000-0000-000000000000', dstImageId='5363a5d4-aa23-4c4f-96f1-a1f9d4a4847a', imageDescription='', dstStorageDomainId='4ce63bb8-71fb-4936-aff1-905c5e131f95', copyVolumeType='LeafVol', volumeFormat='COW', preallocate='Sparse', postZero='false', discard='true', force='false'}), log id: 2538 5167 3. Attach the copied disk to a VM and start it. Actual results: Start VM passes validation and fails on vdsm: 2019-01-02 13:45:34,616+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ForkJoinPool-1-worker-15) [] EVENT_ID: VM_DOWN_ERROR(119), VM test is down with error. Exit message: unsup ported configuration: shared access for disk 'vda' requires use of supported storage format. 2019-01-02 13:45:15,655+0200 ERROR (vm/a77f7c4f) [virt.vm] (vmId='a77f7c4f-f596-4980-9316-87baf31fbc6d') The vm start process failed (vm:937) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 866, in _startUnderlyingVm self._run() File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 2842, in _run dom = self._connection.defineXML(self._domain.xml) File "/usr/lib/python2.7/site-packages/vdsm/common/libvirtconnection.py", line 131, in wrapper ret = f(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/common/function.py", line 94, in wrapper return func(inst, *args, **kwargs) File "/usr/lib64/python2.7/site-packages/libvirt.py", line 3733, in defineXML if ret is None:raise libvirtError('virDomainDefineXML() failed', conn=self) libvirtError: unsupported configuration: shared access for disk 'vda' requires use of supported storage format 2019-01-02 13:45:15,656+0200 INFO (vm/a77f7c4f) [virt.vm] (vmId='a77f7c4f-f596-4980-9316-87baf31fbc6d') Changed state to Down: unsupported configuration: shared access for disk 'vda' requires use of supported s torage format (code=1) (vm:1675) Expected results: Either of the following or both: 1. Copy a sharealbe thin disk from file to block domain should not pass validation on engine 2. Start VM with shareable COW disk should not pass validation on engine Additional info:
Elad, is shareable COW disks allowed on NFS?
Sparse COW disk creation on a file based domain is available only via REST. Setting such disk as shared is not allowed: 2019-01-07 13:34:41,497+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (default task-190) [disks_create_255543c5-c6d0-4cb7] START, CreateImageVDSCommand( CreateImageVDSCommandParamete rs:{storagePoolId='2288b8b4-06e6-44cc-8294-3f6ceec565f5', ignoreFailoverLimit='false', storageDomainId='a85c9e09-8ace-472e-8e10-17d086a4db84', imageGroupId='f51656fb-9f9c-4d3b-b9ca-201d0baf888b', imageSizeInByte s='3221225472', volumeFormat='COW', newImageId='81105167-b50b-4d9d-936d-395af5997ddb', imageType='Sparse', newImageDescription='{"DiskAlias":"disk_random_test_0713343997","DiskDescription":""}', imageInitialSize InBytes='0'}), log id: 1c034c49 2019-01-07 13:35:40,513+02 WARN [org.ovirt.engine.core.bll.storage.disk.UpdateVmDiskCommand] (default task-187) [93408aee-6db9-41bb-81f0-12415535887e] Validation of action 'UpdateVmDisk' failed for user admin@internal-authz. Reasons: VAR__ACTION__UPDATE,VAR__TYPE__DISK,SHAREABLE_DISK_IS_NOT_SUPPORTED_BY_VOLUME_FORMAT
Hi Jarda, can you have a look on that one? Could you tell us what that libvirt error means: libvirtError: unsupported configuration: shared access for disk 'vda' requires use of supported storage format Also: Which formats are supported for shareable disks then (and which ones are not), as we need to block it in RHV then
Jarda, can you check Martin question? Thanks
(In reply to Martin Tessun from comment #3) > Hi Jarda, > > can you have a look on that one? > Could you tell us what that libvirt error means: > libvirtError: unsupported configuration: shared access for disk 'vda' > requires use of supported storage format > > Also: Which formats are supported for shareable disks then (and which ones > are not), as we need to block it in RHV then <shareable/> should be available only with RAW format, I am not sure, whether the COW format is supported. Peter, can you please have a look? Thanks.
Well, based on this... 416 /** 417 * qemuBlockStorageSourceSupportsConcurrentAccess: 418 * @src: disk storage source 419 * 420 * Returns true if the given storage format supports concurrent access from two 421 * separate processes. 422 */ 423 bool 424 qemuBlockStorageSourceSupportsConcurrentAccess(virStorageSourcePtr src) 425 { 426 /* no need to check in backing chain since only RAW storage supports this */ 427 return src->format == VIR_STORAGE_FILE_RAW; 428 } 429 430 It is really only RAW.
Yes, shareable disks can only be raw as qemu does not to any metadata locking. Regarding block-copy I've recently created a patch which moves the check whether a disk is shareable of the copy target way earlier to when the operation is started to avoid having to cancel the job. It does not seem I've posted the patch som I might add the note later.
So basically we need to prevent marking a disk as shareable unless they are RAW disks and prevent taking a snapshot with those disks
The check is now done when starting the copy job in upstream libvirt: commit 023d69dfc8030b8746bb03cde85623f0e2a4a275 Author: Peter Krempa <pkrempa> Date: Thu Jan 24 17:24:56 2019 +0100 qemu: Move shareable disk check for block copy The writing to an image actually starts when the copy job is initiated, so checking this at the time of the pivot operation is too late. Move the check to qemuDomainBlockCopyCommon. Note that modern qemu would have prevented two writers with qcow2 so the slim possibility of a job started with libvirtd without this patch missing the check is not really worth worrying about.
Verified The Copy shareable thin disk from file to block domain doesn't pass validation on engine ovirt-engine-4.3.4.1-0.1.el7.noarch vdsm-4.30.16-3.el7ev.x86_64
This bugzilla is included in oVirt 4.3.4 release, published on June 11th 2019. Since the problem described in this bug report should be resolved in oVirt 4.3.4 release, it has been closed with a resolution of CURRENT RELEASE. If the solution does not work for you, please open a new bug report.