Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1662915

Summary: Copy a shareable thin provisioned disk is allowed, leaving a new COW shareable disk. Hence, start VM with such a disk attached fails on vdsm
Product: [oVirt] ovirt-engine Reporter: Elad <ebenahar>
Component: BLL.StorageAssignee: Ahmad Khiet <akhiet>
Status: CLOSED CURRENTRELEASE QA Contact: Shir Fishbain <sfishbai>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 4.3.0CC: aefrat, bugs, ebenahar, frolland, jsuchane, mtessun, pkrempa, tnisan
Target Milestone: ovirt-4.3.4Keywords: Automation
Target Release: 4.3.4Flags: rule-engine: ovirt-4.3+
Hardware: x86_64   
OS: Unspecified   
Whiteboard:
Fixed In Version: ovirt-engine-4.3.4 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-06-11 06:24:04 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Storage RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
logs none

Description Elad 2019-01-02 11:58:54 UTC
Created attachment 1517935 [details]
logs

Description of problem:
For a RAW shareable thin disk on file based domain, copy to block based domain passes validation on engine. 
Starting a VM with this copied disk (COW shareable) passes validation too and fails on vdsm.


Version-Release number of selected component (if applicable):
ovirt-engine-4.3.0-0.6.alpha2.el7.noarch
vdsm-4.30.4-1.el7ev.x86_64


How reproducible:
Always

Steps to Reproduce:
1. Create a shareable thin disk on a file based domain
2. Copy the disk to a block based domain  


Actual results:
the copied disk is COW shareable:

2019-01-02 13:43:34,015+02 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (EE-ManagedThreadFactory-engine-Thread-68532) [024df732-e22f-4466-aa3d-83f7669de156] START, CopyImageVDSCommand( C
opyImageVDSCommandParameters:{storagePoolId='2288b8b4-06e6-44cc-8294-3f6ceec565f5', ignoreFailoverLimit='false', storageDomainId='a85c9e09-8ace-472e-8e10-17d086a4db84', imageGroupId='cd63a025-1f7a-42b7-8413-1f0d
e29f0385', imageId='e7acb617-6e72-4554-b26f-169fb1523808', dstImageGroupId='f450310a-7d39-40f2-a60b-29958851c101', vmId='00000000-0000-0000-0000-000000000000', dstImageId='5363a5d4-aa23-4c4f-96f1-a1f9d4a4847a', 
imageDescription='', dstStorageDomainId='4ce63bb8-71fb-4936-aff1-905c5e131f95', copyVolumeType='LeafVol', volumeFormat='COW', preallocate='Sparse', postZero='false', discard='true', force='false'}), log id: 2538
5167


3. Attach the copied disk to a VM and start it. 


Actual results:

Start VM passes validation and fails on vdsm:

2019-01-02 13:45:34,616+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ForkJoinPool-1-worker-15) [] EVENT_ID: VM_DOWN_ERROR(119), VM test is down with error. Exit message: unsup
ported configuration: shared access for disk 'vda' requires use of supported storage format.


2019-01-02 13:45:15,655+0200 ERROR (vm/a77f7c4f) [virt.vm] (vmId='a77f7c4f-f596-4980-9316-87baf31fbc6d') The vm start process failed (vm:937)
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 866, in _startUnderlyingVm
    self._run()
  File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 2842, in _run
    dom = self._connection.defineXML(self._domain.xml)
  File "/usr/lib/python2.7/site-packages/vdsm/common/libvirtconnection.py", line 131, in wrapper
    ret = f(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/vdsm/common/function.py", line 94, in wrapper
    return func(inst, *args, **kwargs)
  File "/usr/lib64/python2.7/site-packages/libvirt.py", line 3733, in defineXML
    if ret is None:raise libvirtError('virDomainDefineXML() failed', conn=self)
libvirtError: unsupported configuration: shared access for disk 'vda' requires use of supported storage format
2019-01-02 13:45:15,656+0200 INFO  (vm/a77f7c4f) [virt.vm] (vmId='a77f7c4f-f596-4980-9316-87baf31fbc6d') Changed state to Down: unsupported configuration: shared access for disk 'vda' requires use of supported s
torage format (code=1) (vm:1675)



Expected results:
Either of the following or both:
1. Copy a sharealbe thin disk from file to block domain should not pass validation on engine
2. Start VM with shareable COW disk should not pass validation on engine

Additional info:

Comment 1 Tal Nisan 2019-01-07 10:13:36 UTC
Elad, is shareable COW disks allowed on NFS?

Comment 2 Elad 2019-01-07 11:37:44 UTC
Sparse COW disk creation on a file based domain is available only via REST. Setting such disk as shared is not allowed:


2019-01-07 13:34:41,497+02 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (default task-190) [disks_create_255543c5-c6d0-4cb7] START, CreateImageVDSCommand( CreateImageVDSCommandParamete
rs:{storagePoolId='2288b8b4-06e6-44cc-8294-3f6ceec565f5', ignoreFailoverLimit='false', storageDomainId='a85c9e09-8ace-472e-8e10-17d086a4db84', imageGroupId='f51656fb-9f9c-4d3b-b9ca-201d0baf888b', imageSizeInByte
s='3221225472', volumeFormat='COW', newImageId='81105167-b50b-4d9d-936d-395af5997ddb', imageType='Sparse', newImageDescription='{"DiskAlias":"disk_random_test_0713343997","DiskDescription":""}', imageInitialSize
InBytes='0'}), log id: 1c034c49


2019-01-07 13:35:40,513+02 WARN  [org.ovirt.engine.core.bll.storage.disk.UpdateVmDiskCommand] (default task-187) [93408aee-6db9-41bb-81f0-12415535887e] Validation of action 'UpdateVmDisk' failed for user admin@internal-authz. Reasons: VAR__ACTION__UPDATE,VAR__TYPE__DISK,SHAREABLE_DISK_IS_NOT_SUPPORTED_BY_VOLUME_FORMAT

Comment 3 Martin Tessun 2019-01-14 09:09:38 UTC
Hi Jarda,

can you have a look on that one?
Could you tell us what that libvirt error means:
libvirtError: unsupported configuration: shared access for disk 'vda' requires use of supported storage format

Also: Which formats are supported for shareable disks then (and which ones are not), as we need to block it in RHV then

Comment 4 Fred Rolland 2019-01-21 09:06:12 UTC
Jarda, can you check Martin question? Thanks

Comment 5 Jaroslav Suchanek 2019-01-28 10:22:56 UTC
(In reply to Martin Tessun from comment #3)
> Hi Jarda,
> 
> can you have a look on that one?
> Could you tell us what that libvirt error means:
> libvirtError: unsupported configuration: shared access for disk 'vda'
> requires use of supported storage format
> 
> Also: Which formats are supported for shareable disks then (and which ones
> are not), as we need to block it in RHV then

<shareable/> should be available only with RAW format, I am not sure, whether the COW format is supported. Peter, can you please have a look? Thanks.

Comment 6 Jaroslav Suchanek 2019-01-28 13:05:57 UTC
Well, based on this...

 416 /**                                                                           
 417  * qemuBlockStorageSourceSupportsConcurrentAccess:                            
 418  * @src: disk storage source                                                  
 419  *                                                                            
 420  * Returns true if the given storage format supports concurrent access from two
 421  * separate processes.                                                        
 422  */                                                                           
 423 bool                                                                          
 424 qemuBlockStorageSourceSupportsConcurrentAccess(virStorageSourcePtr src)       
 425 {                                                                             
 426     /* no need to check in backing chain since only RAW storage supports this */
 427     return src->format == VIR_STORAGE_FILE_RAW;                               
 428 }                                                                             
 429                                                                               
 430

It is really only RAW.

Comment 7 Peter Krempa 2019-01-28 16:41:25 UTC
Yes, shareable disks can only be raw as qemu does not to any metadata locking.

Regarding block-copy I've recently created a patch which moves the check whether a disk is shareable of the copy target way earlier to when the operation is started to avoid having to cancel the job. It does not seem I've posted the patch som I might add the note later.

Comment 8 Tal Nisan 2019-01-29 14:13:17 UTC
So basically we need to prevent marking a disk as shareable unless they are RAW disks and prevent taking a snapshot with those disks

Comment 9 Peter Krempa 2019-02-07 13:49:55 UTC
The check is now done when starting the copy job in upstream libvirt:

commit 023d69dfc8030b8746bb03cde85623f0e2a4a275
Author: Peter Krempa <pkrempa>
Date:   Thu Jan 24 17:24:56 2019 +0100

    qemu: Move shareable disk check for block copy
    
    The writing to an image actually starts when the copy job is initiated,
    so checking this at the time of the pivot operation is too late.
    
    Move the check to qemuDomainBlockCopyCommon. Note that modern qemu would
    have prevented two writers with qcow2 so the slim possibility of a job
    started with libvirtd without this patch missing the check is not really
    worth worrying about.

Comment 10 Shir Fishbain 2019-05-29 14:51:23 UTC
Verified 

The Copy shareable thin disk from file to block domain doesn't pass validation on engine

ovirt-engine-4.3.4.1-0.1.el7.noarch
vdsm-4.30.16-3.el7ev.x86_64

Comment 11 Sandro Bonazzola 2019-06-11 06:24:04 UTC
This bugzilla is included in oVirt 4.3.4 release, published on June 11th 2019.

Since the problem described in this bug report should be
resolved in oVirt 4.3.4 release, it has been closed with a resolution of CURRENT RELEASE.

If the solution does not work for you, please open a new bug report.