Bug 1954447 - [CBT] Unable to create snapshot on a RAW disk after incremental backup
Summary: [CBT] Unable to create snapshot on a RAW disk after incremental backup
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: ovirt-engine
Classification: oVirt
Component: BLL.Storage
Version: 4.4.5.11
Hardware: Unspecified
OS: Unspecified
unspecified
high vote
Target Milestone: ovirt-4.4.7
: 4.4.7
Assignee: Eyal Shenitzky
QA Contact: Amit Sharir
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-04-28 07:46 UTC by Yury.Panchenko
Modified: 2021-08-02 07:05 UTC (History)
7 users (show)

Fixed In Version: ovirt-engine-4.4.7
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-07-06 07:27:58 UTC
oVirt Team: Storage
sbonazzo: ovirt-4.4+
asharir: testing_plan_complete+


Attachments (Terms of Use)
engine logs (3.50 MB, application/zip)
2021-04-28 07:46 UTC, Yury.Panchenko
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 1952916 1 unspecified CLOSED [CBT] [VEEAM] Download VM backup disk fails if user deleted the snapshot that contains the backup on a RAW disk with ena... 2021-10-21 07:27:14 UTC
oVirt gerrit 114561 0 master MERGED core: prevent copy bitmaps for RAW disks 2021-05-03 09:00:01 UTC

Description Yury.Panchenko 2021-04-28 07:46:12 UTC
Created attachment 1776391 [details]
engine logs

Description of problem:
image transfer fails if user deleted snapshot on a RAW disk where incremental backup was been enable

Version-Release number of selected component (if applicable):
ovirt-4.4.5
vdsm-4.40.50.10-1.el8ev.x86_64
ovirt-imageio-daemon-2.1.1-1.el8ev.x86_64
libvirt-daemon-6.6.0-13.2.module+el8.3.1+10483+85317cf0.x86_64
qemu-kvm-5.1.0-21.module+el8.3.1+10464+8ad18d1a.x86_64

How reproducible:
Create vm with QCOW and RAW disks
take a snapshot
Enable incremental backup option on both disks
Make full backup
Make incremental backup
Delete the snapshot
start incremental backup

Actual results:
backup crashed

Expected results:
RAW disk sucessfully backuped in full mode

Additional info:
2021-04-23 15:07:09,806+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.StartNbdServerVDSCommand] (default task-276) [e1d58a42-e58b-4e65-8e56-4e385b45d404] Failed in 'StartNbdServerVDS' method
2021-04-23 15:07:09,806+02 WARN  [org.ovirt.engine.core.vdsbroker.vdsbroker.StartNbdServerVDSCommand] (default task-276) [e1d58a42-e58b-4e65-8e56-4e385b45d404] Unexpected return value: Status [code=2009, message=Unsupported operation: 'reason=Cannot export bitmap from RAW volume, context={}']
2021-04-23 15:07:09,815+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-276) [e1d58a42-e58b-4e65-8e56-4e385b45d404] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM pan-rhv44rel1 command StartNbdServerVDS failed: Unsupported operation: 'reason=Cannot export bitmap from RAW volume, context={}'

after unsucessfull backup engine trys to close the ticket, but can't do that

Failed to stop image transfer session. Ticket does not exist for image '4c23c18c-7804-4850-a5d1-41391fdc4482'



Unable to create snapshot on a RAW disk after incremental backup

Description of problem:
I can’t create snapshot on a VM with RAW disk after few incremental backups

Version-Release number of selected component (if applicable):


How reproducible:
ovirt-4.4.5
vdsm-4.40.50.10-1.el8ev.x86_64
ovirt-imageio-daemon-2.1.1-1.el8ev.x86_64
libvirt-daemon-6.6.0-13.2.module+el8.3.1+10483+85317cf0.x86_64
qemu-kvm-5.1.0-21.module+el8.3.1+10464+8ad18d1a.x86_64


Steps to Reproduce:
I’m not sure how reproduce that case.

1. Create vm with RAW disk
2. Take a snapshot 
3. Enable incremental backup option on the disk
4. Do few incremental backups
5. Delete snapshot. VM should be left without any snapshots and incremental backup option is disabled.
6. Try to take a new snapshot

Actual results:
Snapshot doesn’t create

Expected results:
Snapshot successfully created

Additional info:
2021-04-27 17:20:13,159+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-65) [] Failed in 'HSMGetAllTasksStatusesVDS' method
2021-04-27 17:20:13,159+02 WARN  [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-65) [] Unexpected return value: TaskStatus [code=2009, message=value=Unsupported operation: "reason=Cannot add bitmaps from parent volume with raw format, context={'srcVolUUID': '52f18270-5070-4aa6-8305-8caac3248321'}" abortedcode=2009]
2021-04-27 17:20:13,177+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-65) [] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM pan-rhv44rel1 command HSMGetAllTasksStatusesVDS failed: value=Unsupported operation: "reason=Cannot add bitmaps from parent volume with raw format, context={'srcVolUUID': '52f18270-5070-4aa6-8305-8caac3248321'}" abortedcode=2009

Comment 1 Eyal Shenitzky 2021-04-28 08:56:04 UTC
Yury, did you try to create a live snapshot for the VM?
Does the VM run throughout that flow?

Comment 2 Yury.Panchenko 2021-04-28 09:46:08 UTC
Hello, Eyal.

> did you try to create a live snapshot for the VM?
No vm is powered off

> Does the VM run throughout that flow?
So, i started the vm and snapshot with memory sucessfully created.

Comment 3 Eyal Shenitzky 2021-04-28 09:54:17 UTC
OK, so now it is clear and we have a workaround for that issue.

Comment 4 Eyal Shenitzky 2021-04-29 07:52:20 UTC
Steps to reproduce: 
1) Create a VM with RAW format disk
2) Create a snapshot for the VM
3) Start the VM
4) Start a full backup
5) Start an incremental backup
6) Power-off the VM
7) Remove the snapshot
8) Create a new snapshot

Snapshot creation failed while it should succeed.

Comment 5 Eyal Shenitzky 2021-04-29 11:54:35 UTC
(In reply to Eyal Shenitzky from comment #4)
> Steps to reproduce: 
> 1) Create a VM with RAW format disk
> 2) Create a snapshot for the VM
> 3) Start the VM
> 4) Start a full backup
> 5) Start an incremental backup
> 6) Power-off the VM
> 7) Remove the snapshot
> 8) Create a new snapshot
> 
> Snapshot creation failed while it should succeed.

This flow also prevents from moving and copying the disk
so those should be tested also.

8.1) Create a new snapshot
8.2) Try to move the disk to a different domain
8.3) Try to copy the disk

Comment 6 Amit Sharir 2021-06-13 13:37:32 UTC
Version:
ovirt-engine-4.4.7.3-0.3.el8ev.noarch
rhv-release-4.4.7-3-001.noarch
redhat-release-8.4-0.6.el8.x86_64
redhat-release-eula-8.4-0.6.el8.x86_64
vdsm-4.40.70.3-1.el8ev.x86_64



Verification conclusions:
The expected output matched the actual output.
The total flow Eyal (steps 1-8.3) mentioned in #c5 was done with no errors/unexpected logs.
I was able to copy/move the disk as well as create additional snapshots.  


Bug verified.

Comment 7 Sandro Bonazzola 2021-07-06 07:27:58 UTC
This bugzilla is included in oVirt 4.4.7 release, published on July 6th 2021.

Since the problem described in this bug report should be resolved in oVirt 4.4.7 release, it has been closed with a resolution of CURRENT RELEASE.

If the solution does not work for you, please open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.