Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1217820 (Engine_Change_Block_Tracking)

Summary: [RFE] API for changed blocks/sectors for a disk for incremental backup usage
Product: [oVirt] ovirt-engine Reporter: Ademar Reis <areis>
Component: RFEsAssignee: Daniel Erez <derez>
Status: CLOSED CURRENTRELEASE QA Contact: Ilan Zuckerman <izuckerm>
Severity: high Docs Contact:
Priority: high    
Version: ---CC: abhay.marode, aefrat, bugs, derez, ebenahar, eblake, eshenitz, jdurgin, jsnow, jsuchane, juzhang, ketan.pachpande, knoel, lpeer, lsurette, michal.skrivanek, mkalinin, pchavva, pelauter, shan, srevivo, suchitra.herwadkar, tnisan, virt-bugs, virt-maint
Target Milestone: ovirt-4.4.1Keywords: FutureFeature, TechPreview
Target Release: 4.4.1Flags: pm-rhel: ovirt-4.4+
aefrat: testing_plan_complete+
ylavi: planning_ack+
pm-rhel: devel_ack+
pm-rhel: testing_ack+
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of: 1207659 Environment:
Last Closed: 2020-07-08 08:27:10 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Storage RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1207657, 1207659, 1518988, 1518989, 1589508, 1658343, 1734975, 1734976    
Bug Blocks: 1139877, 1547560, 1573335, 1609341, 1647195, 1691340, 1694070    

Description Ademar Reis 2015-05-01 19:10:55 UTC
More recent references:

http://comments.gmane.org/gmane.comp.emulators.qemu/303834
https://github.com/qemu/qemu/blob/master/docs/bitmaps.md

(notice the libvirt API/design is still a work in progress)


+++ This bug was initially created as a clone of Bug #1207659 +++
+++ This bug was initially created as a clone of Bug #1207657 +++

QEMU is getting incremental live backup, where a dirty block bitmap is available to third-party applications so they can read only blocks that changed since the previous backup.

Upstream references:
http://lists.gnu.org/archive/html/qemu-devel/2015-03/msg04501.html
http://lists.gnu.org/archive/html/qemu-devel/2015-03/msg05796.html

Comment 1 Michal Skrivanek 2015-05-04 08:19:24 UTC
if I'm not mistaken this is http://wiki.qemu.org/Features/Livebackup
flagging storage

Comment 2 Ademar Reis 2015-05-05 13:06:11 UTC
(In reply to Michal Skrivanek from comment #1)
> if I'm not mistaken this is http://wiki.qemu.org/Features/Livebackup

Unfortunately not. The wiki above is very outdated, from a previous try at live-backup. Updating the wiki is in our to-do list.

> flagging storage

storage indeed, thanks.

Comment 3 Ademar Reis 2015-05-11 18:25:32 UTC
(In reply to Ademar Reis from comment #2)
> (In reply to Michal Skrivanek from comment #1)
> > if I'm not mistaken this is http://wiki.qemu.org/Features/Livebackup
> 
> Unfortunately not. The wiki above is very outdated, from a previous try at
> live-backup. Updating the wiki is in our to-do list.
> 

http://wiki.qemu.org/Features/IncrementalBackup

(thanks to John Snow)

Comment 5 Yaniv Lavi 2018-08-05 15:38:19 UTC
*** Bug 1608452 has been marked as a duplicate of this bug. ***

Comment 6 Sandro Bonazzola 2019-01-28 09:44:11 UTC
This bug has not been marked as blocker for oVirt 4.3.0.
Since we are releasing it tomorrow, January 29th, this bug has been re-targeted to 4.3.1.

Comment 8 Avihai 2019-08-12 12:59:08 UTC
Hi Daniel,

In order to qa_ack this but I need to know if we(QE) have the capacity meaning I need to know:

1) The basic scenario/s (go/ no go flows) of how to test this feature.

2) Do we have a build (downstream 4.3/4.4 complete build not integration mode) to test the basic scenario on?

3) Are all API's (libvirt) available?  
I see this bug is dependant on bug 1734975 and bug 1734976 which are in NEW state.

Comment 9 Avihai 2020-04-21 05:57:41 UTC
(In reply to Avihai from comment #8)
> Hi Daniel,
> 
> In order to qa_ack this but I need to know if we(QE) have the capacity
> meaning I need to know:
> 
> 1) The basic scenario/s (go/ no go flows) of how to test this feature.
> 
> 2) Do we have a build (downstream 4.3/4.4 complete build not integration
> mode) to test the basic scenario on?
> 
> 3) Are all API's (libvirt) available?  
> I see this bug is dependant on bug 1734975 and bug 1734976 which are in NEW
> state.
I see all dependant bugs are verified so disregard this question.
Questions 1) and 2) are still relevant for 4.4.

As the responsibility for this feature changed to Eyal, I'm changing NEEDINFO accordingly.
Please also change the assignee for this bug.

Comment 10 Eyal Shenitzky 2020-04-27 08:18:27 UTC
(In reply to Avihai from comment #8)
> Hi Daniel,
> 
> In order to qa_ack this but I need to know if we(QE) have the capacity
> meaning I need to know:
> 
> 1) The basic scenario/s (go/ no go flows) of how to test this feature.

Please use - https://ovirt.org/develop/release-management/features/storage/incremental-backup.html

> 
> 2) Do we have a build (downstream 4.3/4.4 complete build not integration
> mode) to test the basic scenario on?

Yes, already available in the latest beta of 4.4

Comment 11 Sandro Bonazzola 2020-05-18 14:46:48 UTC
Moved to 4.4.1 not being marked as blocker for 4.4.0 and we are preparing to GA.

Comment 12 Ilan Zuckerman 2020-06-28 07:43:12 UTC
> Incremental backup flow:
> 
> 1. The customer has a VM with disks and wants to create a backup for it.
> 2. The customer can now create a full backup for the VM disks using a new
> backup API while the VM is running.
> 3. When the backup is done, the customer can download the backup disks using
> a new API provided by imageIO.
> 4. The customer continue to work with the VM and new data is written to the
> disk.
> 5. The customer need to perform a new backup that includes the new data.
> 6. The customer can now create an incremental backup for the VM disks using
> a new backup API while the VM is running, 
>    this backup will include only the blocks that changed on the disks.
> 8. When the backup is done, the customer can download the incremental backup
> disks using a new API provided by imageIO, 
>    the downloaded disks will include only the data that changed in the time
> between the first
>    full backup until the beginning of the incremental backup.


This case is covered by the following TC's:

TestCase27284 This is sanity test for Restoring from full backup and checking the content
TestCase27165 This is sanity test for Restoring from incremental backup and checking the content.

Both case (and others which were tested as well) are described in the 'incremental backup test plan" here:
https://polarion.engineering.redhat.com/polarion/#/project/RHEVM3/wiki/Storage_4_0/4_4_Storage_Incremental_Backup

Verified on rhv-release-4.4.1-3-001.noarch

Comment 13 Sandro Bonazzola 2020-07-08 08:27:10 UTC
This bugzilla is included in oVirt 4.4.1 release, published on July 8th 2020.

Since the problem described in this bug report should be resolved in oVirt 4.4.1 release, it has been closed with a resolution of CURRENT RELEASE.

If the solution does not work for you, please open a new bug report.