Bug 1950752

Summary:	[RFE][CBT] redefine only the checkpoint that the backup is taken from and not the entire chain
Product:	[oVirt] ovirt-engine	Reporter:	Eyal Shenitzky <eshenitz>
Component:	BLL.Storage	Assignee:	Eyal Shenitzky <eshenitz>
Status:	CLOSED CURRENTRELEASE	QA Contact:	Ilan Zuckerman <izuckerm>
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	4.4.6	CC:	asharir, bugs, eshames, sfishbai
Target Milestone:	ovirt-4.4.6	Keywords:	FutureFeature
Target Release:	---	Flags:	asharir: testing_plan_complete+ pm-rhel: planning_ack? pm-rhel: devel_ack? pm-rhel: testing_ack?
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2021-05-05 05:36:08 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	Storage	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1933974
Bug Blocks:

Description Eyal Shenitzky 2021-04-18 11:45:56 UTC

Description of problem:

Libvirt support the options to redefine single checkpoint and not the
entire checkpoints chain, this can be done because now all the
checkpoints in the chain are active and not just the leaf checkpoint.

So instead of redefining the entire checkpoint chain before starting a backup, only the checkpoint that the backup is taken from can be redefined in case of an incremental backup

Version-Release number of selected component (if applicable):
4.4.6

How reproducible:
100%

Steps to Reproduce:
1. Create a VM with disk enabled for incremental backup
2. Start the VM
4. Write some content to the VM
5. Create a full backup for the VM
6. Finalize the backup
7. Start an incremental backup for the VM
8. Finalize the incremental backup
9. Stop the VM
10. Start the VM
11. Add some content to the Disk
12. Start an incremental backup for the VM from the checkpoint created in step 7

Actual results:
Content added in step 11 found in the downloaded backup disk.
Both checkpoints from steps 5 and 7 redefined.


Expected results:
Content added in step 11 found in the downloaded backup disk.
Only the checkpoint from 7 redefined.


Additional info:
Available only in cluster level 4.6

Comment 1 Ilan Zuckerman 2021-04-19 06:03:22 UTC

Hi Eyal, can the checkpoints redefinition be verified from just looking at logs? Or in some other way?

A. Both checkpoints from steps 5 and 7 redefined.

B. Only the checkpoint from 7 redefined.

Comment 2 Eyal Shenitzky 2021-04-19 06:16:19 UTC

(In reply to Ilan Zuckerman from comment #1)
> Hi Eyal, can the checkpoints redefinition be verified from just looking at
> logs? Or in some other way?
> 
> A. Both checkpoints from steps 5 and 7 redefined.
> 
> B. Only the checkpoint from 7 redefined.

You can verify it by looking at the logs - 

Engine - 
2021-04-18 07:27:32,092-04 INFO  [org.ovirt.engine.core.bll.storage.backup.StartVmBackupCommand] (default task-2) [6ef3c2e8-7be7-4eea-aa92-0e057eb362f9] Successfully redefin
ed previous VM checkpoints for VM 'aa533155-dfd5-4fa1-8dc1-a771ad982652'

VDSM - 
2021-04-19 02:14:44,605-0400 INFO  (jsonrpc/7) [api.virt] START redefine_checkpoints(checkpoints=[{'id': 'f2b7dc95-0b16-4970-a673-a5f0283ef5f1', 'config': {'creation_time': 
1618745037, 'backup_id': '00000000-0000-0000-0000-000000000000', 'disks': [{'checkpoint': True, 'imageID': '66c150c9-808b-47b8-b7e1-5b8436d31049', 'volumeID': '3a49ae84-c8cb
-435f-98b8-2c59e6638652', 'domainID': '83046db3-ee83-43eb-a083-0d5c69ce2aac'}], 'from_checkpoint_id': '5a81df46-e164-4c03-a77e-753ada2358c9', 'parent_checkpoint_id': '5a81df
46-e164-4c03-a77e-753ada2358c9', 'require_consistency': False, 'to_checkpoint_id': 'f2b7dc95-0b16-4970-a673-a5f0283ef5f1'}}]) from=::ffff:10.35.0.123,37910, flow_id=be043184
-803a-45cd-beb9-dd8e4749512a, vmId=aa533155-dfd5-4fa1-8dc1-a771ad982652 (api:48)


Or by using vdsm-client on the host that runs the VM to fetch the defined checkpoints for a VM - 

$ vdsm-client VM list_checkpoints vmID=<vm-uuid>

It will show you the checkpoint that was redefined + the checkpoint that was created for the backup.

Comment 3 Sandro Bonazzola 2021-04-23 07:48:01 UTC

All referenced patches have been merged, can you please update on this bug status?

Comment 4 Ilan Zuckerman 2021-04-28 07:10:25 UTC

Verified on rhv-4.4.6-6

1. Create a VM with disk enabled for incremental backup
2. Start the VM
4. Write some content to the VM
5. Create a full backup for the VM

[root@storage-ge13-vdsm1 ~]# vdsm-client VM list_checkpoints vmID=d573b9e8-a383-4f20-9f5e-8bb9fbdfa200
[
    "1fca4252-aeb5-4796-abe1-f47b50f33818"
]


6. Finalize the backup
7. Start an incremental backup for the VM

[root@storage-ge13-vdsm1 ~]# vdsm-client VM list_checkpoints vmID=d573b9e8-a383-4f20-9f5e-8bb9fbdfa200
[
    "1fca4252-aeb5-4796-abe1-f47b50f33818",
    "a8b3c6cd-b331-494e-80de-d67e8aef2fd0"
]


8. Finalize the incremental backup
9. Stop the VM
10. Start the VM - At this point the checkpoints are not defined:

[root@storage-ge13-vdsm1 ~]# vdsm-client VM list_checkpoints vmID=d573b9e8-a383-4f20-9f5e-8bb9fbdfa200
[]


11. Add some content to the Disk
12. Start an incremental backup for the VM from the checkpoint created in step 7 - At this point, the checkpoint from step 7 is redefined:

2021-04-28 10:01:11,135+0300 INFO  (jsonrpc/3) [api.virt] FINISH redefine_checkpoints return={'result': {'checkpoint_ids': ['a8b3c6cd-b331-494e-80de-d67e8aef2fd0']}, 'status': {'code': 0, 'message': 'Done'}} from=::ffff:10.46.12.184,49494, flow_id=fd7a3dc0-4a5d-430f-a948-2be82999a98e, vmId=d573b9e8-a383-4f20-9f5e-8bb9fbdfa200 (api:54)

[root@storage-ge13-vdsm1 ~]# vdsm-client VM list_checkpoints vmID=d573b9e8-a383-4f20-9f5e-8bb9fbdfa200
[
    "a8b3c6cd-b331-494e-80de-d67e8aef2fd0",
    "6dbf4886-6f5b-42c9-bd16-47190c97e6a9"
]

Comment 5 Sandro Bonazzola 2021-05-05 05:36:08 UTC

This bugzilla is included in oVirt 4.4.6 release, published on May 4th 2021.

Since the problem described in this bug report should be resolved in oVirt 4.4.6 release, it has been closed with a resolution of CURRENT RELEASE.

If the solution does not work for you, please open a new bug report.