Bug 1207659

*** Bug 1518988 has been marked as a duplicate of this bug. ***

First batch of patches was merged upstream:

Note that the functionality is not enabled yet as it does not integrate with snapshots/blockjobs/migration properly yet:

1a2934d61c qemu: Add support for VIR_DOMAIN_CAPS_FEATURE_BACKUP
528191c754 conf: domaincaps: Add 'backup' feature flag
9400302477 qemu: blockjob: Implement concluded blockjob handler for backup blockjobs
e0daa03a35 qemu: driver: Allow cancellation of the backup job
5ea6cec9ef qemu: backup: Implement stats gathering while the job is running
a1521f84a5 qemu: Implement backup job APIs and qemu handling
e8ae2ddbb1 doc: Document quirk of getting block job info for a 'backup' blockjob
86d2866a90 conf: backup: Add fields for tracking stats of completed sub-jobs
1b412fb7bf tests: qemustatusxml2xml: Add test for 'pull' type backup job
2c59f0083e qemu: blockjob: Track internal data for 'backup' blockjob
9ac4b7db8d qemu: domain: Track backup job data in the status XML
e87120307a qemu: monitor: Add support for blockdev-backup via 'transaction'
08b810053a Add 'backup' block job type
3656bb0a13 qemu: domain: Introduce QEMU_ASYNC_JOB_BACKUP async job type
fed2cb487b qemu: Add infrastructure for statistics of a backup job
c15b0cd601 backup: Implement virsh support for backup
d0805c27f5 tests: genericxml2xml: Add testing of backup XML files
02f790ffbe backup: Parse and output backup XML
42adc45937 backup: Implement backup APIs for remote driver
9e408454a8 API: Add domain job operation for backups
74ca70507a backup: Introduce virDomainBackup APIs
252958ee16 backup: Document new XML for backups
c3e18c13ca virsh: Implement VIR_DOMAIN_JOB_DISK_TEMP_(USED|TOTAL) in cmdDomjobinfo
2a2f26d321 API: Introduce field for reporting temporary disk space usage of a domain job
1e320c9d83 qemu: domain: Export qemuDomainGetImageIds

Another batch of patches landed upstream.

This set implements checkpoint deletion accross snapshots, configuration of the NBD export name and bitmap name and handling of bitmaps during a virDomainBlockCopy operation.

54030892f5 qemuDomainBlockPivot: Copy bitmaps backing checkpoints for virDomainBlockCopy
9b13af73ac tests: qemublock: Add tests for qemuBlockBitmapsHandleBlockcopy
26f07f6870 qemu: block: Introduce function to calculate bitmap handling for block-copy
8e29a8b151 tests: qemublocktest: Add another synthetic test case for broken bitmaps
41c7e5c2a6 qemu: block: Add validator for bitmap chains accross backing chains
ccd4228aff qemu: blockjob: Store 'flags' for all the block job types
9ba804a1d1 qemu: blockjob: Store 'jobflags' with block job data
509ddcfde2 virHashAddOrUpdateEntry: Simplify allocation of new entry
6cf3ec4428 virhash: Fix the expectations of virHashKeyEqual implementations
d69470a18a virJSONValueNewArray: Use g_new0 to allocate and remove NULL checks from callers
3b60a0c027 util: json: Introduce virJSONValueArrayConcat
5ddfac1169 qemu: block: Extract calls of qemuBlockGetNamedNodeData into a helper function
8e94e29010 qemu: checkpoint: Track and relabel images for bitmap merging
065e548ebf tests: qemublock: Add checkpoint deletion tests for some special cases
3c1c35bada tests: qemublock: Add checkpoint deletion test for deep backing chain
30bc426071 qemu: checkpoint: Introduce support for deleting checkpoints accross snapshots
d7d97e87af tests: qemublock: Add synthetic snapshot+checkpoint test data
0cf33ab8f1 tests: qemublock: Add few more test cases for checkpoint deletion
25f7489972 tests: qemublock: Add test for checkpoint deletion bitmap merge
d9dfc1f7de qemu: checkpoint: Extract calculation of bitmap merging for checkpoint deletion
6796194a28 qemu: checkpoint: Introduce helper to find checkpoint disk definition in parents
180b3422e9 qemu: domain: Remove unused qemuDomainDiskNodeFormatLookup
f19248a139 qemu: checkpoint: tolerate missing disks on checkpoint deletion
7973f7d792 qemu: checkpoint: Use disk definition directly when creating checkpoint
f3e0a45a00 qemu: checkpoint: rename disk->chkdisk in qemuCheckpointAddActions
a303e8ea47 qemu: checkpoint: rename disk->chkdisk in qemuCheckpointDiscardBitmaps
44e1b85717 qemu: checkpoint: split out checkpoint deletion bitmaps
606dc66b09 qemu: checkpoint: Store whether deleted checkpoint is current in a variable
0c3792a155 qemu: backup: Implement support for backup disk bitmap name configuration
bce4ac55f8 qemu: backup: Implement support for backup disk export name configuration
12ccd8d4db conf: backup: Allow configuration of names exported via NBD

Too many test cases for this function, so detailed test info is recorded here for now:
https://docs.google.com/document/d/1kSLPkwwtzBI3IgFhsRLn10fqXV_FUtRvH6_iS5UPNHM/edit?folder=0AF4JRdGcd8KRUk9PVA

Result is PASS, no blocking issue

Following issues are not critical, opened as separated bzs:
Bug 1811906 - [incremental_backup] RFE: support luks encryption when do backup
Bug 1812100 - [incremental_backup] backup to a network destination is not supported
Bug 1812531 - [incremental_backup] when vm is shutoff, deleting checkpoint will produce inaccurate error message
Bug 1812827 - [incremental_backup] RFE: backup job could provide more detailed info when backup failed or provide --wating --verbose as blockcopy does
Bug 1813310 - [incremental_backup] Cannot backup to a RAW file (even for full backup). But this is mentioned in libvirt doc.
Bug 1814573 - [incremental_backup] "--size" doesn't work for checkpoint-dumpxml
Bug 1814664 - [incremental_backup] backup in push mode will actually write ZEROes to target image
Bug 1816692 - [incremental_backup] when scratch file has no space during pull-mode backup, the backup job will still run, this will produce a inconsistent backup file
Bug 1817327 - [incremental_backup] VM will be locked if we start a backup job with dirty-bitmap name conflicting
 
Automation cases are in progress:
1. Add domain backup xml support in avocado-vt:
https://github.com/avocado-framework/avocado-vt/pull/2503 
2. Add checkpoint xml to support backup function
https://github.com/avocado-framework/avocado-vt/pull/2506
3. Add virsh cmds for incremental backup
https://github.com/avocado-framework/avocado-vt/pull/2507
4.Implement test cases for incremental backup
https://github.com/autotest/tp-libvirt/pull/2667 (draft)

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:2017