+++ This bug was initially created as a clone of Bug #1207657 +++ QEMU is getting incremental live backup, where a dirty block bitmap is available to third-party applications so they can read only blocks that changed since the previous backup. Upstream references: http://lists.gnu.org/archive/html/qemu-devel/2015-03/msg04501.html http://lists.gnu.org/archive/html/qemu-devel/2015-03/msg05796.html
*** Bug 1518988 has been marked as a duplicate of this bug. ***
First batch of patches was merged upstream: Note that the functionality is not enabled yet as it does not integrate with snapshots/blockjobs/migration properly yet: 1a2934d61c qemu: Add support for VIR_DOMAIN_CAPS_FEATURE_BACKUP 528191c754 conf: domaincaps: Add 'backup' feature flag 9400302477 qemu: blockjob: Implement concluded blockjob handler for backup blockjobs e0daa03a35 qemu: driver: Allow cancellation of the backup job 5ea6cec9ef qemu: backup: Implement stats gathering while the job is running a1521f84a5 qemu: Implement backup job APIs and qemu handling e8ae2ddbb1 doc: Document quirk of getting block job info for a 'backup' blockjob 86d2866a90 conf: backup: Add fields for tracking stats of completed sub-jobs 1b412fb7bf tests: qemustatusxml2xml: Add test for 'pull' type backup job 2c59f0083e qemu: blockjob: Track internal data for 'backup' blockjob 9ac4b7db8d qemu: domain: Track backup job data in the status XML e87120307a qemu: monitor: Add support for blockdev-backup via 'transaction' 08b810053a Add 'backup' block job type 3656bb0a13 qemu: domain: Introduce QEMU_ASYNC_JOB_BACKUP async job type fed2cb487b qemu: Add infrastructure for statistics of a backup job c15b0cd601 backup: Implement virsh support for backup d0805c27f5 tests: genericxml2xml: Add testing of backup XML files 02f790ffbe backup: Parse and output backup XML 42adc45937 backup: Implement backup APIs for remote driver 9e408454a8 API: Add domain job operation for backups 74ca70507a backup: Introduce virDomainBackup APIs 252958ee16 backup: Document new XML for backups c3e18c13ca virsh: Implement VIR_DOMAIN_JOB_DISK_TEMP_(USED|TOTAL) in cmdDomjobinfo 2a2f26d321 API: Introduce field for reporting temporary disk space usage of a domain job 1e320c9d83 qemu: domain: Export qemuDomainGetImageIds
Another batch of patches landed upstream. This set implements checkpoint deletion accross snapshots, configuration of the NBD export name and bitmap name and handling of bitmaps during a virDomainBlockCopy operation. 54030892f5 qemuDomainBlockPivot: Copy bitmaps backing checkpoints for virDomainBlockCopy 9b13af73ac tests: qemublock: Add tests for qemuBlockBitmapsHandleBlockcopy 26f07f6870 qemu: block: Introduce function to calculate bitmap handling for block-copy 8e29a8b151 tests: qemublocktest: Add another synthetic test case for broken bitmaps 41c7e5c2a6 qemu: block: Add validator for bitmap chains accross backing chains ccd4228aff qemu: blockjob: Store 'flags' for all the block job types 9ba804a1d1 qemu: blockjob: Store 'jobflags' with block job data 509ddcfde2 virHashAddOrUpdateEntry: Simplify allocation of new entry 6cf3ec4428 virhash: Fix the expectations of virHashKeyEqual implementations d69470a18a virJSONValueNewArray: Use g_new0 to allocate and remove NULL checks from callers 3b60a0c027 util: json: Introduce virJSONValueArrayConcat 5ddfac1169 qemu: block: Extract calls of qemuBlockGetNamedNodeData into a helper function 8e94e29010 qemu: checkpoint: Track and relabel images for bitmap merging 065e548ebf tests: qemublock: Add checkpoint deletion tests for some special cases 3c1c35bada tests: qemublock: Add checkpoint deletion test for deep backing chain 30bc426071 qemu: checkpoint: Introduce support for deleting checkpoints accross snapshots d7d97e87af tests: qemublock: Add synthetic snapshot+checkpoint test data 0cf33ab8f1 tests: qemublock: Add few more test cases for checkpoint deletion 25f7489972 tests: qemublock: Add test for checkpoint deletion bitmap merge d9dfc1f7de qemu: checkpoint: Extract calculation of bitmap merging for checkpoint deletion 6796194a28 qemu: checkpoint: Introduce helper to find checkpoint disk definition in parents 180b3422e9 qemu: domain: Remove unused qemuDomainDiskNodeFormatLookup f19248a139 qemu: checkpoint: tolerate missing disks on checkpoint deletion 7973f7d792 qemu: checkpoint: Use disk definition directly when creating checkpoint f3e0a45a00 qemu: checkpoint: rename disk->chkdisk in qemuCheckpointAddActions a303e8ea47 qemu: checkpoint: rename disk->chkdisk in qemuCheckpointDiscardBitmaps 44e1b85717 qemu: checkpoint: split out checkpoint deletion bitmaps 606dc66b09 qemu: checkpoint: Store whether deleted checkpoint is current in a variable 0c3792a155 qemu: backup: Implement support for backup disk bitmap name configuration bce4ac55f8 qemu: backup: Implement support for backup disk export name configuration 12ccd8d4db conf: backup: Allow configuration of names exported via NBD
Too many test cases for this function, so detailed test info is recorded here for now: https://docs.google.com/document/d/1kSLPkwwtzBI3IgFhsRLn10fqXV_FUtRvH6_iS5UPNHM/edit?folder=0AF4JRdGcd8KRUk9PVA Result is PASS, no blocking issue Following issues are not critical, opened as separated bzs: Bug 1811906 - [incremental_backup] RFE: support luks encryption when do backup Bug 1812100 - [incremental_backup] backup to a network destination is not supported Bug 1812531 - [incremental_backup] when vm is shutoff, deleting checkpoint will produce inaccurate error message Bug 1812827 - [incremental_backup] RFE: backup job could provide more detailed info when backup failed or provide --wating --verbose as blockcopy does Bug 1813310 - [incremental_backup] Cannot backup to a RAW file (even for full backup). But this is mentioned in libvirt doc. Bug 1814573 - [incremental_backup] "--size" doesn't work for checkpoint-dumpxml Bug 1814664 - [incremental_backup] backup in push mode will actually write ZEROes to target image Bug 1816692 - [incremental_backup] when scratch file has no space during pull-mode backup, the backup job will still run, this will produce a inconsistent backup file Bug 1817327 - [incremental_backup] VM will be locked if we start a backup job with dirty-bitmap name conflicting Automation cases are in progress: 1. Add domain backup xml support in avocado-vt: https://github.com/avocado-framework/avocado-vt/pull/2503 2. Add checkpoint xml to support backup function https://github.com/avocado-framework/avocado-vt/pull/2506 3. Add virsh cmds for incremental backup https://github.com/avocado-framework/avocado-vt/pull/2507 4.Implement test cases for incremental backup https://github.com/autotest/tp-libvirt/pull/2667 (draft)
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:2017