Bug 1799013 - incremental-backup: RFE: Handle backup bitmaps during virDomainBlockCommit
Summary: incremental-backup: RFE: Handle backup bitmaps during virDomainBlockCommit
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux Advanced Virtualization
Classification: Red Hat
Component: libvirt
Version: 8.2
Hardware: Unspecified
OS: Unspecified
urgent
unspecified
Target Milestone: rc
: 8.0
Assignee: Peter Krempa
QA Contact: yisun
URL:
Whiteboard:
: 1799009 (view as bug list)
Depends On: 1207659 1790482
Blocks: 1799015 1861673
TreeView+ depends on / blocked
 
Reported: 2020-02-06 13:21 UTC by Peter Krempa
Modified: 2020-07-29 08:58 UTC (History)
9 users (show)

Fixed In Version: libvirt-6.0.0-12.el8
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-05-05 09:57:11 UTC
Type: Feature Request
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
copyif3.sh (1005 bytes, text/plain)
2020-03-28 10:44 UTC, yisun
no flags Details
make_chain.py (1.80 KB, text/plain)
2020-03-28 10:44 UTC, yisun
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2020:2017 0 None None None 2020-05-05 09:58:34 UTC

Description Peter Krempa 2020-02-06 13:21:19 UTC
Description of problem:
Libvirt currently doesn't handle bitmaps used to drive the incremental backup feature during virDomainBlockCommit. It's required to allow merging of external snapshots without corrupting the bitmaps.

Comment 1 Peter Krempa 2020-02-07 06:29:06 UTC
*** Bug 1799009 has been marked as a duplicate of this bug. ***

Comment 3 Nir Soffer 2020-02-11 15:17:22 UTC
Until this is fixed, RHV need to either:
- disable blockCommit functionality if incremental backup is enabled
- delete all checkpoints before blockCommit, forcing the next backup to be a full backup

Comment 4 Peter Krempa 2020-03-04 17:28:55 UTC
Patches implementing the requested functionality https://www.redhat.com/archives/libvir-list/2020-March/msg00099.html

Note that the above still depends on blockdev-reopen.

Comment 5 Yash Mankad 2020-03-13 18:53:01 UTC
Granting pm_ack, devel_ack

@Yi - could you kindly grant qa_ack ?

Comment 6 Peter Krempa 2020-03-16 21:14:55 UTC
Implemented upstream by:

a39b25c81 qemu: blockjob: Re-enable bitmaps after failed block-commit
db450a7422 qemu: blockjob: Handle bitmaps after finish of normal block-commit
8502b4b059 qemuDomainBlockPivot: Handle merging of bitmaps when pivoting an active block-commit
2b9091f089 qemuDomainBlockCommit: Handle bitmaps on start of commit
a2bf4a13ac qemuBlockJobDiskNewCommit: Propagate 'disabledBitmapsBase'
4e9bb10cf3 qemublocktest: Add tests of broken bitmap chain handling during block-commit
8f096cd880 qemublocktest: Add more tests for block-commit bitmap handling with snapshots
77b9d574b4 qemublocktest: Add tests for handling of bitmaps during block-commit
1753f60550 qemu: block: Implement helpers for dealing with bitmaps during block commit
f8389505aa qemublocktest: Fix and optimize fake image chain
41de7230ab qemu: blockjob: Store list of bitmaps disabled prior to commit
38d0dd08d6 qemu: domain: Extract parsing of 'commit' blockjob data into a function
c72e9064ce qemu: domain: Extract formatting of 'commit' blockjob data into a function
d4b5bb5d56 qemuDomainBlockCommit: Move checks depending on capabilities after liveness check
0ce8b0fbe2 qemuCheckpointDiscardBitmaps: Use correct field for checkpoint bitmap name
c6d117528c qemuCheckpointDiscardBitmaps: Reopen images for bitmap modifications
96063ce280 qemu: block: implement helpers for blockdev-reopen
9f436e067d qemu: monitor: Add handler for blockdev-reopen
ecdd929761 qemu: capabilities: Add QEMU_CAPS_BLOCKDEV_REOPEN

Comment 11 yisun 2020-03-28 10:44:04 UTC
Created attachment 1674250 [details]
copyif3.sh

Comment 12 yisun 2020-03-28 10:44:37 UTC
Created attachment 1674251 [details]
make_chain.py

Comment 13 yisun 2020-03-28 10:46:16 UTC
Test with:
libvirt-6.0.0-14.module+el8.2.0+6069+78a1cb09.x86_64
qemu-kvm-4.2.0-16.module+el8.2.0+6092+4f2391c1.x86_64

Result is: PASS

All script used during the test can be found in attachments

Comment 14 yisun 2020-03-28 10:46:53 UTC
Senario 1: Do backup in snapshot1, blockcommit snapshot1 to base layer, then do a incremental backup to check bitmaps work fine

Test steps:
1. Do snapshot
[root@dell-per740xd-11 images]# virsh snapshot-create-as vm1 s1 --disk-only
Domain snapshot s1 created

2. Do full backup
[root@dell-per740xd-11 inc_bkup]# virsh backup-begin vm1 backup_full_pull.xml checkpoint_full_pull.xml
Backup started

[root@dell-per740xd-11 inc_bkup]# qemu-img convert -f raw nbd://localhost:10809/vda -O qcow2 /var/lib/libvirt/images/vda.full.backup

3. Create data in vm
[root@localhost tmp]# dd if=/dev/urandom of=/tmp/100M_bf_inc1 bs=1M count=100
[root@localhost tmp]# md5sum 100M_bf_inc1
5a61e0a40736c6a69e38e200324161a5  100M_bf_inc1

4. Do blockcommit from s1 to base
[root@dell-per740xd-11 inc_bkup]# virsh domjobabort vm1
[root@dell-per740xd-11 inc_bkup]# virsh blockcommit vm1 vda --base /var/lib/libvirt/images/RHEL-8.2-x86_64-latest.active --active --pivot
Successfully pivoted

5. Do incremental backup
[root@dell-per740xd-11 inc_bkup]# virsh backup-begin vm1 backup_inc1_pull.xml checkpoint_inc2_pull.xml
Backup started
[root@dell-per740xd-11 images]# ./copyif3.sh nbd://127.0.0.1:10809/vda /var/lib/libvirt/images/vda.inc1.backup backup-vda

6. Check data consistency in backup file
[root@dell-per740xd-11 images]# python make_chain.py vda.full.backup vda.inc1.backup
Making Chain: 'vda.inc1.backup.copy' based on 'vda.full.backup.copy'
NOW THE CHAIN IS:
======================================================================
	vda.full.backup.copy <=== vda.inc1.backup.copy
======================================================================

[root@dell-per740xd-11 images]# virsh start checker
Domain checker started

[root@localhost tmp]# md5sum 100M_bf_inc1
5a61e0a40736c6a69e38e200324161a5  100M_bf_inc1

[root@dell-per740xd-11 images]# virsh checkpoint-list vm1
 Name         Creation Time
-----------------------------------------
 check_full   2020-03-27 09:40:02 -0400
 check_inc1   2020-03-27 09:46:32 -0400

Comment 15 yisun 2020-03-28 10:47:50 UTC
Scenario 2: A full test with blockcommit and backup
steps:
1. Create snapshot s1 for vm
[root@dell-per740xd-11 images]# virsh domblklist vm1
 Target   Source
-----------------------------------------------
 vda      /var/lib/libvirt/images/rhel8.qcow2

[root@dell-per740xd-11 images]# virsh snapshot-create-as vm1 s1 --disk-only
Domain snapshot s1 created

2. Do a full backup for vm
[root@dell-per740xd-11 inc_bkup]# virsh backup-begin vm1 backup_full_pull.xml checkpoint_full_pull.xml
Backup started

[root@dell-per740xd-11 inc_bkup]# qemu-img convert -f raw nbd://localhost:10809/vda -O qcow2 /var/lib/libvirt/images/vda.full.backup

3. Create some random data in vm, and get a md5sum value form that data
IN VM:
[root@localhost ~]# dd if=/dev/urandom of=100M_bf_inc1 count=100 bs=1M
100+0 records in
100+0 records out
104857600 bytes (105 MB, 100 MiB) copied, 0.631286 s, 166 MB/s
[root@localhost ~]# sync
[root@localhost ~]# md5sum 100M_bf_inc1
448c56f7884ec27bd1730ad326471a9a  100M_bf_inc1

4. Create snapshot s2 for vm
[root@dell-per740xd-11 inc_bkup]# virsh domjobabort vm1

[root@dell-per740xd-11 inc_bkup]# virsh snapshot-create-as vm1 s2 --disk-only
Domain snapshot s2 created

5. Do incremental backup 'inc1' for vm
[root@dell-per740xd-11 inc_bkup]# virsh backup-begin vm1 backup_inc1_pull.xml checkpoint_inc1_pull.xml
Backup started

[root@dell-per740xd-11 inc_bkup]# qemu-img create -f qcow2 /var/lib/libvirt/images/vda.inc1.backup 10G
Formatting '/var/lib/libvirt/images/vda.inc1.backup', fmt=qcow2 size=10737418240 cluster_size=65536 lazy_refcounts=off refcount_bits=16

6. Create some random data in vm, and get a md5sum value form that data
IN VM:
[root@localhost ~]# dd if=/dev/urandom of=100M_bf_inc2 count=100 bs=1M
100+0 records in
100+0 records out
104857600 bytes (105 MB, 100 MiB) copied, 0.638617 s, 164 MB/s
[root@localhost ~]# sync
[root@localhost ~]# md5sum 100M_bf_inc2
8bcb7f76035e99d838712bb602af6349  100M_bf_inc2

7. Create snapshot s3 for vm
[root@dell-per740xd-11 inc_bkup]# virsh domjobabort vm1

[root@dell-per740xd-11 inc_bkup]# virsh snapshot-create-as vm1 s3 --disk-only
Domain snapshot s3 created

8. Do blockcommit for vm1, from s2 to s1 (from middle layer to middle layer)
[root@dell-per740xd-11 inc_bkup]# virsh blockcommit vm1 vda --top /var/lib/libvirt/images/rhel8.s2 --base /var/lib/libvirt/images/rhel8.s1
Block Commit started
[root@dell-per740xd-11 inc_bkup]# virsh blockjob vm1 vda
No current block job for vda

9. Do incremental backup 'inc2' for vm
[root@dell-per740xd-11 inc_bkup]# virsh backup-begin vm1 backup_inc2_pull.xml checkpoint_inc2_pull.xml
Backup started

[root@dell-per740xd-11 inc_bkup]# qemu-img create -f qcow2 /var/lib/libvirt/images/vda.inc2.backup 10G
Formatting '/var/lib/libvirt/images/vda.inc2.backup', fmt=qcow2 size=10737418240 cluster_size=65536 lazy_refcounts=off refcount_bits=16

[root@dell-per740xd-11 inc_bkup]# ./copyif3.sh nbd://127.0.0.1:10809/vda /var/lib/libvirt/images/vda.inc2.backup backup-vda

10. Create some random data in vm, and get the md5sum value of that data
IN VM:
[root@localhost ~]# dd if=/dev/urandom of=100M_bf_inc3 count=100 bs=1M
100+0 records in
100+0 records out
104857600 bytes (105 MB, 100 MiB) copied, 0.641846 s, 163 MB/s
[root@localhost ~]# sync
[root@localhost ~]# md5sum 100M_bf_inc3
9c3baa101e07724a88597a3618d0523c  100M_bf_inc3

11. Blockcommit from s3 to base image (top layer to bottom layer)
[root@dell-per740xd-11 inc_bkup]# virsh blockcommit vm1 vda --active --pivot --verbose --wait
Block commit: [100 %]
Successfully pivoted

[root@dell-per740xd-11 inc_bkup]# virsh domblklist vm1
 Target   Source
-----------------------------------------------
 vda      /var/lib/libvirt/images/rhel8.qcow2


12. Do incremental backup 'inc3' for vm:
[root@dell-per740xd-11 inc_bkup]# virsh backup-begin vm1 backup_inc3_pull.xml checkpoint_inc3_pull.xml
Backup started

[root@dell-per740xd-11 inc_bkup]# qemu-img create -f qcow2 /var/lib/libvirt/images/vda.inc3.backup 10G
Formatting '/var/lib/libvirt/images/vda.inc3.backup', fmt=qcow2 size=10737418240 cluster_size=65536 lazy_refcounts=off refcount_bits=16

[root@dell-per740xd-11 inc_bkup]# ./copyif3.sh nbd://127.0.0.1:10809/vda /var/lib/libvirt/images/vda.inc3.backup backup-vda

[root@dell-per740xd-11 inc_bkup]# virsh domjobabort vm1

13. Do a differential backup 'diff' for vm, the baseline checkpoint is 'check_full'
[root@dell-per740xd-11 inc_bkup]# virsh backup-begin vm1 backup_diff_pull.xml checkpoint_diff_pull.xml
Backup started

[root@dell-per740xd-11 inc_bkup]# qemu-img create -f qcow2 /var/lib/libvirt/images/vda.diff.backup 10G
Formatting '/var/lib/libvirt/images/vda.diff.backup', fmt=qcow2 size=10737418240 cluster_size=65536 lazy_refcounts=off refcount_bits=16

[root@dell-per740xd-11 inc_bkup]# ./copyif3.sh nbd://127.0.0.1:10809/vda /var/lib/libvirt/images/vda.diff.backup backup-vda


[root@dell-per740xd-11 inc_bkup]# virsh domjobabort vm1

14. Check all checkpoints are correct now
[root@dell-per740xd-11 inc_bkup]# virsh checkpoint-list vm1
 Name         Creation Time
-----------------------------------------
 check_diff   2020-03-28 05:54:00 -0400
 check_full   2020-03-28 05:40:15 -0400
 check_inc1   2020-03-28 05:43:34 -0400
 check_inc2   2020-03-28 05:48:55 -0400
 check_inc3   2020-03-28 05:52:35 -0400

15. Delete all checkpoints to make sure they can be deleted
[root@dell-per740xd-11 inc_bkup]# virsh checkpoint-delete vm1 check_full
Domain checkpoint check_full deleted

[root@dell-per740xd-11 inc_bkup]# virsh checkpoint-delete vm1 check_inc1
Domain checkpoint check_inc1 deleted

[root@dell-per740xd-11 inc_bkup]# virsh checkpoint-delete vm1 check_inc2
Domain checkpoint check_inc2 deleted

[root@dell-per740xd-11 inc_bkup]# virsh checkpoint-delete vm1 check_inc3
Domain checkpoint check_inc3 deleted

[root@dell-per740xd-11 inc_bkup]# virsh checkpoint-delete vm1 check_diff
Domain checkpoint check_diff deleted

[root@dell-per740xd-11 inc_bkup]# virsh checkpoint-list vm1
 Name   Creation Time
-----------------------

[root@dell-per740xd-11 inc_bkup]# virsh qemu-monitor-command vm1 --cmd '{"execute": "x-debug-block-dirty-bitmap-sha256","arguments": {"node":"libvirt-1-format","name":"check_full"}}'
{"id":"libvirt-522","error":{"class":"GenericError","desc":"Dirty bitmap 'check_full' not found"}}

[root@dell-per740xd-11 inc_bkup]# virsh qemu-monitor-command vm1 --cmd '{"execute": "x-debug-block-dirty-bitmap-sha256","arguments": {"node":"libvirt-1-format","name":"check_inc1"}}'
{"id":"libvirt-523","error":{"class":"GenericError","desc":"Dirty bitmap 'check_inc1' not found"}}

[root@dell-per740xd-11 inc_bkup]# virsh qemu-monitor-command vm1 --cmd '{"execute": "x-debug-block-dirty-bitmap-sha256","arguments": {"node":"libvirt-1-format","name":"check_inc2"}}'
{"id":"libvirt-524","error":{"class":"GenericError","desc":"Dirty bitmap 'check_inc2' not found"}}

[root@dell-per740xd-11 inc_bkup]# virsh qemu-monitor-command vm1 --cmd '{"execute": "x-debug-block-dirty-bitmap-sha256","arguments": {"node":"libvirt-1-format","name":"check_inc3"}}'
{"id":"libvirt-525","error":{"class":"GenericError","desc":"Dirty bitmap 'check_inc3' not found"}}

[root@dell-per740xd-11 inc_bkup]# virsh qemu-monitor-command vm1 --cmd '{"execute": "x-debug-block-dirty-bitmap-sha256","arguments": {"node":"libvirt-1-format","name":"check_diff"}}'
{"id":"libvirt-526","error":{"class":"GenericError","desc":"Dirty bitmap 'check_diff' not found"}}


16. Check data consistency in backup files, make sure they all contain correct files and all files have correct md5 values. 
[root@dell-per740xd-11 images]# python make_chain.py vda.full.backup vda.inc1.backup vda.inc2.backup vda.inc3.backup
Making Chain: 'vda.inc1.backup.copy' based on 'vda.full.backup.copy'
Making Chain: 'vda.inc2.backup.copy' based on 'vda.inc1.backup.copy'
Making Chain: 'vda.inc3.backup.copy' based on 'vda.inc2.backup.copy'
NOW THE CHAIN IS:
======================================================================
	vda.full.backup.copy <=== vda.inc1.backup.copy <=== vda.inc2.backup.copy <=== vda.inc3.backup.copy
======================================================================

[root@dell-per740xd-11 images]# python make_chain.py vda.full.backup vda.diff.backup
Making Chain: 'vda.diff.backup.copy' based on 'vda.full.backup.copy'
NOW THE CHAIN IS:
======================================================================
	vda.full.backup.copy <=== vda.diff.backup.copy
======================================================================

16.1 check the diff backup
[root@dell-per740xd-11 images]# virsh dumpxml checker| awk '/<disk/,/<\/disk/'
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2' cache='none'/>
      <source file='/var/lib/libvirt/images/vda.diff.backup.copy' index='1'/>
      <backingStore type='file' index='2'>
        <format type='qcow2'/>
        <source file='/var/lib/libvirt/images/vda.full.backup.copy'/>
        <backingStore/>
      </backingStore>
      <target dev='vda' bus='virtio'/>
      <alias name='virtio-disk0'/>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
    </disk>

[root@localhost ~]# md5sum 100M_bf_inc1 100M_bf_inc2 100M_bf_inc3
448c56f7884ec27bd1730ad326471a9a  100M_bf_inc1
8bcb7f76035e99d838712bb602af6349  100M_bf_inc2
9c3baa101e07724a88597a3618d0523c  100M_bf_inc3

16.2 check the inc3 backup
[root@dell-per740xd-11 inc_bkup]# virsh dumpxml checker | awk '/<disk/,/<\/disk>/'
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2' cache='none'/>
      <source file='/var/lib/libvirt/images/vda.inc3.backup.copy'/>
      <backingStore type='file'>
        <format type='qcow2'/>
        <source file='/var/lib/libvirt/images/vda.inc2.backup.copy'/>
        <backingStore type='file'>
          <format type='qcow2'/>
          <source file='/var/lib/libvirt/images/vda.inc1.backup.copy'/>
          <backingStore type='file'>
            <format type='qcow2'/>
            <source file='/var/lib/libvirt/images/vda.full.backup.copy'/>
            <backingStore/>
          </backingStore>
        </backingStore>
      </backingStore>
      <target dev='vda' bus='virtio'/>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
    </disk>

[root@localhost ~]# md5sum 100M_bf_inc1 100M_bf_inc2 100M_bf_inc3
448c56f7884ec27bd1730ad326471a9a  100M_bf_inc1
8bcb7f76035e99d838712bb602af6349  100M_bf_inc2
9c3baa101e07724a88597a3618d0523c  100M_bf_inc3

16.3 check the inc2 backup
[root@dell-per740xd-11 inc_bkup]# virsh dumpxml checker | awk '/<disk/,/<\/disk>/'
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2' cache='none'/>
      <source file='/var/lib/libvirt/images/vda.inc2.backup.copy'/>
      <backingStore type='file'>
        <format type='qcow2'/>
        <source file='/var/lib/libvirt/images/vda.inc1.backup.copy'/>
        <backingStore type='file'>
          <format type='qcow2'/>
          <source file='/var/lib/libvirt/images/vda.full.backup.copy'/>
          <backingStore/>
        </backingStore>
      </backingStore>
      <target dev='vda' bus='virtio'/>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
    </disk>


[root@localhost ~]# md5sum 100M_bf_inc1 100M_bf_inc2 100M_bf_inc3
448c56f7884ec27bd1730ad326471a9a  100M_bf_inc1
8bcb7f76035e99d838712bb602af6349  100M_bf_inc2
md5sum: 100M_bf_inc3: No such file or directory

16.4 check the inc1 backup
[root@dell-per740xd-11 inc_bkup]# virsh dumpxml checker | awk '/<disk/,/<\/disk>/'
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2' cache='none'/>
      <source file='/var/lib/libvirt/images/vda.inc1.backup.copy'/>
      <backingStore type='file'>
        <format type='qcow2'/>
        <source file='/var/lib/libvirt/images/vda.full.backup.copy'/>
        <backingStore/>
      </backingStore>
      <target dev='vda' bus='virtio'/>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
    </disk>

[root@localhost ~]# md5sum 100M_bf_inc1 100M_bf_inc2 100M_bf_inc3
448c56f7884ec27bd1730ad326471a9a  100M_bf_inc1
md5sum: 100M_bf_inc2: No such file or directory
md5sum: 100M_bf_inc3: No such file or directory

Comment 16 yisun 2020-03-28 10:48:20 UTC
Scenario 3: Check the blockjob and backupjob are working mutual exclusively
1. Create a snapshot
[root@dell-per740xd-11 images]# virsh snapshot-create-as vm1 s1 --disk-only
Domain snapshot s1 created

2. Start a blockcommit, and make sure backup cannot be executed when blockcommit job still in progress
[root@dell-per740xd-11 images]# virsh blockcommit vm1 vda --bytes 100 --active --timeout 20

[root@dell-per740xd-11 inc_bkup]# virsh backup-begin vm1 backup_full_pull.xml checkpoint_full_pull.xml
error: internal error: disk 'vda' has a blockjob assigned

[root@dell-per740xd-11 images]# virsh blockjob vm1 vda --abort

3. Start a backup job, and make sure blockjob cannot be executed when backup job still in progress
[root@dell-per740xd-11 inc_bkup]# virsh backup-begin vm1 backup_full_pull.xml checkpoint_full_pull.xml
Backup started

[root@dell-per740xd-11 images]# virsh blockcommit vm1 vda --bytes 100 --active --timeout 20
error: Operation not supported: disk 'vda' already in active block job

Comment 17 yisun 2020-03-28 10:48:45 UTC
Scenario 4: When blockcommit job aborted/cancelled/timeout, check the bitmap is correct
1. Do snapshot
2. Do fullbackup
3. Create file
4. Do commit
5. Abort commit
6. Do incremental backup
7. Check the backup file contains correct data

1. Create snapshot s1 for vm
[root@dell-per740xd-11 inc_bkup]# virsh snapshot-create-as vm1 s1 --disk-only
Domain snapshot s1 created

2. Start full backup
[root@dell-per740xd-11 inc_bkup]# virsh backup-begin vm1 backup_full_pull.xml checkpoint_full_pull.xml
Backup started

[root@dell-per740xd-11 inc_bkup]# qemu-img convert -f raw nbd://localhost:10809/vda -O qcow2 /var/lib/libvirt/images/vda.full.backup

[root@dell-per740xd-11 inc_bkup]# virsh domjobabort vm1

3. Create some data in vm
[root@localhost ~]# dd if=/dev/urandom of=/tmp/100M_bf_inc1 bs=1M count=100
[root@localhost ~]# sync
[root@localhost ~]# md5sum /tmp/100M_bf_inc1
0a9f16cd70f989d201bb26646c63f809  /tmp/100M_bf_inc1

4. Do blockcommit job, but make it failed
[root@dell-per740xd-11 inc_bkup]# virsh blockcommit vm1 vda --bytes 10 --pivot --active
^C
Commit aborted

[root@dell-per740xd-11 inc_bkup]# virsh blockcommit vm1 vda --bytes 10 --active
Active Block Commit started

[root@dell-per740xd-11 inc_bkup]# virsh blockjob vm1 vda
Active Block Commit: [  0 %]    Bandwidth limit: 10 bytes/s (10.000 B/s)

[root@dell-per740xd-11 inc_bkup]# virsh blockjob vm1 vda --abort


[root@dell-per740xd-11 inc_bkup]# virsh blockcommit vm1 vda --bytes 100 --active --timeout 10
Commit aborted

5. Do a incremental backup
[root@dell-per740xd-11 inc_bkup]# virsh backup-begin vm1 backup_inc1_pull.xml checkpoint_inc1_pull.xml
Backup started

[root@dell-per740xd-11 inc_bkup]# qemu-img create -f qcow2 /var/lib/libvirt/images/vda.inc1.backup 10G
Formatting '/var/lib/libvirt/images/vda.inc1.backup', fmt=qcow2 size=10737418240 cluster_size=65536 lazy_refcounts=off refcount_bits=16
[root@dell-per740xd-11 inc_bkup]# ./copyif3.sh nbd://127.0.0.1:10809/vda /var/lib/libvirt/images/vda.inc1.backup backup-vda

6. Check data consistency in backup file
[root@dell-per740xd-11 images]# python make_chain.py vda.full.backup vda.inc1.backup
Making Chain: 'vda.inc1.backup.copy' based on 'vda.full.backup.copy'
NOW THE CHAIN IS:
======================================================================
	vda.full.backup.copy <=== vda.inc1.backup.copy
======================================================================

[root@dell-per740xd-11 images]# virsh dumpxml checker --inactive| awk '/<disk/,/<\/disk/'
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2' cache='none'/>
      <source file='/var/lib/libvirt/images/vda.inc1.backup.copy'/>
      <backingStore type='file'>
        <format type='qcow2'/>
        <source file='/var/lib/libvirt/images/vda.full.backup.copy'/>
        <backingStore/>
      </backingStore>
      <target dev='vda' bus='virtio'/>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
    </disk>

[root@localhost tmp]# md5sum 100M_bf_inc1
0a9f16cd70f989d201bb26646c63f809  100M_bf_inc1

Comment 18 yisun 2020-03-28 10:51:03 UTC
All backup and checkpoint xml files used in above test as follow:
[root@dell-per740xd-11 inc_bkup]# cat backup_full_pull.xml
<domainbackup mode='pull'>
  <server name="localhost" port="10809"/>
  <disks>
    <disk name='vda' backup='yes' type='file'>
	    <scratch file='/mnt/sratch.vda'/>
    </disk>
  </disks>
</domainbackup>

[root@dell-per740xd-11 inc_bkup]# cat backup_inc1_pull.xml
<domainbackup mode='pull'>
  <incremental>check_full</incremental>
  <server name="localhost" port="10809"/>
  <disks>
    <disk name='vda' backup='yes' type='file'>
        <scratch file='/tmp/vda.inc1.backup'/>
    </disk>
  </disks>
</domainbackup>

[root@dell-per740xd-11 inc_bkup]# cat backup_inc2_pull.xml
<domainbackup mode='pull'>
  <incremental>check_inc1</incremental>
  <server name="localhost" port="10809"/>
  <disks>
    <disk name='vda' backup='yes' type='file'>
        <scratch file='/tmp/vda.inc2.backup'/>
    </disk>
  </disks>
</domainbackup>

[root@dell-per740xd-11 inc_bkup]# cat backup_inc3_pull.xml
<domainbackup mode='pull'>
  <incremental>check_inc2</incremental>
  <server name="localhost" port="10809"/>
  <disks>
    <disk name='vda' backup='yes' type='file'>
        <scratch file='/tmp/vda.inc3.backup'/>
    </disk>
  </disks>
</domainbackup>

[root@dell-per740xd-11 inc_bkup]# cat backup_diff_pull.xml
<domainbackup mode='pull'>
  <incremental>check_full</incremental>
  <server name="localhost" port="10809"/>
  <disks>
    <disk name='vda' backup='yes' type='file'>
        <scratch file='/tmp/vda.diff.backup'/>
    </disk>
  </disks>
</domainbackup>

[root@dell-per740xd-11 inc_bkup]# cat checkpoint_full_pull.xml
<domaincheckpoint>
  <name>check_full</name>
  <disks>
    <disk name='vda' checkpoint='bitmap'/>
  </disks>
</domaincheckpoint>

[root@dell-per740xd-11 inc_bkup]# cat checkpoint_inc1_pull.xml
<domaincheckpoint>
  <name>check_inc1</name>
  <disks>
    <disk name='vda' checkpoint='bitmap'/>
  </disks>
</domaincheckpoint>

[root@dell-per740xd-11 inc_bkup]# cat checkpoint_inc2_pull.xml
<domaincheckpoint>
  <name>check_inc2</name>
  <disks>
    <disk name='vda' checkpoint='bitmap'/>
  </disks>
</domaincheckpoint>

[root@dell-per740xd-11 inc_bkup]# cat checkpoint_inc3_pull.xml
<domaincheckpoint>
  <name>check_inc3</name>
  <disks>
    <disk name='vda' checkpoint='bitmap'/>
  </disks>
</domaincheckpoint>

[root@dell-per740xd-11 inc_bkup]# cat checkpoint_diff_pull.xml
<domaincheckpoint>
  <name>check_diff</name>
  <disks>
    <disk name='vda' checkpoint='bitmap'/>
  </disks>
</domaincheckpoint>

Comment 20 errata-xmlrpc 2020-05-05 09:57:11 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:2017


Note You need to log in before you can comment on or make changes to this bug.