Bug 1734976 - libvirt incremental backup language bindings: python
Summary: libvirt incremental backup language bindings: python
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux Advanced Virtualization
Classification: Red Hat
Component: libvirt-python
Version: 8.0
Hardware: All
OS: Unspecified
urgent
high
Target Milestone: rc
: 8.1
Assignee: Cole Robinson
QA Contact: lcheng
URL:
Whiteboard:
Depends On: 760547 1207657 1207659
Blocks: Engine_Change_Block_Tracking 1482928
TreeView+ depends on / blocked
 
Reported: 2019-07-31 17:48 UTC by Ademar Reis
Modified: 2020-06-08 07:47 UTC (History)
28 users (show)

Fixed In Version: libvirt-python-6.0.0-1.el8
Doc Type: Enhancement
Doc Text:
Clone Of: 1207659
Environment:
Last Closed: 2020-05-05 09:46:33 UTC
Type: Feature Request
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Ademar Reis 2019-07-31 17:48:50 UTC
Incremental backup introduces new APIs in libvirt and we need language bindings for them.

+++ This bug was initially created as a clone of Bug #1207659 +++

Comment 1 Xuesong Zhang 2019-08-01 03:29:26 UTC
Hi, Ademar,

The component of Perl Language binding for libvirt is called "libvirt-python", should we change the component to "libvirt-python" not libvirt here? Thx.

Comment 3 Cole Robinson 2020-01-23 20:26:31 UTC
libvirt-6.0.0-1.el8 should cover this, the python bindings are working for the incremental backup APIs. Moving to POST, not sure if that's the full process here

Comment 5 lcheng 2020-04-07 03:59:16 UTC
Test with:
python3-libvirt-6.0.0-1.module+el8.2.0+5453+31b2b136.x86_64

Because of bug 1799015, prepare a guest with following xml.
<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
…
  <qemu:capabilities>
    <qemu:add capability='incremental-backup'/>
  </qemu:capabilities>
…


Push mode test:

Senario 1: full backup
1. Prepare checkpoint and backup xml.
[root@dell-per740-08 full_backup]# cat checkpoint_push_full.xml 
<domaincheckpoint>
  <name>check_full</name>
</domaincheckpoint>

[root@dell-per740-08 full_backup]# cat backup_push_full.xml 
<domainbackup mode='push'>
  <disks>
    <disk name='vda' type='file'>
      <target file='/var/lib/libvirt/images/vda.full.backup'/>
      <driver type='qcow2'/>
    </disk>
  </disks>
</domainbackup>

2. Start full backup.
# python
Python 3.6.8 (default, Dec  5 2019, 15:45:45) 
[GCC 8.3.1 20191121 (Red Hat 8.3.1-5)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import libvirt
>>> import lxml
>>> import lxml.etree
>>> conn = libvirt.open()
>>> dom = conn.lookupByName('test')
>>> dom_xml = dom.XMLDesc(0)
>>> tree = lxml.etree.fromstring(dom_xml)
>>> tree
<Element domain at 0x7f52cf17fdc8>
>>> tree.xpath("/domain/devices/disk/target/@dev")
['vda']
>>> backup_push_full_fd = open('/root/push_mode/full_backup/backup_push_full.xml', 'r')
>>> backup_push_full_str = backup_push_full_fd.read()
>>> backup_push_full_str
"<domainbackup mode='push'>\n  <disks>\n    <disk name='vda' type='file'>\n\t    <target file='/var/lib/libvirt/images/vda.full.backup'/>\n      <driver type='qcow2'/>\n    </disk>\n  </disks>\n</domainbackup>\n"
>>> checkpoint_push_full_fd = open('/root/push_mode/full_backup/checkpoint_push_full.xml', 'r')
>>> checkpoint_push_full_str = checkpoint_push_full_fd.read()
>>> checkpoint_push_full_str
'<domaincheckpoint>\n  <name>check_full</name>\n</domaincheckpoint>\n'
>>> dom.backupBegin(backup_push_full_str, checkpoint_push_full_str, 0)
0
>>> dom.blockJobInfo('vda', 0)
{'type': 5, 'bandwidth': 0, 'cur': 2146959360, 'end': 10737418240}
>>> dom.blockJobInfo('vda', 0)
{'type': 5, 'bandwidth': 0, 'cur': 9663676416, 'end': 10737418240}
>>> dom.blockJobInfo('vda', 0)
{}
>>> checkpoint_list = dom.listAllCheckpoints(0)
>>> checkpoint_list[0]
<libvirt.virDomainCheckpoint object at 0x7f52cf1a6550>
>>> checkpoint_list[0].getName()
'check_full'

3. Check the full backup image created.
# qemu-img info -U --backing-chain /var/lib/libvirt/images/vda.full.backup 
image: /var/lib/libvirt/images/vda.full.backup
file format: qcow2
virtual size: 10 GiB (10737418240 bytes)
disk size: 10 GiB
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false


Senario 2: incremental backup
1. Do a full backup as above section.
2. Prepare two incremental backup and checkpoint xmls.
[root@dell-per740-08 incremental_backup]# cat backup_push_inc1.xml
<domainbackup mode='push'>
  <incremental>check_full</incremental>
  <disks>
    <disk name='vda' type='file'>
      <target file='/var/lib/libvirt/images/vda.inc1.backup'/>
      <driver type='qcow2'/>
    </disk>
  </disks>
</domainbackup>

[root@dell-per740-08 incremental_backup]# cat checkpoint_push_inc1.xml
<domaincheckpoint>
  <name>check_inc1</name>
</domaincheckpoint>

[root@dell-per740-08 incremental_backup]# cat backup_push_inc2.xml
<domainbackup mode='push'>
  <incremental>check_inc1</incremental>
  <disks>
    <disk name='vda' type='file'>
      <target file='/var/lib/libvirt/images/vda.inc2.backup'/>
      <driver type='qcow2'/>
    </disk>
  </disks>
</domainbackup>

[root@dell-per740-08 incremental_backup]# cat checkpoint_push_inc2.xml
<domaincheckpoint>
  <name>check_inc2</name>
</domaincheckpoint>

3. Do the first incremental backup.
>>> backup_push_inc1_fd = open('/root/push_mode/incremental_backup/backup_push_inc1.xml', 'r')
>>> backup_push_inc1_str = backup_push_inc1_fd.read()
>>> backup_push_inc1_str
"<domainbackup mode='push'>\n  <incremental>check_full</incremental>\n  <disks>\n    <disk name='vda' type='file'>\n      <target file='/var/lib/libvirt/images/vda.inc1.backup'/>\n      <driver type='qcow2'/>\n    </disk>\n  </disks>\n</domainbackup>\n"
>>> checkpoint_push_inc1_fd = open('/root/push_mode/incremental_backup/checkpoint_push_inc1.xml', 'r')
>>> checkpoint_push_inc1_str = checkpoint_push_inc1_fd.read()
>>> checkpoint_push_inc1_str
'<domaincheckpoint>\n  <name>check_inc1</name>\n</domaincheckpoint>\n'
>>> dom.backupBegin(backup_push_inc1_str, checkpoint_push_inc1_str, 0)
0
>>> dom.blockJobInfo('vda', 0)
{}
>>> checkpoint_list = dom.listAllCheckpoints(0)
>>> checkpoint_list
[<libvirt.virDomainCheckpoint object at 0x7f52cf1a6860>, <libvirt.virDomainCheckpoint object at 0x7f52cf1a67f0>]
>>> checkpoint_list[0].getName()
'check_inc1'
>>> checkpoint_list[1].getName()
'check_full'

4. Check the incremental backup image created.
# qemu-img info --backing-chain /var/lib/libvirt/images/vda.inc1.backup 
image: /var/lib/libvirt/images/vda.inc1.backup
file format: qcow2
virtual size: 10 GiB (10737418240 bytes)
disk size: 2.37 MiB
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false

5. Do the second incremental backup.
>>> backup_push_inc2_fd = open('/root/push_mode/incremental_backup/backup_push_inc2.xml', 'r')
>>> backup_push_inc2_str = backup_push_inc2_fd.read()
>>> backup_push_inc2_str
"<domainbackup mode='push'>\n  <incremental>check_inc1</incremental>\n  <disks>\n    <disk name='vda' type='file'>\n      <target file='/var/lib/libvirt/images/vda.inc2.backup'/>\n      <driver type='qcow2'/>\n    </disk>\n  </disks>\n</domainbackup>\n"
>>> checkpoint_push_inc2_fd = open('/root/push_mode/incremental_backup/checkpoint_push_inc2.xml', 'r')
>>> checkpoint_push_inc2_str = checkpoint_push_inc2_fd.read()
>>> checkpoint_push_inc2_str
'<domaincheckpoint>\n  <name>check_inc2</name>\n</domaincheckpoint>\n'
>>> dom.backupBegin(backup_push_inc2_str, checkpoint_push_inc2_str, 0)
0
>>> dom.blockJobInfo('vda', 0)
{}
>>> checkpoint_list = dom.listAllCheckpoints(0)
>>> checkpoint_list
[<libvirt.virDomainCheckpoint object at 0x7f52cf1a6898>, <libvirt.virDomainCheckpoint object at 0x7f52cf1a6550>, <libvirt.virDomainCheckpoint object at 0x7f52cf1a68d0>]
>>> checkpoint_list[0].getName()
'check_inc2'
>>> checkpoint_list[1].getName()
'check_inc1'
>>> checkpoint_list[2].getName()
'check_full'

6. Check the incremental backup image created.
# qemu-img info --backing-chain /var/lib/libvirt/images/vda.inc2.backup 
image: /var/lib/libvirt/images/vda.inc2.backup
file format: qcow2
virtual size: 10 GiB (10737418240 bytes)
disk size: 1 MiB
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false


Senario 3: differential backup
1. Do full backup and two incremental backups as above sections.
2. Prepare a differential backup xml.
[root@dell-per740-08 differential_backup]# cat backup_push_diff.xml 
<domainbackup mode='push'>
  <incremental>check_inc1</incremental>
  <disks>
    <disk name='vda' type='file'>
      <target file='/var/lib/libvirt/images/vda.diff.backup'/>
      <driver type='qcow2'/>
    </disk>
  </disks>
</domainbackup>

3. Do differential backup.
>>> backup_push_diff_fd = open('/root/push_mode/differential_backup/backup_push_diff.xml', 'r')
>>> backup_push_diff_str = backup_push_diff_fd.read()
>>> backup_push_diff_str
"<domainbackup mode='push'>\n  <incremental>check_inc1</incremental>\n  <disks>\n    <disk name='vda' type='file'>\n      <target file='/var/lib/libvirt/images/vda.diff.backup'/>\n      <driver type='qcow2'/>\n    </disk>\n  </disks>\n</domainbackup>\n"
>>> dom.backupBegin(backup_push_diff_str, None, 0)
0
>>> dom.blockJobInfo('vda', 0)
{}
>>> checkpoint_list = dom.listAllCheckpoints(0)
>>> checkpoint_list
[<libvirt.virDomainCheckpoint object at 0x7f52cf1a6a90>, <libvirt.virDomainCheckpoint object at 0x7f52cf1a69b0>, <libvirt.virDomainCheckpoint object at 0x7f52cf1a6940>]

4. Check the differential backup image created.
# qemu-img info --backing-chain /var/lib/libvirt/images/vda.diff.backup 
image: /var/lib/libvirt/images/vda.diff.backup
file format: qcow2
virtual size: 10 GiB (10737418240 bytes)
disk size: 1.07 MiB
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false

Comment 6 lcheng 2020-04-07 15:54:05 UTC
Pull mode test:

Senario 1: full backup
1. Prepare a guest.
2. Prepare full backup and checkpoint xml.
[root@dell-per740-08 pull_mode]# cat backup_pull_full.xml
<domainbackup mode='pull'>
  <server name="localhost" port="10809"/>
  <disks>
    <disk name='vda' backup='yes' type='file'>
      <scratch file='/var/lib/libvirt/images/scratch/vda.full.backup'/>
    </disk>
  </disks>
</domainbackup>

[root@dell-per740-08 pull_mode]# cat checkpoint_pull_full.xml
<domaincheckpoint>
  <name>check_full</name>
</domaincheckpoint>

3. Do full backup.
# python
Python 3.6.8 (default, Dec  5 2019, 15:45:45) 
[GCC 8.3.1 20191121 (Red Hat 8.3.1-5)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import libvirt
>>> import lxml
>>> import lxml.etree
>>> conn = libvirt.open()
>>> dom = conn.lookupByName('test')
>>> dom_xml = dom.XMLDesc(0)
>>> tree = lxml.etree.fromstring(dom_xml)
>>> path = tree.xpath("/domain/devices/disk/target/@dev")
>>> path
['vda']
>>> backup_pull_full_fd = open('/root/pull_mode/backup_pull_full.xml', 'r')
>>> backup_pull_full_str = backup_pull_full_fd.read()
>>> backup_pull_full_str
'<domainbackup mode=\'pull\'>\n  <server name="localhost" port="10809"/>\n  <disks>\n    <disk name=\'vda\' backup=\'yes\' type=\'file\'>\n\t    <scratch file=\'/var/lib/libvirt/images/scratch/vda.full.backup\'/>\n    </disk>\n  </disks>\n</domainbackup>\n'
>>> checkpoint_pull_full_fd = open('/root/pull_mode/checkpoint_pull_full.xml', 'r')
>>> checkpoint_pull_full_str = checkpoint_pull_full_fd.read()
>>> checkpoint_pull_full_str
'<domaincheckpoint>\n  <name>check_full</name>\n</domaincheckpoint>\n'
>>> dom.backupBegin(backup_pull_full_str, checkpoint_pull_full_str, 0)
0

4. Dump the full backup data into a local file.
# qemu-img convert -f raw nbd://localhost:10809/vda -O qcow2 vda.full.backup

5. Check the full backup image.
# qemu-img info vda.full.backup
image: vda.full.backup
file format: qcow2
virtual size: 10 GiB (10737418240 bytes)
disk size: 1.55 GiB
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false

6. Stop the backup process.
>>> dom.blockJobAbort(path[0], 0)
0


Senario 2: incremental backup
1. Prepare a guest with two images.
...
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/var/lib/libvirt/images/libvirt-test-api' index='2'/>
      <backingStore/>
      <target dev='vda' bus='virtio'/>
      <alias name='virtio-disk0'/>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
    </disk>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/var/lib/libvirt/images/vdb.qcow2' index='1'/>
      <backingStore/>
      <target dev='vdb' bus='virtio'/>
      <alias name='virtio-disk1'/>
      <address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
    </disk>

...
2. Prepare incremental backup and checkpoint xmls.
[root@dell-per740-08 incremental_backup]# cat backup_pull_full.xml
<domainbackup mode='pull'>
  <server name="localhost" port="10809"/>
  <disks>
    <disk name='vdb' backup='yes' type='file'>
      <scratch file='/var/lib/libvirt/images/scratch/vdb.full.backup'/>
    </disk>
  </disks>
</domainbackup>

[root@dell-per740-08 incremental_backup]# cat checkpoint_pull_full.xml
<domaincheckpoint>
  <name>check_full</name>
</domaincheckpoint>

[root@dell-per740-08 incremental_backup]# cat backup_pull_inc1.xml
<domainbackup mode='pull'>
  <incremental>check_full</incremental>
  <server name="localhost" port="10809"/>
  <disks>
    <disk name='vda' backup='no'/>
    <disk name='vdb' backup='yes' type='file'>
      <scratch file='/var/lib/libvirt/images/scratch/vdb.inc1.backup'/>
    </disk>
  </disks>
</domainbackup>

[root@dell-per740-08 incremental_backup]# cat checkpoint_pull_inc1.xml
<domaincheckpoint>
  <name>check_inc1</name>
  <description>Completion of updates after OS install</description>
  <disks>
    <disk name='vda' checkpoint='no'/>
    <disk name='vdb' checkpoint='bitmap'/>
  </disks>
</domaincheckpoint>

[root@dell-per740-08 incremental_backup]# cat backup_pull_inc2.xml
<domainbackup mode='pull'>
  <incremental>check_inc1</incremental>
  <server name="localhost" port="10809"/>
  <disks>
    <disk name='vdb' backup='yes' type='file'>
      <scratch file='/var/lib/libvirt/images/scratch/vdb.inc2.backup'/>
    </disk>
  </disks>
</domainbackup>

[root@dell-per740-08 incremental_backup]# cat checkpoint_pull_inc2.xml
<domaincheckpoint>
  <name>check_inc2</name>
  <description>Completion of updates after OS install</description>
  <disks>
    <disk name='vda' checkpoint='no'/>
    <disk name='vdb' checkpoint='bitmap'/>
  </disks>
</domaincheckpoint>

3. Do full backup. 
# python
Python 3.6.8 (default, Dec  5 2019, 15:45:45) 
[GCC 8.3.1 20191121 (Red Hat 8.3.1-5)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import libvirt
>>> import lxml
>>> import lxml.etree
>>> conn = libvirt.open()
>>> dom = conn.lookupByName('test')
>>> dom_xml = dom.XMLDesc(0)
>>> tree = lxml.etree.fromstring(dom_xml)
>>> path = tree.xpath("/domain/devices/disk/target/@dev")
>>> path
['vda', 'vdb']
>>> backup_pull_full_fd = open('/root/pull_mode/incremental_backup/backup_pull_full.xml', 'r')
>>> backup_pull_full_str = backup_pull_full_fd.read()
>>> backup_pull_full_str
'<domainbackup mode=\'pull\'>\n  <server name="localhost" port="10809"/>\n  <disks>\n    <disk name=\'vdb\' backup=\'yes\' type=\'file\'>\n      <scratch file=\'/var/lib/libvirt/images/scratch/vdb.full.backup\'/>\n    </disk>\n  </disks>\n</domainbackup>\n'
>>> checkpoint_pull_full_fd = open('/root/pull_mode/incremental_backup/checkpoint_pull_full.xml', 'r')
>>> checkpoint_pull_full_str = checkpoint_pull_full_fd.read()
>>> checkpoint_pull_full_str
'<domaincheckpoint>\n  <name>check_full</name>\n</domaincheckpoint>\n'
>>> dom.backupBegin(backup_pull_full_str, checkpoint_pull_full_str, 0)
0
>>> dom.blockJobInfo('vdb', 0)
{'type': 5, 'bandwidth': 0, 'cur': 0, 'end': 1073741824}

4. Check backup image.
# qemu-img convert -f raw nbd://localhost:10809/vdb -O qcow2 vdb.full.backup
# qemu-img info vdb.full.backup
image: vdb.full.backup
file format: qcow2
virtual size: 1 GiB (1073741824 bytes)
disk size: 196 KiB
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false

5. Stop the backup process.
>>> dom.blockJobAbort('vdb', 0)
0
>>> dom.blockJobInfo('vdb', 0)
{}

6. Do first incremental backup.
>>> backup_pull_inc1_fd = open('/root/pull_mode/incremental_backup/backup_pull_inc1.xml', 'r')
>>> backup_pull_inc1_str = backup_pull_inc1_fd.read()
>>> backup_pull_inc1_str
'<domainbackup mode=\'pull\'>\n  <incremental>check_full</incremental>\n  <server name="localhost" port="10809"/>\n  <disks>\n    <disk name=\'vda\' backup=\'no\'/>\n    <disk name=\'vdb\' backup=\'yes\' type=\'file\'>\n\t    <scratch file=\'/var/lib/libvirt/images/scratch/vdb.inc1.backup\'/>\n    </disk>\n  </disks>\n</domainbackup>\n'
>>> checkpoint_pull_inc1_fd = open('/root/pull_mode/incremental_backup/checkpoint_pull_inc1.xml', 'r')
>>> checkpoint_pull_inc1_str = checkpoint_pull_inc1_fd.read()
>>> checkpoint_pull_inc1_str
"<domaincheckpoint>\n  <name>check_inc1</name>\n  <description>Completion of updates after OS install</description>\n  <disks>\n    <disk name='vda' checkpoint='no'/>\n    <disk name='vdb' checkpoint='bitmap'/>\n  </disks>\n</domaincheckpoint>\n\n"
>>> dom.backupBegin(backup_pull_inc1_str, checkpoint_pull_inc1_str, 0)
0
>>> dom.blockJobInfo('vdb', 0)
{'type': 5, 'bandwidth': 0, 'cur': 0, 'end': 1073741824}

7. Check incremental backup image.
# qemu-img info -U vdb.inc1.backup
image: vdb.inc1.backup
file format: qcow2
virtual size: 1 GiB (1073741824 bytes)
disk size: 200 KiB
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false

8. Stop incremental backup.
>>> dom.blockJobAbort('vdb', 0)
0

9. Do second incremental backup.
>>> backup_pull_inc2_fd = open('/root/pull_mode/incremental_backup/backup_pull_inc2.xml', 'r')
>>> backup_pull_inc2_str = backup_pull_inc2_fd.read()
>>> backup_pull_inc2_str
'<domainbackup mode=\'pull\'>\n  <incremental>check_inc1</incremental>\n  <server name="localhost" port="10809"/>\n  <disks>\n    <disk name=\'vdb\' backup=\'yes\' type=\'file\'>\n      <scratch file=\'/var/lib/libvirt/images/scratch/vdb.inc2.backup\'/>\n    </disk>\n  </disks>\n</domainbackup>\n'
>>> checkpoint_pull_inc2_fd = open('/root/pull_mode/incremental_backup/checkpoint_pull_inc2.xml', 'r')
>>> checkpoint_pull_inc2_str = checkpoint_pull_inc2_fd.read()
>>> checkpoint_pull_inc2_str
"<domaincheckpoint>\n  <name>check_inc2</name>\n  <description>Completion of updates after OS install</description>\n  <disks>\n    <disk name='vda' checkpoint='no'/>\n    <disk name='vdb' checkpoint='bitmap'/>\n  </disks>\n</domaincheckpoint>\n"
>>> dom.backupBegin(backup_pull_inc2_str, checkpoint_pull_inc2_str, 0)
0
>>> checkpoint_list = dom.listAllCheckpoints(0)
>>> checkpoint_list
[<libvirt.virDomainCheckpoint object at 0x7fc0c4c03898>, <libvirt.virDomainCheckpoint object at 0x7fc0c4c03908>, <libvirt.virDomainCheckpoint object at 0x7fc0c4c03940>]
>>> dom.blockJobInfo('vdb', 0)
{'type': 5, 'bandwidth': 0, 'cur': 0, 'end': 1073741824}

10. Check incremental backup image.
# qemu-img info -U vdb.inc2.backup
image: vdb.inc2.backup
file format: qcow2
virtual size: 1 GiB (1073741824 bytes)
disk size: 452 KiB
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false

11. Stop incremental backup.
>>> dom.blockJobAbort('vdb', 0)
0


Senario 3: differential backup
1. Prepare a guest with one image.
2. Do a full backup and a incremental backup.
# python
Python 3.6.8 (default, Dec  5 2019, 15:45:45) 
[GCC 8.3.1 20191121 (Red Hat 8.3.1-5)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import libvirt
>>> import lxml
>>> import lxml.etree
>>> conn = libvirt.open()
>>> dom = conn.lookupByName('test')
>>> dom_xml = dom.XMLDesc()
>>> tree = lxml.etree.fromstring(dom_xml)
>>> path = tree.xpath("/domain/devices/disk/target/@dev")
>>> path
['vda']
>>> backup_pull_full_fd = open('/root/pull_mode/differential_backup/backup_pull_full.xml', 'r')
>>> backup_pull_full_str = backup_pull_full_fd.read()
>>> backup_pull_full_str
'<domainbackup mode=\'pull\'>\n  <server name="localhost" port="10809"/>\n  <disks>\n    <disk name=\'vda\' backup=\'yes\' type=\'file\'>\n      <scratch file=\'/var/lib/libvirt/images/scratch/vda.full.backup\'/>\n    </disk>\n  </disks>\n</domainbackup>\n'
>>> checkpoint_pull_full_fd = open('/root/pull_mode/differential_backup/checkpoint_pull_full.xml', 'r')
>>> checkpoint_pull_full_str = checkpoint_pull_full_fd.read()
>>> checkpoint_pull_full_str'<domaincheckpoint>\n  <name>check_full</name>\n</domaincheckpoint>\n'
>>> dom.backupBegin(backup_pull_full_str, checkpoint_pull_full_str, 0)
0
>>> dom.blockJobInfo('vda', 0)
{'type': 5, 'bandwidth': 0, 'cur': 1310720, 'end': 10737418240}
>>> dom.blockJobAbort('vda', 0)
0
>>> backup_pull_inc_fd = open('/root/pull_mode/differential_backup/backup_pull_inc.xml', 'r')
>>> backup_pull_inc_str = backup_pull_inc_fd.read()
>>> backup_pull_inc_str
'<domainbackup mode=\'pull\'>\n  <incremental>check_full</incremental>\n  <server name="localhost" port="10809"/>\n  <disks>\n    <disk name=\'vda\' backup=\'yes\' type=\'file\'>\n      <scratch file=\'/var/lib/libvirt/images/scratch/vda.inc1.backup\'/>\n    </disk>\n  </disks>\n</domainbackup>\n'
>>> checkpoint_pull_inc_fd = open('/root/pull_mode/differential_backup/checkpoint_pull_inc.xml', 'r')
>>> checkpoint_pull_inc_str = checkpoint_pull_inc_fd.read()
>>> checkpoint_pull_inc_str
"<domaincheckpoint>\n  <name>check_inc1</name>\n  <description>Completion of updates after OS install</description>\n  <disks>\n    <disk name='vda' checkpoint='bitmap'/>\n  </disks>\n</domaincheckpoint>\n\n"
>>> dom.backupBegin(backup_pull_inc_str, checkpoint_pull_inc_str, 0)
0
>>> checkpoint_list = dom.listAllCheckpoints(0)
>>> checkpoint_list
[<libvirt.virDomainCheckpoint object at 0x7f9ef6dd46a0>, <libvirt.virDomainCheckpoint object at 0x7f9ef6dd4710>]
>>> dom.blockJobInfo('vda', 0)
{'type': 5, 'bandwidth': 0, 'cur': 0, 'end': 10737418240}
>>> dom.blockJobAbort('vda', 0)
0

3. Do differential backup.
>>> backup_pull_diff_fd = open('/root/pull_mode/differential_backup/backup_pull_diff.xml', 'r')
>>> backup_pull_diff_str = backup_pull_diff_fd.read()
>>> backup_pull_diff_str
'<domainbackup mode=\'pull\'>\n  <incremental>check_full</incremental>\n  <server name="localhost" port="10809"/>\n  <disks>\n    <disk name=\'vda\' backup=\'yes\' type=\'file\'>\n      <scratch file=\'/tmp/vda.diff.backup\'/>\n    </disk>\n  </disks>\n</domainbackup>\n'
>>> checkpoint_pull_diff_fd = open('/root/pull_mode/differential_backup/checkpoint_pull_diff.xml', 'r')
>>> checkpoint_pull_diff_str = checkpoint_pull_diff_fd.read()
>>> checkpoint_pull_diff_str
"<domaincheckpoint>\n  <name>check_diff</name>\n  <disks>\n    <disk name='vda' checkpoint='bitmap'/>\n  </disks>\n</domaincheckpoint>\n"
>>> dom.backupBegin(backup_pull_diff_str, checkpoint_pull_diff_str, 0)
0
>>> dom.blockJobInfo('vda', 0)
{'type': 5, 'bandwidth': 0, 'cur': 131072, 'end': 10737418240}

4. Check image.
# qemu-img info -U /tmp/vda.diff.backup 
image: /tmp/vda.diff.backup
file format: qcow2
virtual size: 10 GiB (10737418240 bytes)
disk size: 776 KiB
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false

5. Stop backup.
>>> dom.blockJobAbort('vda', 0)
0

Comment 7 lcheng 2020-04-08 02:40:34 UTC
Test VIR_DOMAIN_BACKUP_BEGIN_REUSE_EXTERNAL flag.

Test steps:
1. Prepare a guest.
2. Do full backup.
# python
Python 3.6.8 (default, Dec  5 2019, 15:45:45) 
[GCC 8.3.1 20191121 (Red Hat 8.3.1-5)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import libvirt
>>> conn = libvirt.open()
>>> dom = conn.lookupByName('test')
>>> backup_push_full_fd = open('/root/push_mode/full_backup/backup_push_full.xml', 'r')
>>> backup_push_full_str = backup_push_full_fd.read()
>>> backup_push_full_str
"<domainbackup mode='push'>\n  <disks>\n    <disk name='vda' type='file'>\n\t    <target file='/var/lib/libvirt/images/vda.full.backup'/>\n      <driver type='qcow2'/>\n    </disk>\n  </disks>\n</domainbackup>\n"
>>> checkpoint_push_full_fd = open('/root/push_mode/full_backup/checkpoint_push_full.xml', 'r')
>>> checkpoint_push_full_str = checkpoint_push_full_fd.read()
>>> checkpoint_push_full_str
'<domaincheckpoint>\n  <name>check_full</name>\n</domaincheckpoint>\n'
>>> dom.backupBegin(backup_push_full_str, checkpoint_push_full_str, 0)
0
>>> dom.blockJobInfo('vda', 0)
{'type': 5, 'bandwidth': 0, 'cur': 4949278720, 'end': 10737418240}
>>> dom.blockJobInfo('vda', 0)
{}


3. Remove checkpoint and do full backup without VIR_DOMAIN_BACKUP_BEGIN_REUSE_EXTERNAL flag.
>>> dom.backupBegin(backup_push_full_str, checkpoint_push_full_str, 0)
libvirt: QEMU Driver error : invalid argument: store '/var/lib/libvirt/images/vda.full.backup' for backup of 'vda' exists
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib64/python3.6/site-packages/libvirt.py", line 704, in backupBegin
    if ret == -1: raise libvirtError ('virDomainBackupBegin() failed', dom=self)
libvirt.libvirtError: invalid argument: store '/var/lib/libvirt/images/vda.full.backup' for backup of 'vda' exists

4. Do full backup with VIR_DOMAIN_BACKUP_BEGIN_REUSE_EXTERNAL flag.
>>> dom.backupBegin(backup_push_full_str, checkpoint_push_full_str, libvirt.VIR_DOMAIN_BACKUP_BEGIN_REUSE_EXTERNAL)
0
>>> dom.blockJobInfo('vda', 0)
{'type': 5, 'bandwidth': 0, 'cur': 2448359424, 'end': 10737418240}

Comment 8 lcheng 2020-04-08 02:42:13 UTC
According to comment 5, comment 6 and comment 7, move this bug to VERIFIED.

Comment 10 errata-xmlrpc 2020-05-05 09:46:33 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:2017


Note You need to log in before you can comment on or make changes to this bug.