Bug 1866400

After evaluating this issue, there are no plans to address it further or fix it in an upcoming release.  Therefore, it is being closed.  If plans change such that this issue will be fixed in an upcoming release, then the bug can be reopened.

commit ccffb60a9d8d855871a1b4fa79223351dbf9d606
Author: Peter Krempa <pkrempa>
Date:   Wed Mar 9 17:05:14 2022 +0100

    kbase: Introduce 'snapshots' page and describe the new 'manual' snapshot
    
    Signed-off-by: Peter Krempa <pkrempa>
    Reviewed-by: Ján Tomko <jtomko>

commit 75aaf806c9081fe5de04525426e1f12685089b58
Author: Peter Krempa <pkrempa>
Date:   Fri Mar 4 14:37:13 2022 +0100

    qemuSnapshotCreateActiveExternal: Implement manual snapshot mode
    
    In case we are snapshotting at least one 'manual' disk we will pause the
    VM and keep it paused.
    
    Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1866400
    Signed-off-by: Peter Krempa <pkrempa>
    Reviewed-by: Ján Tomko <jtomko>

commit a1465e661e7629acfee6f2b6a6268d071c413a3b
Author: Peter Krempa <pkrempa>
Date:   Fri Mar 4 15:50:19 2022 +0100

    conf: snapshot: Introduce 'manual' mode for snapshot of a disk
    
    The idea of the manual mode is to allow a synchronized snapshot in cases
    when the storage is outsourced to an unmanaged storage provider which
    requires cooperation with snapshotting.
    
    The mode will instruct the hypervisor to pause along when the other
    components are snapshotted and the 'manual' disk can be snapshotted
    along. This increases latency of the snapshot but allows them in
    otherwise impossible situations.
    
    Signed-off-by: Peter Krempa <pkrempa>
    Reviewed-by: Ján Tomko <jtomko>

v8.1.0-201-gccffb60a9d

Test with:
libvirt-8.2.0-1.el9.x86_64

POSTIVE TESTS:
===========================================
create snapshot with cmd
===========================================
1. have a running vm with two disks vda and vdb
[root@dell-per730-61 ~]# virsh domstate vm1
runnin

[root@dell-per730-61 ~]# virsh domblklist vm1
 Target   Source
--------------------------------------------------------
 vda      /var/lib/libvirt/images/jeos-27-x86_64.qcow2
 vdb      /var/lib/libvirt/images/vdb.qcow2

2. create external snapshot with vdb=manual, vda=external and dump mem snapshot
[root@dell-per730-61 ~]# virsh snapshot-create-as vm1 snap1 --diskspec vda,snapshot=external --diskspec vdb,snapshot=manual --memspec file=/tmp/vm1.mem
Domain snapshot snap1 created

3. check:
3.1 vda pointing to new image layer - jeos-27-x86_64.snap1
[root@dell-per730-61 ~]# virsh domblklist vm1 | grep vda
 vda      /var/lib/libvirt/images/jeos-27-x86_64.snap1

3.2 vdb pointing to origial image - vdb.qcow2
[root@dell-per730-61 ~]# virsh domblklist vm1 | grep vdb
 vdb      /var/lib/libvirt/images/vdb.qcow2

3.3 check snapshot xml has correct content
[root@dell-per730-61 ~]# virsh snapshot-dumpxml vm1 snap1 | egrep "[vda|vdb|vdc].*snapshot" 
<domainsnapshot>
    <disk name='vda' snapshot='external' type='file'>
    <disk name='vdb' snapshot='manual'/>
</domainsnapshot>

3.4 vm now in PAUSED status
[root@dell-per730-61 ~]# virsh domstate vm1
paused

3.5 the vm can be resumed
[root@dell-per730-61 ~]# virsh domstate vm1
running

3.6 check the snapshot metadata can be removed
[root@dell-per730-61 ~]# virsh snapshot-delete vm1 snap1 --metadata
Domain snapshot snap1 deleted

3.7 mem dumpted:
[root@dell-per730-61 ~]# ll -h /tmp/vm1.mem 
-rw-------. 1 root root 411M Apr  8 05:14 /tmp/vm1.mem



===========================================
create snapshot with xml
===========================================
1. have a running vm with 3 disks
[root@dell-per730-61 ~]# virsh domstate vm1
running

[root@dell-per730-61 ~]# virsh domblklist vm1
 Target   Source
--------------------------------------------------------
 vda      /var/lib/libvirt/images/jeos-27-x86_64.qcow2
 vdb      /var/lib/libvirt/images/vdb.qcow2
 vdc      /var/lib/libvirt/images/vdc.qcow2

2. prepare the snapshot xml as follow:
[root@dell-per730-61 ~]# cat snapshot.xml 
<domainsnapshot>
    <name>snap1</name>
    <memory file='/tmp/vm1.mem'/>
    <disks>
      <disk name='vda' snapshot='external'/>
      <disk name='vdb' snapshot='manual'/>
      <disk name='vdc' snapshot='no'/>
    </disks>
</domainsnapshot>

3. create snapshot
[root@dell-per730-61 ~]# virsh snapshot-create vm1 snapshot.xml 
Domain snapshot snap1 created from 'snapshot.xml'

4. check:
4.1 vda pointing to new image layer, vdb and vdc still pointing to original images
[root@dell-per730-61 ~]# virsh domblklist vm1
 Target   Source
--------------------------------------------------------
 vda      /var/lib/libvirt/images/jeos-27-x86_64.snap1
 vdb      /var/lib/libvirt/images/vdb.qcow2
 vdc      /var/lib/libvirt/images/vdc.qcow2

4.2 vm now in PAUSED status
[root@dell-per730-61 ~]# virsh domstate vm1
paused

4.3 check snapshot xml has correct content
[root@dell-per730-61 ~]# virsh snapshot-dumpxml vm1 snap1 | egrep "[vda|vdb|vdc].*snapshot" 
<domainsnapshot>
    <disk name='vda' snapshot='external' type='file'>
    <disk name='vdb' snapshot='manual'/>
    <disk name='vdc' snapshot='no'/>
</domainsnapshot>

4.4 mem dumped:
[root@dell-per730-61 ~]# ll -h /tmp/vm1.mem 
-rw-------. 1 root root 414M Apr  8 05:28 /tmp/vm1.mem

4.5 vm can be resumed:
[root@dell-per730-61 ~]# virsh resume vm1
Domain 'vm1' resumed

[root@dell-per730-61 ~]# virsh domstate vm1
running



===========================================
set defualt snapshot mode in disk xmls
===========================================
1. edit vm's xml and try to set snapshot=manual with vdb
[root@dell-per730-61 ~]# virsh edit vm1
...
    <disk type='file' device='disk' snapshot='manual'>
...

error: XML document failed to validate against schema: Unable to validate doc against /usr/share/libvirt/schemas/domain.rng
Extra element devices in interleave
Element domain failed to validate content

Failed. Try again? [y,n,i,f,?]: 

2. check the domaincommon.rng, there is no <value>manual</value>
[root@dell-per730-61 schemas]# cat domaincommon.rng | grep   '<define name="snapshot"' -A8
  <define name="snapshot">
    <attribute name="snapshot">
      <choice>
        <value>no</value>
        <value>internal</value>
        <value>external</value>
      </choice>
    </attribute>
  </define>

3. [root@dell-per730-61 schemas]# rpm -q --whatprovides /usr/share/libvirt/schemas/domaincommon.rng 
libvirt-libs-8.2.0-1.el9.x86_64

NEGATIVE TESTS:
1. Mixed manual and internal snapshots:
[root@dell-per730-61 ~]# virsh snapshot-create-as vm1 snap1 --diskspec vdb,snapshot=manual --diskspec vda,snapshot=internal --memspec file=/tmp/vm1.mem
error: unsupported configuration: internal and full system snapshots require all disks to be selected for snapshot

2. When vm is shutoff:
[root@dell-per730-61 ~]# virsh snapshot-create-as vm1 snap1 --diskspec vdb,snapshot=manual --diskspec vda,snapshot=external --memspec file=/tmp/vm1.mem
error: XML error: memory state cannot be saved with offline or disk-only snapshot

Hi Peter, 
Seems the domaincommon.rng was not updated within the commits, I cannot set vm's xml with <disk...snapshot='manual'> with virsh edit cmd, pls help to check.

Steps as follow:
===========================================
set defualt snapshot mode in disk xmls
===========================================
1. edit vm's xml and try to set snapshot=manual with vdb
[root@dell-per730-61 ~]# virsh edit vm1
...
    <disk type='file' device='disk' snapshot='manual'>
...

error: XML document failed to validate against schema: Unable to validate doc against /usr/share/libvirt/schemas/domain.rng
Extra element devices in interleave
Element domain failed to validate content

Failed. Try again? [y,n,i,f,?]: 

2. check the domaincommon.rng, there is no <value>manual</value>
[root@dell-per730-61 schemas]# cat domaincommon.rng | grep   '<define name="snapshot"' -A8
  <define name="snapshot">
    <attribute name="snapshot">
      <choice>
        <value>no</value>
        <value>internal</value>
        <value>external</value>
      </choice>
    </attribute>
  </define>

3. [root@dell-per730-61 schemas]# rpm -q --whatprovides /usr/share/libvirt/schemas/domaincommon.rng 
libvirt-libs-8.2.0-1.el9.x86_64

(In reply to yisun from comment #10)
> Hi Peter, 
> Seems the domaincommon.rng was not updated within the commits, I cannot set
> vm's xml with <disk...snapshot='manual'> with virsh edit cmd, pls help to
> check.

[...]

>     <disk type='file' device='disk' snapshot='manual'>
> 
> error: XML document failed to validate against schema: Unable to validate

Indeed. I've posted patches:

https://listman.redhat.com/archives/libvir-list/2022-May/231687.html

For the rng issue mentioned above, the patch is ok. Since it's a minor fix, and after editing the rng file in test env, all things are good. So set this bz to VERIFIED.

Tested with
libvirt-8.3.0-1.el9.x86_64
Same steps as comment 9, and for the rng related part, steps as follow:

[root@dell-per740xd-09 ~]# virsh domstate vm1
running

[root@dell-per740xd-09 ~]# virsh dumpxml vm1 | grep snapshot -a10
...
    <disk type='file' device='disk' snapshot='external'>
      <driver name='qemu' type='qcow2'/>
      <source file='/var/lib/libvirt/images/jeos-27-x86_64.qcow2' index='2'/>
      <backingStore/>
      <target dev='vda' bus='virtio'/>
      <alias name='virtio-disk0'/>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
    </disk>
    <disk type='file' device='disk' snapshot='manual'>
      <driver name='qemu' type='qcow2'/>
      <source file='/var/lib/libvirt/images/vdb.qcow2' index='1'/>
      <backingStore/>
      <target dev='vdb' bus='virtio'/>
      <alias name='virtio-disk1'/>
      <address type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x0'/>
    </disk>

[root@dell-per740xd-09 ~]# virsh snapshot-create-as vm1 snap1 --memspec file=/tmp/vm1.mem
Domain snapshot snap1 created

[root@dell-per740xd-09 ~]# virsh domstate vm1
paused

[root@dell-per740xd-09 ~]# virsh dumpxml vm1 | grep snapshot -a10
...
    <disk type='file' device='disk' snapshot='external'>
      <driver name='qemu' type='qcow2'/>
      <source file='/var/lib/libvirt/images/jeos-27-x86_64.snap1' index='3'/>
      <backingStore type='file' index='2'>
        <format type='qcow2'/>
        <source file='/var/lib/libvirt/images/jeos-27-x86_64.qcow2'/>
        <backingStore/>
      </backingStore>
      <target dev='vda' bus='virtio'/>
      <alias name='virtio-disk0'/>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
    </disk>
    <disk type='file' device='disk' snapshot='manual'>
      <driver name='qemu' type='qcow2'/>
      <source file='/var/lib/libvirt/images/vdb.qcow2' index='1'/>
      <backingStore/>
      <target dev='vdb' bus='virtio'/>
      <alias name='virtio-disk1'/>
      <address type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x0'/>
    </disk>
...

[root@dell-per740xd-09 ~]# ll /tmp/vm1.mem 
-rw-------. 1 root root 450097674 May 27 07:22 /tmp/vm1.mem

[root@dell-per740xd-09 ~]# virsh resume vm1
Domain 'vm1' resumed

[root@dell-per740xd-09 ~]# virsh domblklist vm1
 Target   Source
--------------------------------------------------------
 vda      /var/lib/libvirt/images/jeos-27-x86_64.snap1
 vdb      /var/lib/libvirt/images/vdb.qcow2

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Low: libvirt security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:8003