735553 – libvirt should check for ABI compatibility of snapshots

RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 735553 - libvirt should check for ABI compatibility of snapshots

Summary: libvirt should check for ABI compatibility of snapshots

Keywords:
Status:	CLOSED ERRATA
Alias:	None
Product:	Red Hat Enterprise Linux 6
Classification:	Red Hat
Component:	libvirt
Sub Component:
Version:	6.2
Hardware:	Unspecified
OS:	Unspecified
Priority:	unspecified
Severity:	unspecified
Target Milestone:	rc
Target Release:	---
Assignee:	Eric Blake
QA Contact:	Virtualization Bugs
Docs Contact:
URL:
Whiteboard:
Depends On:
Blocks:	638510 743047
TreeView+	depends on / blocked

Reported:	2011-09-03 13:54 UTC by Eric Blake
Modified:	2011-12-06 11:28 UTC (History)
CC List:	7 users (show)
Fixed In Version:	libvirt-0.9.4-12.el6
Doc Type:	Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed:	2011-12-06 11:28:21 UTC
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)

Links
System	ID	Private	Priority	Status	Summary	Last Updated
Red Hat Product Errata	RHBA-2011:1513	0	normal	SHIPPED_LIVE	libvirt bug fix and enhancement update	2011-12-06 01:23:30 UTC

Description Eric Blake 2011-09-03 13:54:35 UTC

Description of problem:
Libvirt is not maintaining enough information about snapshots, and risks corrupting guests that have done any hot plug or other configuration changes between the time a snapshot is created then later reverted to.

Version-Release number of selected component (if applicable):
libvirt-0.9.4-7.el6

How reproducible:
100%

Steps to Reproduce:
1. virsh start dom
2. virsh snapshot-create-as dom snap
3. virsh snapshot-dumpxml dom snap
4. virsh shutdown dom
5. virsh setmaxmem dom $(larger value)
6. virsh start dom
7. virsh snapshot-revert dom snap

Actual results:
in step 2, libvirt saves only the domain uuid in the snapshot. As a result, in step 7, libvirt has no way of knowing that the snapshot was tied to a different amount of guest RAM, tries to revert to the snapshot using a monitor command while keeping a single qemu process, and the difference in memory sizes makes things go haywire

Expected results:
libvirt should have tracked the full domain xml (visible in step 3), and used that in step 7 to either reject the revert (due to incompatible ABI) or to stop the one qemu process, and start another one with the former memory sizing.

Additional info:
Upstream patches to reject things based on incompatible ABI:
https://www.redhat.com/archives/libvir-list/2011-September/msg00137.html
Upstream plan to add a VIR_DOMAIN_SNAPSHOT_REVERT_FORCE flag to allow starting a new qemu (note that restarting qemu is not seamless, which is why it should require a flag):
https://www.redhat.com/archives/libvir-list/2011-August/msg00361.html

Comment 1 Eric Blake 2011-09-03 13:56:44 UTC

Getting this fixed, at least to the point of recognizing incompatible ABI, is a prereq to bug 638510 support for live snapshots via the snapshot_blkdev qemu monitor command.  Upstream does not yet have a solution for VIR_DOMAIN_SNAPSHOT_REVERT_FORCE implemented, so depending on time constraints, we may have to split that half into a separate BZ for later enhancement.

Comment 2 Eric Blake 2011-09-04 01:41:53 UTC

In POST:
http://post-office.corp.redhat.com/archives/rhvirt-patches/2011-September/msg00135.html

Comment 5 Nan Zhang 2011-09-13 12:18:41 UTC

Got an error when revert from created snapshot, move back to ASSIGNED.

# virsh start foo
Domain foo started

# virsh snapshot-create-as foo snap
Domain snapshot snap created

# virsh snapshot-dumpxml foo snap
<domainsnapshot>
  <name>snap</name>
  <state>running</state>
  <creationTime>1315915439</creationTime>
<domain type='kvm'>
  <name>foo</name>
  <uuid>d3782e1f-e34a-bf7c-414b-4f74d3837e96</uuid>
  <memory>524288</memory>
  <currentMemory>524288</currentMemory>
  <vcpu>1</vcpu>
  <os>
    <type arch='x86_64' machine='rhel6.1.0'>hvm</type>
    <boot dev='hd'/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <pae/>
  </features>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/libexec/qemu-kvm</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2' cache='writeback'/>
      <source file='/var/lib/libvirt/images/foo.qcow2'/>
      <target dev='vda' bus='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/var/lib/libvirt/boot/cdrom.img'/>
      <target dev='hdc' bus='ide'/>
      <readonly/>
      <address type='drive' controller='0' bus='1' unit='0'/>
    </disk>
    <controller type='ide' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='fdc' index='0'/>
    <interface type='bridge'>
      <mac address='52:54:00:85:15:ce'/>
      <source bridge='rhevm'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </interface>
    <serial type='pty'>
      <target port='0'/>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>
    <input type='tablet' bus='usb'/>
    <input type='mouse' bus='ps2'/>
    <graphics type='spice' autoport='yes'>
      <image compression='auto_glz'/>
      <jpeg compression='auto'/>
      <zlib compression='auto'/>
      <playback compression='on'/>
    </graphics>
    <video>
      <model type='cirrus' vram='9216' heads='1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>
    <memballoon model='virtio'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </memballoon>
  </devices>
</domain>
</domainsnapshot>

# virsh destroy foo
Domain foo destroyed

# virsh setmaxmem foo 1048576

# virsh start foo
Domain foo started

# virsh snapshot-revert foo snap
error: operation failed: Error -22 while loading VM state

Comment 6 Eric Blake 2011-09-13 20:30:11 UTC

Aha - the ABI compatibility checking was incomplete, and we happened to trip on one of the few places where the change is guest-visible but the code didn't flag it.  Proposed this upstream:
https://www.redhat.com/archives/libvir-list/2011-September/msg00471.html

Comment 7 Eric Blake 2011-09-14 16:22:51 UTC

Back in POST:
http://post-office.corp.redhat.com/archives/rhvirt-patches/2011-September/msg00458.html

Comment 9 Nan Zhang 2011-09-22 07:51:20 UTC

Tested with libvirt-0.9.4-12.el6.x86_64, already been fixed. Move to VERIFIED.

[root@localhost ~]# virsh start foo
Domain foo started

[root@localhost ~]# virsh snapshot-create-as foo snap
Domain snapshot snap created

[root@localhost ~]# virsh snapshot-list foo
 Name                 Creation Time             State
------------------------------------------------------------
 snap                 2010-02-08 20:58:16 -0500 running

[root@localhost ~]# virsh snapshot-dumpxml foo snap
<domainsnapshot>
  <name>snap</name>
  <state>running</state>
  <creationTime>1265680696</creationTime>
<domain type='kvm'>
  <name>foo</name>
  <uuid>5035868c-6f23-63d1-d05c-1fc5ff963134</uuid>
  <memory>524288</memory>
  <currentMemory>524288</currentMemory>
  <vcpu>1</vcpu>
  <os>
    <type arch='x86_64' machine='rhel6.2.0'>hvm</type>
    <boot dev='hd'/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <pae/>
  </features>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/libexec/qemu-kvm</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2' cache='none'/>
      <source file='/var/lib/libvirt/images/foo.img'/>
      <target dev='vda' bus='ide'/>
      <address type='drive' controller='0' bus='0' unit='0'/>
    </disk>
    <controller type='virtio-serial' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
    </controller>
    <controller type='ide' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <interface type='network'>
      <mac address='52:54:00:11:df:7f'/>
      <source network='default'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>
    <serial type='pty'>
      <target port='0'/>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>
    <channel type='pty'>
      <target type='virtio' name='org.linux-kvm.port.0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='tablet' bus='usb'/>
    <input type='mouse' bus='ps2'/>
    <graphics type='vnc' port='-1' autoport='yes'/>
    <sound model='ich6'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </sound>
    <video>
      <model type='cirrus' vram='9216' heads='1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>
    <memballoon model='virtio'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </memballoon>
  </devices>
</domain>
</domainsnapshot>

[root@localhost ~]# virsh destroy foo
Domain foo destroyed

[root@localhost ~]# virsh setmaxmem foo 1048576

[root@localhost ~]# virsh start foo
Domain foo started

[root@localhost ~]# virsh snapshot-revert foo snap
error: unsupported configuration: Target domain max memory 524288 does not match source 1048576

Comment 10 errata-xmlrpc 2011-12-06 11:28:21 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2011-1513.html

Note You need to log in before you can comment on or make changes to this bug.