Bug 1008340

Summary: cannot restore VM with disabled usb support to snapshot with RAM
Product: Red Hat Enterprise Linux 7 Reporter: Arik <ahadas>
Component: libvirtAssignee: Peter Krempa <pkrempa>
Status: CLOSED CURRENTRELEASE QA Contact: Virtualization Bugs <virt-bugs>
Severity: unspecified Docs Contact:
Priority: urgent    
Version: 7.0CC: acathrow, ahadas, bili, cwei, didi, dyuan, michal.skrivanek, mjenner, pkrempa, shyu
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Linux   
Whiteboard:
Fixed In Version: libvirt-1.1.1-6.el7 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1011520 (view as bug list) Environment:
Last Closed: 2014-06-13 12:59:27 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1001709, 1011520    
Attachments:
Description Flags
libvirt's log none

Description Arik 2013-09-16 08:19:38 UTC
Created attachment 798155 [details]
libvirt's log

Description of problem:
Cannot restore snapshot that contains memory state when the VM is set (in ovirt) with disabled/legacy usb support. it works fine when it is set with native usb support.
The error I see in libvirt's log is:
unsupported configuration: Target controller type ide does not match source usb

We're using the restoreFlags verb with XML which is based on the XML we got back from libvirt when the snapshot was taken.
I tried to use the restore verb with the same snapshot that failed to restore using restoreFlags and it worked, but I couldn't detect which configuration in the XML we send to the restoreFlags verb cause the problem. I compared the xml configuration libvirt stores in the binary file that contains the binary memory state, with the one we pass to the restoreFlags verb and besides aliases and id attribute for the 'domain' element, it looks very similar.

I'm attaching the libvirt log where on 7:41:13 you can see the xml we pass to the restoreFlags verb and right after that the error that is raised, and the xml configuration part in the file the ram was dumped to is pasted below.

Version-Release number of selected component (if applicable):
I'm using libvirt 1.0.5.5-1.fc19 & qemu 1.4.2-9.fc19.

BTW, it doesn't reproduce with libvirt 0.10.2-18.el6_4.9 & qemu-kvm-rhev 0.12.1.2-2.355.el6_4.7.

How reproducible:
100%

Steps to Reproduce:
Actual results:
Expected results:
see bz 1001709

Additional info:
LibvirtQemudSave^B^@^@^@?^R^@^@^A^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@<domain type='kvm'>
  <name>live_snapshot7</name>
  <uuid>f7f82714-253b-4a18-a01d-e35d4305aabe</uuid>
  <memory unit='KiB'>1048576</memory>
  <currentMemory unit='KiB'>1048576</currentMemory>
  <memtune>
    <min_guarantee unit='KiB'>1048576</min_guarantee>
  </memtune>
  <vcpu placement='static'>1</vcpu>
  <cputune>
    <shares>1020</shares>
  </cputune>
  <resource>
    <partition>/machine</partition>
  </resource>
  <sysinfo type='smbios'>
    <system>
      <entry name='manufacturer'>oVirt</entry>
      <entry name='product'>oVirt Node</entry>
      <entry name='version'>19-3</entry>
      <entry name='serial'>4C4C4544-0043-5610-8054-C7C04F44354A_18:03:73:21:48:8a</entry>
      <entry name='uuid'>f7f82714-253b-4a18-a01d-e35d4305aabe</entry>
    </system>
  </sysinfo>
  <os>
    <type arch='x86_64' machine='pc-i440fx-1.4'>hvm</type>
    <smbios mode='sysinfo'/>
  </os>
  <features>
    <acpi/>
  </features>
  <cpu mode='custom' match='exact'>
    <model fallback='allow'>Conroe</model>
    <topology sockets='1' cores='1' threads='1'/>
  </cpu>
  <clock offset='variable' adjustment='0' basis='utc'>
    <timer name='rtc' tickpolicy='catchup'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <devices>
    <emulator>/usr/bin/qemu-kvm</emulator>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/rhev/data-center/mnt/multipass.eng.lab.tlv.redhat.com:_export_images_rnd_ahadas_iso__domain/5e5d7267-6dc5-493a-af99-a786eb865642/images/11111111-1111-1111-1111-111111111111/Fedora-18-x86_64-Live-Desktop.iso' startupPolicy='optional'/>
      <target dev='hdc' bus='ide'/>
      <readonly/>
      <serial></serial>
      <boot order='1'/>
      <address type='drive' controller='0' bus='1' target='0' unit='0'/>
    </disk>
    <disk type='file' device='disk' snapshot='no'>
      <driver name='qemu' type='raw' cache='none' error_policy='stop' io='threads'/>
      <source file='/var/run/vdsm/storage/28d208e8-ce96-4cb8-80f7-b6b968490858/bf9b12e9-3e55-46e7-9935-6bbe3143570d/ad3c9415-9c1c-4466-b8fe-68d01204df63'/>
      <target dev='vda' bus='virtio'/>
      <serial>bf9b12e9-3e55-46e7-9935-6bbe3143570d</serial>
      <boot order='2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </disk>
    <controller type='scsi' index='0' model='virtio-scsi'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </controller>
    <controller type='usb' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
    </controller>
    <controller type='ide' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'/>
    <interface type='bridge'>
      <mac address='00:1a:4a:16:01:b9'/>
      <source bridge=';vdsmdummy;'/>
      <model type='virtio'/>
      <filterref filter='vdsm-no-mac-spoofing'/>
      <link state='up'/>
      <boot order='3'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>
    <channel type='unix'>
      <source mode='bind' path='/var/lib/libvirt/qemu/channels/f7f82714-253b-4a18-a01d-e35d4305aabe.com.redhat.rhevm.vdsm'/>
      <target type='virtio' name='com.redhat.rhevm.vdsm'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <channel type='unix'>
      <source mode='bind' path='/var/lib/libvirt/qemu/channels/f7f82714-253b-4a18-a01d-e35d4305aabe.org.qemu.guest_agent.0'/>
      <target type='virtio' name='org.qemu.guest_agent.0'/>
      <address type='virtio-serial' controller='0' bus='0' port='2'/>
    </channel>
    <channel type='spicevmc'>
      <target type='virtio' name='com.redhat.spice.0'/>
      <address type='virtio-serial' controller='0' bus='0' port='3'/>
    </channel>
    <input type='mouse' bus='ps2'/>
    <graphics type='spice' port='5900' autoport='yes' listen='0' keymap='en-us' passwd='*****' passwdValidTo='1970-01-01T00:00:01'>
      <listen type='address' address='0'/>
    </graphics>
    <video>
      <model type='qxl' ram='65536' vram='32768' heads='1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>
    <memballoon model='virtio'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
    </memballoon>
  </devices>
  <seclabel type='none'/>
</domain>
^@QEVM^@^@^@^C^A^@^@^@^B^

Comment 1 Peter Krempa 2013-09-16 10:12:13 UTC
There are two problems that lead to this bug:
1) The XML stored in the snapshot memory file is NOT a "migratable" definition due to a mistake in the original implementation.

2) The check done when restoring was fixed to take migratable XMLs that are used to restore saved vm images. This produces the incompatible error message.

commit 07966f6a8b5ccb5bb4c716b25deb8ba2e572cc67
Author: Ján Tomko <jtomko>
Date:   Tue Jun 11 15:03:17 2013 +0200

    qemu: allow restore with non-migratable XML input
    
    Convert input XML to migratable before using it in
    qemuDomainSaveImageOpen.
    
    XML in the save image is migratable, i.e. doesn't contain implicit
    controllers. If these controllers were in a non-default order in the
    input XML, the ABI check would fail. Removing and re-adding these
    controllers fixes it.
    
    https://bugzilla.redhat.com/show_bug.cgi?id=834196

git desc --contains 07966f6a
v1.1.0-rc1~96

Comment 4 Peter Krempa 2013-09-16 12:37:27 UTC
Posted patches for upstream review:

http://www.redhat.com/archives/libvir-list/2013-September/msg00850.html

Comment 5 Peter Krempa 2013-09-17 08:16:11 UTC
Fixed upstream:

commit 1b7bfa65e36996fc3a204452d2a844ab9f4b52b3
Author: Peter Krempa <pkrempa>
Date:   Mon Sep 16 13:40:42 2013 +0200

    qemu: Use "migratable" XML definition when doing external checkpoints
    
    In the original implementation of external checkpoints I've mistakenly
    used the live definition to be stored in the save image. The normal
    approach is to use the "migratable" definition. This was discovered when
    commit 07966f6a8b5ccb5bb4c716b25deb8ba2e572cc67 changed the behavior to
    use a converted XML from the user to do the compatibility check to fix
    problem when using the regular machine saving.
    
    As the previous patch added a compatibility layer, we can now change the
    type of the XML in the image.
    
    Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1008340

commit 59898a88ce8431bd3ea249b8789edc2ef9985827
Author: Peter Krempa <pkrempa>
Date:   Mon Sep 16 13:37:34 2013 +0200

    qemu: Fix checking of ABI stability when restoring external checkpoints
    
    External checkpoints have a bug in the implementation where they use the
    normal definition instead of the "migratable" one. This causes errors
    when the snapshot is being reverted using the workaround method via
    qemuDomainRestoreFlags() with a custom XML. This issue was introduced
    when commit 07966f6a8b5ccb5bb4c716b25deb8ba2e572cc67 changed the code to
    compare "migratable" XMLs from the user as we should have used
    migratable in the image too.
    
    This patch adds a compatibility layer, so that fixing the snapshot code
    won't make existing snapshots fail to load.
    
    Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1008340

Comment 8 Shanzhi Yu 2013-11-26 10:24:42 UTC
Reproduce it with libvirt-1.0.5-1.el7.x86_64, verify it with libvirt-1.1.1-13.el7.x86_64

with libvirt-1.0.5-1.el7.x86_64:
# virsh snapshot-create-as rhel6 ss1 --memspec file=/tmp/rhel6.ss1
Domain snapshot ss1 created

# virsh snapshot-revert rhel6 ss1
error: revert requires force: Target controller type virtio-serial does not match source usb

with libvirt-1.1.1-13.el7.x86_64:
# virsh snapshot-create-as rhel6 s1 --memspec file=/tmp/rhel6.s1
Domain snapshot s1 created

# virsh snapshot-revert rhel6 s1

so change this bug to verify.

Comment 9 Ludek Smid 2014-06-13 12:59:27 UTC
This request was resolved in Red Hat Enterprise Linux 7.0.

Contact your manager or support representative in case you have further questions about the request.