Bug 1209300

Summary: migration: 7.2 (2.2.x) -> 7.1: error while loading state for instance 0x1 of device 'apic'
Product: Red Hat Enterprise Linux 7 Reporter: Luyao Huang <lhuang>
Component: qemu-kvm-rhevAssignee: Dr. David Alan Gilbert <dgilbert>
Status: CLOSED ERRATA QA Contact: Virtualization Bugs <virt-bugs>
Severity: medium Docs Contact:
Priority: medium    
Version: 7.2CC: amit.shah, dyuan, hhuang, honzhang, huding, juzhang, lhuang, mrezanin, mzhan, quintela, virt-maint, xfu
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: qemu 2.3 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-12-04 16:33:35 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On: 1209798    
Bug Blocks: 1174920, Red Hat1191307    

Description Luyao Huang 2015-04-07 01:59:58 UTC
description of problem:
failed to migrate a vm with apic when migrate from 7.2 -> 7.1

Version-Release number of selected component (if applicable):

source:
libvirt-1.2.13-1.el7.x86_64
qemu-kvm-rhev-2.2.0-8.el7.x86_64

target:
libvirt-1.2.13-1.el7.x86_64
qemu-kvm-rhev-2.1.2-23.el7_1.1.x86_64

How reproducible:
100%

Steps to Reproduce:
1.prepare a running vm in a 7.2 host:
# virsh dumpxml test3
...
  <features>
    <acpi/>
    <apic/>
    <pae/>
  </features>
...

2.migrate to 7.1 host

# virsh migrate test3 qemu+ssh://10.66.6.12/system --live
error: internal error: early end of file from monitor: possible problem:
qemu: warning: error while loading state for instance 0x1 of device 'apic'
2015-03-30T02:09:03.553100Z qemu-kvm: load of migration failed: No such file or directory


Actual results:
failed to migrate a vm when migrate from 7.2 -> 7.1

Expected results:
migrate success


Additional info:
VM full xml:

<domain type='kvm'>
  <name>test3</name>
  <uuid>7347d748-f7ce-448f-8d49-3d29c9bcac30</uuid>
  <memory unit='KiB'>1024000</memory>
  <currentMemory unit='KiB'>1024000</currentMemory>
  <vcpu placement='static' current='2'>4</vcpu>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='x86_64' machine='rhel6.5.0'>hvm</type>
    <boot dev='hd'/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <pae/>
  </features>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/libexec/qemu-kvm</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw'/>
      <source file='/var/lib/libvirt/images/r7_latest.img'/>
      <target dev='hda' bus='ide'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <controller type='virtio-serial' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </controller>
    <controller type='ide' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='scsi' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </controller>
    <controller type='ccid' index='0'/>
    <controller type='usb' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'/>
    <serial type='pty'>
      <target type='isa-serial' port='0'/>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>
    <channel type='unix'>
      <source mode='bind' path='/var/lib/libvirt/qemu/r6.agent'/>
      <target type='virtio' name='org.qemu.guest_agent.0'/>
      <address type='virtio-serial' controller='0' bus='0' port='2'/>
    </channel>
    <channel type='spicevmc'>
      <target type='virtio' name='com.redhat.spice.0'/>
      <address type='virtio-serial' controller='0' bus='0' port='9'/>
    </channel>
    <input type='tablet' bus='usb'/>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <graphics type='spice' autoport='yes' listen='0.0.0.0' keymap='en-us'>
      <listen type='address' address='0.0.0.0'/>
    </graphics>
    <video>
      <model type='qxl' ram='131072' vram='65536' vgamem='16384' heads='1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>
    <memballoon model='virtio'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </memballoon>
  </devices>
  <seclabel type='dynamic' model='selinux' relabel='yes'/>
</domain>



qemu command line:

LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin QEMU_AUDIO_DRV=spice /usr/libexec/qemu-kvm -name test3 -S -machine rhel6.5.0,accel=kvm,usb=off -m 1000 -realtime mlock=off -smp 2,maxcpus=4,sockets=4,cores=1,threads=1 -uuid 7347d748-f7ce-448f-8d49-3d29c9bcac30 -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/test3.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -boot strict=on -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device virtio-scsi-pci,id=scsi0,bus=pci.0,addr=0x6 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x5 -device usb-ccid,id=ccid0 -drive file=/var/lib/libvirt/images/r7_latest.img,if=none,id=drive-ide0-0-0,format=raw -device ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -chardev socket,id=charchannel0,path=/var/lib/libvirt/qemu/r6.agent,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=2,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 -chardev spicevmc,id=charchannel1,name=vdagent -device virtserialport,bus=virtio-serial0.0,nr=9,chardev=charchannel1,id=channel1,name=com.redhat.spice.0 -device usb-tablet,id=input0 -spice port=5901,addr=0.0.0.0,disable-ticketing,seamless-migration=on -k en-us -device qxl-vga,id=video0,ram_size=134217728,vram_size=67108864,vgamem_mb=16,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3 -msg timestamp=on

Comment 4 Dr. David Alan Gilbert 2015-05-05 10:31:58 UTC
Should be fixed in qemu-kvm-rhev-2.3.0-1 due to the fix as part of bz 1210050

Comment 7 huiqingding 2015-07-01 07:07:43 UTC
Reproduce this bug using the following version:
RHEL7.1 host:
qemu-kvm-rhev-2.1.2-23.el7_1.4.x86_64
kernel-3.10.0-229.7.2.el7.x86_64

RHEL7.2 host:
qemu-kvm-rhev-2.2.0-8.el7.x86_64
kernel-3.10.0-287.el7.x86_64

Reproduce steps:
1. boot a guest on RHEL7.2 host:
# /usr/libexec/qemu-kvm -cpu Opteron_G2 -name test3 -S -machine pc-i440fx-rhel6.5.0,accel=kvm,usb=off -m 1000 -realtime mlock=off -smp 2,maxcpus=4,sockets=4,cores=1,threads=1 -uuid 7347d748-f7ce-448f-8d49-3d29c9bcac30 -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/test3.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -boot strict=on -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device virtio-scsi-pci,id=scsi0,bus=pci.0,addr=0x6 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x5 -device usb-ccid,id=ccid0 -drive file=/mnt/rhel7.1.qcow2,if=none,id=drive-ide0-0-0,format=qcow2 -device ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -chardev socket,id=charchannel0,path=/var/lib/libvirt/qemu/r6.agent,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=2,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 -chardev spicevmc,id=charchannel1,name=vdagent -device virtserialport,bus=virtio-serial0.0,nr=9,chardev=charchannel1,id=channel1,name=com.redhat.spice.0 -device usb-tablet,id=input0 -spice port=5901,addr=0.0.0.0,disable-ticketing,seamless-migration=on -k en-us -device qxl-vga,id=video0,ram_size=134217728,vram_size=67108864,vgamem_mb=16,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3 -msg timestamp=on -monitor stdio

2. boot the guest on dst RHEL7.1 host using "-incoming tcp:0:5800"

3. do migration
(qemu) migrate -d tcp:10.66.106.84:5800

Actual results:
qemu-kvm on dst host quits with the error info:
qemu: warning: error while loading state for instance 0x1 of device 'apic'
2015-07-01T07:02:11.822939Z qemu-kvm: load of migration failed: No such file or directory

Comment 8 huiqingding 2015-07-01 07:39:16 UTC
Test this bug using the following version:
RHEL7.1 host:
qemu-kvm-rhev-2.1.2-23.el7_1.4.x86_64
kernel-3.10.0-229.7.2.el7.x86_64

RHEL7.2 host:
qemu-kvm-rhev-2.3.0-6.el7.x86_64
kernel-3.10.0-287.el7.x86_64

Use the same steps of comment #7 to do test, the results is as following:
after step3, migration can be finished normally, inside guest, run "dmesg", no error info.

I also test -machine pc-i440fx-rhel7.1.0/pc-i440fx-rhel7.0.0, the results are same.

Based on the result, I think this bug is fixed.

Comment 11 errata-xmlrpc 2015-12-04 16:33:35 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-2546.html