Bug 1320847

Summary: Migration between qemu-kvm-rhev 2.5 and older version failed
Product: Red Hat Enterprise Linux 7 Reporter: Fangge Jin <fjin>
Component: qemu-kvm-rhevAssignee: Dr. David Alan Gilbert <dgilbert>
Status: CLOSED CURRENTRELEASE QA Contact: jingzhao <jinzhao>
Severity: medium Docs Contact:
Priority: medium    
Version: 7.3CC: chayang, dgilbert, dyuan, hhan, juzhang, knoel, mzhan, qizhu, virt-maint, xfu, yafu, zpeng
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-05-13 08:14:14 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
qemu log on target
none
libvirtd log on source host
none
libvirtd log on target host none

Description Fangge Jin 2016-03-24 07:52:25 UTC
Created attachment 1139857 [details]
qemu log on target

Description of problem:
Migrate a guest using qemu-kvm-rhev 2.5 to older version, or versa, both failed.


Version-Release number of selected component:
RHEL7.2:
libvirt-1.2.17-13.el7.x86_64
qemu-kvm-rhev-2.3.0-31.el7.x86_64

RHEL7.3:
libvirt-1.3.2-1.el7.x86_64
qemu-kvm-rhev-2.5.0-3.el7.x86_64

RHEL6:
qemu-kvm-rhev-0.12.1.2-2.479.el6_7.3.x86_64
libvirt-0.10.2-54.el6_7.6.x86_64

How reproducible:
100%

Steps to Reproduce:
Migrate from qemu-kvm-rhev 2.3 to qemu-kvm-rhev 2.5

1.Start a guest on RHEL7.2 host
2.Try to migrate the guest to RHEL7.3 host
# virsh migrate rhel7.2 qemu+ssh://10.66.6.200/system --live --verbose
root.6.200's password: 
error: internal error: early end of file from monitor, possible problem:  -chardev file,id=charserial1,path=serial.log -device isa-serial,chardev=charserial1,id=serial1 -chardev spicevmc,id=charchannel0,name=vdagent -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.spice.0 -chardev socket,id=charchannel1,path=/var/lib/libvirt/qemu/channel/target/domain-rhel7.2/org.qemu.guest_agent.0,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=2,chardev=charchannel1,id=channel1,name=org.qemu.guest_agent.0 -spice port=5900,addr=0.0.0.0,disable-ticketing,seamless-migration=on -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -incoming defer -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x7 -device pvpanic,ioport=1285 -msg timestamp=on
2016-03-24T07:41:49.477295Z qemu-kvm: Missing section footer for ram
2016-03-24T07:41:49.477815Z qemu-kvm: load of migration failed: Invalid argume


Migrate from qemu-kvm-rhev 0.12 to qemu-kvm-rhev 2.5
1.Start a guest on RHEL6 host
2.Try to migrate the guest to RHEL7.3 host
# virsh migrate rhel6.6 qemu+ssh://10.66.6.200/system --live --verbose --unsafe
root.6.200's password:
error: operation failed: migration job: unexpectedly failed
3.Check qemu log on target:
2016-03-24T07:46:01.066249Z qemu-kvm: Missing section footer for block
copying E and F segments from pc.bios to pc.ram
copying C and D segments from pc.rom to pc.ram
2016-03-24T07:46:01.066965Z qemu-kvm: load of migration failed: Invalid argument


Migrate from qemu-kvm-rhev 2.5 to qemu-kvm-rhev 2.3
1.Start a guest on RHEL7.3 host
2.Try to migrate the guest to RHEL7.2 host
# virsh migrate rhel7.2 qemu+ssh://intel-q9400-4-2.englab.nay.redhat.com/system --live --verbose
root.nay.redhat.com's password: 
error: internal error: early end of file from monitor: possible problem:
2016-03-24T07:47:33.028780Z qemu-kvm: Unknown savevm section type 126
2016-03-24T07:47:33.029029Z qemu-kvm: load of migration failed: Invalid argument


Actual results:
Migration failed.

Expected results:
Migration succeed.

Comment 1 Fangge Jin 2016-03-24 07:59:42 UTC
Created attachment 1139858 [details]
libvirtd log on source host

Comment 2 Fangge Jin 2016-03-24 08:00:25 UTC
Created attachment 1139859 [details]
libvirtd log on target host

Comment 4 Dr. David Alan Gilbert 2016-03-24 12:06:44 UTC
OK, yes this is the section footer stuff that needs turning off in the older rhel machine types.

Comment 5 Dr. David Alan Gilbert 2016-03-24 18:32:10 UTC
OK, so fixing section footers and the like get me a bit further, but then I hit:

(qemu) qemu-kvm: get_pci_config_device: Bad config data: i=0x8 read: 2 device: 0 cmask: ff wmask: 0 w1cmask:0
qemu-kvm: error while loading state for instance 0x0 of device '0000:00:02.0/vga'
qemu-kvm: load of migration failed: Invalid argument

But also we have no 7.3 machine type yet so it's a bit early to be testing.

Comment 8 Dr. David Alan Gilbert 2016-06-23 16:05:47 UTC
I think this bug is fixed with the combinations of bz 1344269, 1344320, 1342015

Comment 9 huiqingding 2016-09-13 08:37:27 UTC
Do migration compatibility test for rhel7.3:
kernel-3.10.0-505.el7.x86_64
qemu-kvm-rhev-2.6.0-24.el7.x86_64

1. rhel7.2.z <-> rhel7.3 (amd host)
1-1 machine type: rhel7.2/rhel7.1/rhel7.0/rhel6.6/rhel6.5
1-2 guest: rhel7.3
1-3 result: pass

2. rhel7.1.z <-> rhel7.3 (intel host)
2-1 machine type: rhel7.1/rhel7.0/rhel6.6/rhel6.5
2-2 guest: rhel7.3
2-3 result: pass

3. rhel6.9-> rhel7.3 (amd host)
3-1 machine type: rhel6.6/rhel6.5
3-2 guest: win2008r2
3-2 result: pass

4. rhel7.3 <-> rhel7.3 (amd host)
4-1 machine type: rhel6.6/rhel6.5/rhel7.2/rhel7.1
4-2 guest: rhel7.3
4-2 result: pass

Comment 10 huiqingding 2016-09-13 08:38:04 UTC
Based on comment #9, set this bug to be verified.