Bug 997702 - Migration from RHEL6.5 host to RHEL7.0 host is failed due to iPXE ROM size mismatch
Migration from RHEL6.5 host to RHEL7.0 host is failed due to iPXE ROM size mi...
Status: CLOSED CURRENTRELEASE
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: qemu-kvm (Show other bugs)
7.0
x86_64 Linux
urgent Severity high
: rc
: ---
Assigned To: Eduardo Habkost
Virtualization Bugs
: Reopened
Depends On:
Blocks: 889670 915399
  Show dependency treegraph
 
Reported: 2013-08-15 23:24 EDT by huiqingding
Modified: 2014-06-17 23:34 EDT (History)
15 users (show)

See Also:
Fixed In Version: -kvm-1.5.3-14.el7
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2014-06-13 07:59:34 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description huiqingding 2013-08-15 23:24:02 EDT
Description of problem:
Migration is failed, src host is RHEL6.5 and dst host is RHEL7.0.

Version-Release number of selected component (if applicable):
src host:
The kernel is kernel-2.6.32-412.el6.x86_64
The qemu-kvm is qemu-kvm-0.12.1.2-2.393.el6.x86_64
dst host:
The kernel is kernel-3.10.0-5.el7.x86_64
The qemu-kvm is qemu-kvm-1.5.2-3.el7.x86_64

How reproducible:
100%

Steps to Reproduce:
1.Boot RHEL6.5 guest on src RHEL6.5 host
# /usr/libexec/qemu-kvm -M rhel6.5.0 -cpu SandyBridge -enable-kvm -m 2G  -smp 4 -name rhel6.5 -uuid 6afa5f93-2d4f-420f-81c6-e5fdddbd1c83 -drive file=/home/dhq/RHEL-Server-6.5-64.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,serial=40c061dd-5d60-4fc5-865f-55db700407f0,cache=none,werror=stop,rerror=stop,aio=threads -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0 -netdev tap,id=hostnet0,vhost=on -device virtio-net-pci,netdev=hostnet0,id=net0,mac=00:1a:4a:42:0a:0a,bus=pci.0,addr=0x3 -vnc :1  -monitor stdio

2.Boot RHEL6.5 guest on dst RHEL7.0 host with listening mode
# /usr/libexec/qemu-kvm -M rhel6.5.0 -cpu SandyBridge -enable-kvm -m 2G  -smp 4 -name rhel6.5 -uuid 6afa5f93-2d4f-420f-81c6-e5fdddbd1c83 -drive file=/home/dhq/RHEL-Server-6.5-64.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,serial=40c061dd-5d60-4fc5-865f-55db700407f0,cache=none,werror=stop,rerror=stop,aio=threads -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0 -netdev tap,id=hostnet0,vhost=on -device virtio-net-pci,netdev=hostnet0,id=net0,mac=00:1a:4a:42:0a:0a,bus=pci.0,addr=0x3 -vnc :1  -monitor stdio -incoming tcp:0:5800

3. Do migration from src host to dst host
(qemu) migrate -d tcp:10.66.5.221:5800

Actual results:
Migration is failed. The qemu-kvm on dst host quits with the error information:
(qemu) qemu: warning: error while loading state for instance 0x0 of device 'ram'
load of migration failed

Expected results:
Migration shoulde finish successfully.

Additional info:
Comment 2 huiqingding 2013-08-16 02:00:38 EDT
I test "-M rhel6.4.0" and "-M rhel6.3.0", migrate rhel6.5 guest. The result is also failed.
Comment 6 Paolo Bonzini 2013-09-05 07:52:24 EDT
Please try reproducing with "virsh save" and place somewhere the resulting file.  I identified one possible issue and opened bug 1004743.
Comment 7 juzhang 2013-09-05 21:27:25 EDT
Can you do some test according to comment?
Comment 8 huiqingding 2013-09-06 05:57:21 EDT
Hi, Paolo,
(In reply to Paolo Bonzini from comment #6)
> Please try reproducing with "virsh save" and place somewhere the resulting
> file.  I identified one possible issue and opened bug 1004743.

I make the following test:
1. start a rhel6.5 guest on the rhel6.5 host
2. run "virsh save" on the rhel6.5 host to save vm state to a file test.save
3. run "virsh restore test.save" on the rhel7.0 host, the result is that the guest cannot be restored and the log info in "/var/log/libvirt/qemu/test.log" is as following:

2013-09-06 09:34:32.069+0000: starting up
LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin QEMU_AUDIO_DRV=none /usr/libexec/qemu-kvm -name test -S -machine rhel6.5.0,accel=kvm,usb=off -m 2024 -realtime mlock=off -smp 2,sockets=2,cores=1,threads=1 -uuid 2bc0773b-1809-2fcd-fd51-06aee07b1b16 -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/test.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/var/lib/libvirt/images/RHEL-Server-6.5-64.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,cache=writethrough,aio=native -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -device usb-tablet,id=input0 -vnc 127.0.0.1:0 -vga cirrus -incoming fd:25 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5
Unknown savevm section type 112
load of migration failed
2013-09-06 09:34:32.705+0000: shutting down
Comment 9 Paolo Bonzini 2013-09-06 11:19:04 EDT
Great, this is exactly bug 1004743.

*** This bug has been marked as a duplicate of bug 1004743 ***
Comment 11 Paolo Bonzini 2013-09-10 10:08:33 EDT
> I have a problem that I filed this bug earlier, the later bug should be
> duplicated to the earlier one.
>
> Would you please tell me your consideration? 

It's the same for me; if you want to close the other one and reopen this one, go ahead.
Comment 12 huiqingding 2013-09-10 21:43:21 EDT
Hi, Paolo,
 
> It's the same for me; if you want to close the other one and reopen this
> one, go ahead.

Thanks for understanding. I repoen this bug.

Best regards,
Huiqing
Comment 13 huiqingding 2013-09-10 22:04:29 EDT
*** Bug 1004743 has been marked as a duplicate of this bug. ***
Comment 14 Eduardo Habkost 2013-10-04 14:49:41 EDT
The reported on comment #0 is not a duplicate of bug 100473. It is a problem caused by the difference in size of RAM blocks "*/*.rom" because the PXE rom files on RHEL-6 and RHEL-7 have different sizes. I will unduplicate the bugs.

On comment #8 a different problem was found because there was no network device in the VM.
Comment 15 huiqingding 2013-10-10 03:40:58 EDT
Migrate a RHEL7.0 guest from RHEL6.5 host to RHEL7.0 host using the following setting:
1. "-M rhel6.5.0 -cpu Conroe"
2. "-M rhel6.5.0 -cpu Penryn"
3. "-M rhel6.5.0 -cpu Nehalem"
4. "-M rhel6.5.0 -cpu Westmere"

The result is also failed. The qemu-kvm on dst host quits with the error information:
(qemu) qemu: warning: error while loading state for instance 0x0 of device 'ram'
load of migration failed

src host:
The kernel is kernel-2.6.32-421.el6.x86_64
The qemu-kvm is qemu-kvm-rhev-0.12.1.2-2.412.el6.x86_64
dst host:
The kernel is kernel-3.10.0-33.el7.x86_64
The qemu-kvm is qemu-kvm-rhev-1.5.3-8.el7.x86_64
Comment 18 juzhang 2013-10-10 04:40:18 EDT
Thanks Huding.

Hi Orit,

According to comment17, Does QE need to open new bug?

Best Regards,
Junyi
Comment 19 juzhang 2013-10-10 05:25:19 EDT
Remove needinfo since comment14 has replied this question. But new questions comes out. According to bz1004743 description, this issue only affect newer cpu model(SandyBridge, Haswell, Opteron_G4 and Opteron_G5 CPU models) but our QE hit this problem with older cpu model(Conroe). It's new bug?

Best Regards,
Junyi
Comment 20 Paolo Bonzini 2013-10-10 05:32:13 EDT
I fixed the description for bug 1004743.
Comment 21 huiqingding 2013-10-10 06:32:11 EDT
Hello, Paolo and juzhang,

I summary test result(wihtout network).
1.For RHEL7 and RHEL6.4 guest. Migration RHEL6.5 host ->RHEL7.0 host.
qemu-kvm quit in des

2.For RHEL7 and RHEL6.4 guest. Migration RHEL7.0 host ->RHEL6.5 host.
Guest black screen(guest is running status)

3.For windows guest. Migration RHEL7.0 host <->RHEL6.5 host
same as 2

Additional:
1. If booting guest with virtio-net device, will hit this bug as comment 0
2. src and des command line below

The src host command line is:
/usr/libexec/qemu-kvm -M rhel6.5.0 -cpu Nehalem -enable-kvm -m 1G -smp 4,sockets=1,cores=2,threads=2 -no-kvm-pit-reinjection -usb -device usb-tablet,id=input0 -name virtual-blk-device -rtc base=localtime,clock=host,driftfix=slew  -drive file=/home/RHEL-Server-6.4-64.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,serial=40c061dd-5d60-4fc5-865f-55db700407f0,cache=none,werror=stop,rerror=stop,aio=threads -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0  -net none  -k en-us -boot menu=on -qmp tcp:0:5557,server,nowait -serial unix:/tmp/ttyS0-2,server,nowait -monitor stdio  -vnc :2 -vga std -boot menu=on

The dst host command line is:
/usr/libexec/qemu-kvm -M rhel6.5.0 -cpu Nehalem -enable-kvm -m 1G -smp 4,sockets=1,cores=2,threads=2 -no-kvm-pit-reinjection -usb -device usb-tablet,id=input0 -name virtual-blk-device -rtc base=localtime,clock=host,driftfix=slew  -drive file=/mnt/RHEL-Server-6.4-64.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,serial=40c061dd-5d60-4fc5-865f-55db700407f0,cache=none,werror=stop,rerror=stop,aio=threads -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0  -net none  -k en-us -boot menu=on -qmp tcp:0:5557,server,nowait -serial unix:/tmp/ttyS0-2,server,nowait -monitor stdio  -vnc :2 -vga std -boot menu=on -incoming tcp:0:5800
Comment 22 Paolo Bonzini 2013-10-10 07:18:20 EDT
Hi huding, migration from RHEL7 to RHEL6.5 will not be supported.
Comment 24 Miroslav Rezanina 2013-11-06 06:26:59 EST
Fix included in -kvm-1.5.3-14.el7
Comment 34 Paolo Bonzini 2014-03-06 07:53:36 EST
This is not fixed yet.  The RHEL6 e1000 ROM was not included and in the meanwhile it grew from 128k to 256k.
Comment 36 FuXiangChun 2014-03-07 03:00:06 EST
Tested this issue according to comment0 by using 3.10.0-98.el7.x86_64 and qemu-kvm-1.5.3-50.el7.x86_64' 

Results: 1. vrtio-nic-pci and rtl8139, migration can be finished successful. 
         2. e1000 cause migration fail((qemu) qemu: warning: error while loading state for instance 0x0 of device 'ram' load of migration failed)

From KVM QE POV, this bug is about virtio-net and has been fixed in comment #31". QE would like to file a new bug for e1000 about this issue since it would be clear one bug for one issue.  For e1000, I have filed a new bug 1073774.
Comment 37 juzhang 2014-03-07 03:03:26 EST
Hi Hai,

According to comment36, I would like to set this bz as verified. Could you me your option?

Best Regards,
Junyi
Comment 38 Paolo Bonzini 2014-03-07 10:05:06 EST
Ok.
Comment 39 Hai Huang 2014-03-07 10:59:43 EST
Yes, please (see Paolo's response) and proceed with setting this BZ to verified.
We will follow-up with the remaining work with BZ 1073744.
Comment 43 Ludek Smid 2014-06-13 07:59:34 EDT
This request was resolved in Red Hat Enterprise Linux 7.0.

Contact your manager or support representative in case you have further questions about the request.

Note You need to log in before you can comment on or make changes to this bug.