Bug 816787

Summary: internal-error on dest host after migration finished at getting pxe config file list
Product: Red Hat Enterprise Linux 6 Reporter: Chao Yang <chayang>
Component: qemu-kvmAssignee: Orit Wasserman <owasserm>
Status: CLOSED DUPLICATE QA Contact: Virtualization Bugs <virt-bugs>
Severity: medium Docs Contact:
Priority: high    
Version: 6.3CC: acathrow, areis, bsarathy, dyasny, hhuang, juzhang, michen, mkenneth, shuang, shu, tburke, virt-maint
Target Milestone: rc   
Target Release: 6.4   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-10-17 08:40:01 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 798682    
Attachments:
Description Flags
cli and qemu-kvm log
none
migration internal error at pxe booting
none
system log on source host
none
system log on destination host none

Description Chao Yang 2012-04-27 02:08:29 UTC 
Description of problem:
Live migrate a VM during transferring pxelinux.0, dest host hit "paused (internal-error)" when migration finished at getting and parsing config file list.

on destination host: 
(qemu) info status
VM Status: paused (internal-error)
(qemu) cont
Resetting the Virtual Machine is required

Version-Release number of selected component (if applicable):
qemu-kvm-0.12.1.2-2.285.el6.x86_64


How reproducible:
every time, if migration finished at getting/parsing config file list.

Steps to Reproduce:
1.
2.
3.
  
Actual results:


Expected results:


Additional info:
The issue exists on 6.2.z
Comment 1 Chao Yang 2012-04-27 02:09:43 UTC 
CLI:
# /usr/libexec/qemu-kvm -M rhel6.3.0 -cpu Nehalem -enable-kvm -m 2048 -smp 2,sockets=1,cores=2,threads=1 -usb -device usb-tablet,id=input0 -name sluo_test -uuid `uuidgen` -drive file=/home/win_2008_r2_sp1_x64.raw,format=raw,if=none,id=drive-disk,cache=none,werror=ignore,rerror=ignore -device virtio-blk-pci,scsi=off,drive=drive-disk,id=image,bootindex=2 -netdev tap,id=hostnet0,script=/etc/qemu-ifup -device virtio-net-pci,netdev=hostnet0,id=virtio-net-pci0,bootindex=1 -device virtio-balloon-pci,id=ballooning -boot menu=on -spice port=5931,disable-ticketing -vga qxl -monitor stdio
Comment 4 Orit Wasserman 2012-05-01 08:04:46 UTC 
(In reply to comment #1)
> CLI:
> # /usr/libexec/qemu-kvm -M rhel6.3.0 -cpu Nehalem -enable-kvm -m 2048 -smp
> 2,sockets=1,cores=2,threads=1 -usb -device usb-tablet,id=input0 -name sluo_test
> -uuid `uuidgen` -drive
> file=/home/win_2008_r2_sp1_x64.raw,format=raw,if=none,id=drive-disk,cache=none,werror=ignore,rerror=ignore
> -device virtio-blk-pci,scsi=off,drive=drive-disk,id=image,bootindex=2 -netdev
> tap,id=hostnet0,script=/etc/qemu-ifup -device
> virtio-net-pci,netdev=hostnet0,id=virtio-net-pci0,bootindex=1 -device
> virtio-balloon-pci,id=ballooning -boot menu=on -spice
> port=5931,disable-ticketing -vga qxl -monitor stdio

can you provide logs and cpuinfo please (on both source and destination).
Comment 5 Chao Yang 2012-05-02 03:36:42 UTC 
Created attachment 581516 [details]
cli and qemu-kvm log
Comment 6 Chao Yang 2012-05-02 03:37:45 UTC 
Created attachment 581517 [details]
migration internal error at pxe booting
Comment 7 Chao Yang 2012-05-02 03:38:14 UTC 
(In reply to comment #4)
> can you provide logs and cpuinfo please (on both source and destination).
I retested on 6.2 host, reproduced again when doing ping-pong migration.
Log and CLI attached as well as a snapshot
Comment 8 Orit Wasserman 2012-05-02 07:13:08 UTC 
(In reply to comment #7)
> (In reply to comment #4)
> > can you provide logs and cpuinfo please (on both source and destination).
> I retested on 6.2 host, reproduced again when doing ping-pong migration.
> Log and CLI attached as well as a snapshot

What is the kernel version ?
Can you also add systems logs (/var/log/messages) especially for the dest host
Comment 9 Chao Yang 2012-05-03 02:53:39 UTC 
(In reply to comment #8)
> (In reply to comment #7)
> > (In reply to comment #4)
> > > can you provide logs and cpuinfo please (on both source and destination).
> > I retested on 6.2 host, reproduced again when doing ping-pong migration.
> > Log and CLI attached as well as a snapshot
> 
> What is the kernel version ?
Source: 2.6.32-220.el6.x86_64
Destination: 2.6.32-220.16.1.el6.x86_64

> Can you also add systems logs (/var/log/messages) especially for the dest host
Sure.
Comment 10 Chao Yang 2012-05-03 03:11:57 UTC 
Created attachment 581758 [details]
system log on source host
Comment 11 Chao Yang 2012-05-03 03:13:25 UTC 
Created attachment 581760 [details]
system log on destination host
Comment 12 Orit Wasserman 2012-05-09 07:35:09 UTC 
(In reply to comment #11)
> Created attachment 581760 [details]
> system log on destination host

I want to confirm, the internal error was in the destination machine which was the Sandy bridge ?
I'm asking because I saw that you did live migration ping pong.
Comment 13 Chao Yang 2012-05-09 10:15:40 UTC 
(In reply to comment #12)
> (In reply to comment #11)
> > Created attachment 581760 [details]
> > system log on destination host
> 
> I want to confirm, the internal error was in the destination machine which was
> the Sandy bridge ?
> I'm asking because I saw that you did live migration ping pong.

Yes, one of the Intel host was Sandy Bridge. This is reproducible on local Sandy Bridge when local migrating. Let me try non Sandy Bridge host and update here.
Comment 15 Orit Wasserman 2012-06-21 04:46:52 UTC 
There is a fix for BZ 756044 https://bugzilla.redhat.com/show_bug.cgi?id=756044
Can you try to see if it helps with the issue ?
Comment 19 RHEL Program Management 2012-07-10 07:16:50 UTC 
This request was not resolved in time for the current release.
Red Hat invites you to ask your support representative to
propose this request, if still desired, for consideration in
the next release of Red Hat Enterprise Linux.
Comment 20 Orit Wasserman 2012-07-10 11:13:07 UTC 
Hi, any progress with checking the patch ?
Comment 21 RHEL Program Management 2012-07-11 02:06:52 UTC 
This request was erroneously removed from consideration in Red Hat Enterprise Linux 6.4, which is currently under development.  This request will be evaluated for inclusion in Red Hat Enterprise Linux 6.4.
Comment 22 Chao Yang 2012-07-11 11:57:11 UTC 
It is still reproducible while local migrating on 2.6.32-284.el6.x86_64, qemu-kvm-0.12.1.2-2.295.el6.x86_64 on Sandy Bridge host.

# /usr/libexec/qemu-kvm -M rhel6.3.0 -cpu Nehalem -enable-kvm -m 2048 -smp 2,sockets=1,cores=2,threads=1 -usb -device usb-tablet,id=input0 -name test -uuid `uuidgen` -drive file=sys.raw,format=raw,if=none,id=drive-disk,cache=none,werror=ignore,rerror=ignore -device virtio-blk-pci,scsi=off,drive=drive-disk,id=image,bootindex=2 -netdev tap,id=hostnet0,script=/etc/qemu-ifup -device virtio-net-pci,netdev=hostnet0,id=virtio-net-pci0,bootindex=1 -device virtio-balloon-pci,id=ballooning -boot menu=on -spice port=8000,disable-ticketing -vga qxl -monitor stdio -incoming tcp:0:7000
do_spice_init: starting 0.10.1
spice_server_add_interface: SPICE_INTERFACE_MIGRATION
spice_server_add_interface: SPICE_INTERFACE_KEYBOARD
spice_server_add_interface: SPICE_INTERFACE_MOUSE
spice_server_add_interface: SPICE_INTERFACE_QXL
red_worker_main: begin
display_channel_create: create display channel
cursor_channel_create: create cursor channel
QEMU 0.12.1 monitor - type 'help' for more information
(qemu) kvm: unhandled exit 80000021
kvm_run returned -22
handle_dev_stop: stop

(qemu) info status
VM status: paused (internal-error)
(qemu) cont
Resetting the Virtual Machine is required
Comment 23 Orit Wasserman 2012-08-07 13:41:00 UTC 
what guest are you using ?
Comment 24 Chao Yang 2012-08-08 02:17:43 UTC 
(In reply to comment #23)
> what guest are you using ?

I reported this bug with win2k8r2sp1 guest. I remember also tried rhel6.3 guest, it didn't get this issue disappeared.
Comment 25 Orit Wasserman 2012-08-08 08:43:32 UTC 
(In reply to comment #24)
> (In reply to comment #23)
> > what guest are you using ?
> 
> I reported this bug with win2k8r2sp1 guest. I remember also tried rhel6.3
> guest, it didn't get this issue disappeared.
32 or 64 bit guest ?
Comment 26 Chao Yang 2012-08-08 10:30:50 UTC 
(In reply to comment #25)
> (In reply to comment #24)
> > (In reply to comment #23)
> > > what guest are you using ?
> > 
> > I reported this bug with win2k8r2sp1 guest. I remember also tried rhel6.3
> > guest, it didn't get this issue disappeared.
> 32 or 64 bit guest ?

64 bit guest.
Comment 27 Orit Wasserman 2012-10-16 10:41:00 UTC 
I think this is a duplicate of BZ 852612

can you try to reproduce with qemu-kvm-0.12.1.2-2.325.el6?
Comment 28 Chao Yang 2012-10-17 07:11:16 UTC 
(In reply to comment #27)
> I think this is a duplicate of BZ 852612
> 
> can you try to reproduce with qemu-kvm-0.12.1.2-2.325.el6?

Yep, it makes the problem disappears.
Comment 29 Chao Yang 2012-10-17 07:13:53 UTC 
(In reply to comment #28)
> (In reply to comment #27)
> > I think this is a duplicate of BZ 852612
> > 
> > can you try to reproduce with qemu-kvm-0.12.1.2-2.325.el6?
> 
> Yep, it makes the problem disappears.

Passed with qemu-kvm-0.12.1.2-2.327.el6.x86_64.
Comment 30 Orit Wasserman 2012-10-17 08:40:01 UTC 

*** This bug has been marked as a duplicate of bug 852612 ***