Bug 1037455 - Migration from RHEL6.5 host to RHEL7.0 host with spice failed (aborted)
Summary: Migration from RHEL6.5 host to RHEL7.0 host with spice failed (aborted)
Keywords:
Status: CLOSED DUPLICATE of bug 1035184
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: spice
Version: 7.0
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: rc
: ---
Assignee: Default Assignee for SPICE Bugs
QA Contact: Desktop QE
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2013-12-03 08:28 UTC by Qunfang Zhang
Modified: 2013-12-04 03:23 UTC (History)
9 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2013-12-03 14:57:59 UTC
Target Upstream Version:


Attachments (Terms of Use)

Description Qunfang Zhang 2013-12-03 08:28:22 UTC
Description of problem:
Boot a guest with spice and qxl on RHEL6.5 released host, and then migrate to RHEL7.0 host with the same command line (both host with -M rhel6.5.0 machine type). Migration failed and qemu-kvm aborted. 

VNC has no problem. 

Version-Release number of selected component (if applicable):
RHEL6.5:
kernel-2.6.32-431.el6.x86_64
qemu-kvm-rhev-0.12.1.2-2.415.el6_5.3.x86_64
spice-server-0.12.4-6.el6.x86_64

RHEL7.0:
kernel-3.10.0-57.el7.x86_64
qemu-kvm-1.5.3-20.el7.x86_64
spice-server-0.12.4-3.el7.x86_64

How reproducible:
Always

Steps to Reproduce:
1. Boot up a guest on RHEL6.5 host with spice:

(gdb) r -cpu SandyBridge -M rhel6.5.0 -enable-kvm -m 4096 -smp 2,sockets=2,cores=1,threads=1 -name rhel7.0-64 -uuid 9a0e67ec-f286-d8e7-0548-0c1c9ec93009 -nodefconfig -nodefaults -monitor stdio -rtc base=utc,clock=host,driftfix=slew -no-kvm-pit-reinjection -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x7 -device virtio-scsi-pci,id=scsi0,bus=pci.0,addr=0x4 -drive file=gluster://10.66.**.**/gv0/rhel7.0-64-qzhang.raw,if=none,id=drive-disk0,format=raw,cache=none,id=disk0 -device scsi-hd,drive=drive-disk0,bus=scsi0.0,id=scsi-disk0,bootindex=1  -drive file=gluster://10.66.**.**/gv0/boot.iso,if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw -device scsi-cd,drive=drive-ide0-1-0,bus=scsi0.0,id=scsi-cdrom -netdev tap,id=hostnet0,vhost=on,script=/etc/ovs-ifup,downscript=/etc/ovs-ifdown -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:d5:51:8a,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -chardev socket,id=channel1,path=/tmp/helloworld1,server,nowait -device virtserialport,chardev=channel1,name=port1,bus=virtio-serial0.0,id=port1 -chardev socket,id=channel2,path=/tmp/helloworld2,server,nowait -device virtserialport,chardev=channel2,name=port2,bus=virtio-serial0.0,id=port2 -chardev spicevmc,id=charchannel0,name=vdagent -device virtserialport,bus=virtio-serial0.0,nr=3,chardev=charchannel0,id=channel0,name=com.redhat.spice.0 -device usb-tablet,id=input0 -spice port=5900,disable-ticketing,seamless-migration=on -vga qxl -global qxl-vga.vram_size=67108864 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6  -drive if=none,id=drive-fdc0-0-0,format=raw,cache=none -global isa-fdc.driveA=drive-fdc0-0-0 -qmp tcp:0:5555,server,nowait -global PIIX4_PM.disable_s3=0 -global PIIX4_PM.disable_s4=0 -drive file=gluster://10.66.**.**/gv0/disk1.qcow2,if=none,id=disk1,format=qcow2,cache=none,id=disk1 -device virtio-blk-pci,bus=pci.0,addr=0x9,drive=disk1,id=disk1 -drive file=gluster://10.66.**.**/gv0/disk2.qcow2,if=none,id=disk2,format=qcow2,cache=none,id=disk2 -device ide-drive,bus=ide.0,unit=1,drive=disk2,id=disk2  -netdev tap,id=hostnet1,script=/etc/ovs-ifup,downscript=/etc/ovs-ifdown -device e1000,netdev=hostnet1,id=net1,mac=52:54:00:d5:51:2a,bus=pci.0,addr=0xa -netdev tap,id=hostnet2,script=/etc/ovs-ifup,downscript=/etc/ovs-ifdown -device e1000,netdev=hostnet2,id=net2,mac=52:54:00:d5:51:3a,bus=pci.0,addr=0xb 


2. Boot up the guest on destination host with "-incoming tcp:0:5800"

3. Migrate the guest from rhel6.5 host to rhel7.0 host. 

Actual results:
Guest aborted on destination host. 

Expected results:
Guest should work well on destination host after migration.

Additional info:

Program received signal SIGABRT, Aborted.
0x00007ffff30db979 in raise () from /lib64/libc.so.6
(gdb) bt
#0  0x00007ffff30db979 in raise () from /lib64/libc.so.6
#1  0x00007ffff30dd088 in abort () from /lib64/libc.so.6
#2  0x00007ffff3e9051c in spice_logv (log_domain=0x7ffff3f061c6 "Spice", log_level=SPICE_LOG_LEVEL_ERROR, 
    strloc=0x7ffff3f06569 "char_device.c:937", 
    function=0x7ffff3f06c60 <__FUNCTION__.21514> "spice_char_device_state_restore", 
    format=0x7ffff3f0619e "assertion `%s' failed", args=args@entry=0x7fffffffd188) at log.c:109
#3  0x00007ffff3e90668 in spice_log (log_domain=log_domain@entry=0x7ffff3f061c6 "Spice", 
    log_level=log_level@entry=SPICE_LOG_LEVEL_ERROR, strloc=strloc@entry=0x7ffff3f06569 "char_device.c:937", 
    function=function@entry=0x7ffff3f06c60 <__FUNCTION__.21514> "spice_char_device_state_restore", 
    format=format@entry=0x7ffff3f0619e "assertion `%s' failed") at log.c:123
#4  0x00007ffff3e384da in spice_char_device_state_restore (dev=0x5555569b9db0, mig_data=0x5555569b9320)
    at char_device.c:937
#5  0x00007ffff3e7217b in reds_agent_state_restore (mig_data=<optimized out>) at reds.c:1364
#6  0x00007ffff3e75835 in attach_to_red_agent (sin=0x555556502bf8) at reds.c:3594
#7  spice_server_char_device_add_interface (sin=0x555556502bf8, s=0x555556504810) at reds.c:3698
#8  spice_server_add_interface (s=0x555556504810, sin=sin@entry=0x555556502bf8) at reds.c:3836
#9  0x0000555555749995 in qemu_spice_add_interface (sin=sin@entry=0x555556502bf8) at ui/spice-core.c:831
#10 0x0000555555740ebb in vmc_register_interface (scd=0x555556502bf0) at spice-qemu-char.c:124
#11 0x000055555578b7fe in virtio_serial_post_load_timer_cb (opaque=<optimized out>)
    at /usr/src/debug/qemu-1.5.3/hw/char/virtio-serial-bus.c:602
#12 0x0000555555728156 in qemu_run_timers (clock=0x5555564f9f30) at qemu-timer.c:394
#13 0x00005555557283f5 in qemu_run_timers (clock=<optimized out>) at qemu-timer.c:459
#14 qemu_run_all_timers () at qemu-timer.c:452
#15 0x00005555556f608e in main_loop_wait (nonblocking=<optimized out>) at main-loop.c:470
#16 0x00005555555ff3f8 in main_loop () at vl.c:1986
#17 main (argc=<optimized out>, argv=<optimized out>, envp=<optimized out>) at vl.c:4343
(gdb)

Comment 1 Qunfang Zhang 2013-12-03 08:32:34 UTC
Not sure whether it is relevant, but the "info qtree" result for qxl is not the same between RHEL6.5 host and RHEL7.0 host. The command line on the two hosts are the same, and both using "-M rhel6.5.0"

On rhel 6.5:

 dev: qxl-vga, id ""
        dev-prop: ram_size = 67108864
        dev-prop: vram_size = 67108864
        dev-prop: revision = 4
        dev-prop: debug = 0
        dev-prop: guestdebug = 0
        dev-prop: cmdlog = 0
        bus-prop: addr = 02.0
        bus-prop: romfile = "vgabios-qxl.bin"
        bus-prop: rombar = 1
        bus-prop: multifunction = off
        class VGA controller, addr 00:02.0, pci id 1b36:0100 (sub 1af4:1100)
        bar 0: mem at 0xffffffffffffffff [0x3fffffe]
        bar 1: mem at 0xffffffffffffffff [0x3fffffe]
        bar 2: mem at 0xffffffffffffffff [0x1ffe]
        bar 3: i/o at 0xffffffffffffffff [0x1e]
        bar 6: mem at 0xffffffffffffffff [0xfffe]

On rhel7.0:

 dev: qxl-vga, id ""
        ram_size = 67108864
        vram_size = 67108864
        revision = 4
        debug = 0
        guestdebug = 0
        cmdlog = 0
        ram_size_mb = 4294967295
        vram_size_mb = 4294967295
        vram64_size_mb = 4294967295
        vgamem_mb = 16
        surfaces = 1024
        addr = 02.0
        romfile = "vgabios-qxl.bin"
        rombar = 1
        multifunction = off
        command_serr_enable = off
        class VGA controller, addr 00:02.0, pci id 1b36:0100 (sub 1af4:1100)
        bar 0: mem at 0xffffffffffffffff [0x3fffffe]
        bar 1: mem at 0xffffffffffffffff [0x3fffffe]
        bar 2: mem at 0xffffffffffffffff [0x1ffe]
        bar 3: i/o at 0xffffffffffffffff [0x1e]
        bar 6: mem at 0xffffffffffffffff [0xfffe]

Comment 3 Gerd Hoffmann 2013-12-03 14:29:21 UTC
Stack trace looks like the bug is in the spice agent channel setup.
Failed assert in spice_char_device_state_restore().
Assigning to spice-server for investigation.
Hmm?  No spice-server component?  Picking spice instead.

Does it work if you configure the machine without spicevmc?
Is a client connected while migrating?
Do you use seamless migration?

Comment 4 Christophe Fergeau 2013-12-03 14:57:59 UTC
This is a duplicae of rhbz#1035184

*** This bug has been marked as a duplicate of bug 1035184 ***

Comment 5 Qunfang Zhang 2013-12-04 03:22:01 UTC
(In reply to Gerd Hoffmann from comment #3)
> Stack trace looks like the bug is in the spice agent channel setup.
> Failed assert in spice_char_device_state_restore().
> Assigning to spice-server for investigation.
> Hmm?  No spice-server component?  Picking spice instead.
> 
> Does it work if you configure the machine without spicevmc?
> Is a client connected while migrating?
> Do you use seamless migration?

Seems the problems only occurs when "seamless-migration=on"+"withspicevmc"+"login guest". Please check the following detail info:

spice client connected,seamless-migration=on:

1. with spicevmc, do not login guest: pass.
2. with spicevmc, login guest:  fail.
3. without spicevmc, do not login guest: pass.
4. without spicevmc, login guest: pass.

spice client disconnected,seamless-migration=off:
5. with spicevmc, login guest: pass

spice client connected, seamless-migration=off:
6. withspicevmc,login guest: pass.

Comment 6 Qunfang Zhang 2013-12-04 03:23:28 UTC
(In reply to Qunfang Zhang from comment #5)
> (In reply to Gerd Hoffmann from comment #3)
> > Stack trace looks like the bug is in the spice agent channel setup.
> > Failed assert in spice_char_device_state_restore().
> > Assigning to spice-server for investigation.
> > Hmm?  No spice-server component?  Picking spice instead.
> > 
> > Does it work if you configure the machine without spicevmc?
> > Is a client connected while migrating?
> > Do you use seamless migration?
> 
> Seems the problems only occurs when
> "seamless-migration=on"+"withspicevmc"+"login guest". Please check the
> following detail info:
> 
> spice client connected,seamless-migration=on:
> 
> 1. with spicevmc, do not login guest: pass.
> 2. with spicevmc, login guest:  fail.
> 3. without spicevmc, do not login guest: pass.
> 4. without spicevmc, login guest: pass.
> 
> spice client disconnected,seamless-migration=off:
Should be: spice client disconnected,seamless-migration=on

> 5. with spicevmc, login guest: pass
> 
> spice client connected, seamless-migration=off:
> 6. withspicevmc,login guest: pass.


Note You need to log in before you can comment on or make changes to this bug.