Bug 985205 - QEMU core dumped when do hot-unplug virtio serial port during transfer file between host to guest with virtio serial through TCP socket
QEMU core dumped when do hot-unplug virtio serial port during transfer file b...
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: qemu-kvm (Show other bugs)
6.5
Unspecified Unspecified
high Severity medium
: rc
: ---
Assigned To: Amit Shah
Virtualization Bugs
: Regression
Depends On:
Blocks: 1000882 1003232 1007222
  Show dependency treegraph
 
Reported: 2013-07-17 01:52 EDT by Sibiao Luo
Modified: 2013-11-21 02:02 EST (History)
15 users (show)

See Also:
Fixed In Version: qemu-kvm-0.12.1.2-2.412.el6
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1007222 (view as bug list)
Environment:
Last Closed: 2013-11-21 02:02:33 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Sibiao Luo 2013-07-17 01:52:58 EDT
Description of problem:
boot guest with virtio serial though the unix socket, try to do hot-unplug the port device during transfering big size data from host, the qemu will core dumped.

Version-Release number of selected component (if applicable):
host info:
2.6.32-398.el6.x86_64
qemu-kvm-0.12.1.2-2.378.el6.x86_64
guest info:
2.6.32-398.el6.x86_64

How reproducible:
always

Steps to Reproduce:
1.boot guest with virtio serial though the unix socket(two ports).
e.g:...-device virtio-serial-pci,id=virtio-serial0,max_ports=16,vectors=0,bus=pci.0,addr=0x3 -chardev socket,id=channel1,host=localhost,port=12345,server,nowait -device virtserialport,chardev=channel1,name=com.redhat.rhevm.vdsm,bus=virtio-serial0.0,id=port1 -chardev socket,id=channel2,host=localhost,port=12346,server,nowait -device virtserialport,chardev=channel2,name=com.redhat.rhevm.vdsm,bus=virtio-serial0.0,id=port2
2.dd a 2G file in the host.
guest ]# dd if=/dev/zero of=test-file bs=1M count=2000
2000+0 records in
2000+0 records out
2097152000 bytes (2.1 GB) copied, 5.20488 s, 403 MB/s
3.try to do hot-unplug the port device during transfering big size data from host.
host ]# cat test-file | nc localhost 12345
guest]# cat /dev/vport0p1 >> sluo
(qemu) device_del port1

Actual results:
after step 3, qemu will core dumped.
(qemu) device_del port1
(qemu) Segmentation fault (core dumped)
(gdb) bt
#0  0x0000000000000000 in ?? ()
#1  0x00007f88f6178d6a in qemu_chr_be_write (chan=<value optimized out>, cond=<value optimized out>, 
    opaque=0x7f88f6bc7740) at /usr/src/debug/qemu-kvm-0.12.1.2/qemu-char.c:191
#2  tcp_chr_read (chan=<value optimized out>, cond=<value optimized out>, opaque=0x7f88f6bc7740)
    at /usr/src/debug/qemu-kvm-0.12.1.2/qemu-char.c:2349
#3  0x00007f88f5780eb2 in g_main_context_dispatch () from /lib64/libglib-2.0.so.0
#4  0x00007f88f60f4fba in glib_select_poll (timeout=1000) at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:3993
#5  main_loop_wait (timeout=1000) at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:4066
#6  0x00007f88f611785a in kvm_main_loop () at /usr/src/debug/qemu-kvm-0.12.1.2/qemu-kvm.c:2244
#7  0x00007f88f60f8938 in main_loop (argc=60, argv=<value optimized out>, envp=<value optimized out>)
    at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:4260
#8  main (argc=60, argv=<value optimized out>, envp=<value optimized out>) at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:6627
(gdb) 

Expected results:
qemu should no core dumped.

Additional info:
Comment 1 Sibiao Luo 2013-07-17 01:55:04 EDT
My qemu-kvm command line:
# /usr/libexec/qemu-kvm -S -M rhel6.5.0 -cpu SandyBridge -enable-kvm -m 4096 -smp 4,sockets=2,cores=2,threads=1 -no-kvm-pit-reinjection -name sluo -uuid 43425b70-86e5-4664-bf2c-3b76699b8bec -rtc base=localtime,clock=host,driftfix=slew -device virtio-serial-pci,id=virtio-serial0,max_ports=16,vectors=0,bus=pci.0,addr=0x3 -chardev socket,id=channel1,host=localhost,port=12345,server,nowait -device virtserialport,chardev=channel1,name=com.redhat.rhevm.vdsm,bus=virtio-serial0.0,id=port1 -chardev socket,id=channel2,host=localhost,port=12346,server,nowait -device virtserialport,chardev=channel2,name=com.redhat.rhevm.vdsm,bus=virtio-serial0.0,id=port2 -drive file=/home/RHEL6.5-20130712.n.0-6.5-Server-x64.qcow2,if=none,id=drive-system-disk,format=qcow2,cache=none,aio=native,werror=stop,rerror=stop,serial="QEMU-DISK1" -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-system-disk,id=system-disk,bootindex=1 -device virtio-balloon-pci,id=ballooning,bus=pci.0,addr=0x5 -global PIIX4_PM.disable_s3=0 -global PIIX4_PM.disable_s4=0 -netdev tap,id=hostnet0,vhost=on,script=/etc/qemu-ifup -device virtio-net-pci,netdev=hostnet0,id=virtio-net-pci0,mac=2C:41:38:B6:40:21,bus=pci.0,addr=0x6,bootindex=2 -drive file=/home/my-data-disk.qcow2,if=none,id=drive-data-disk,format=qcow2,cache=none,aio=native,werror=stop,rerror=stop,serial="QEMU-DISK2" -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x7,drive=drive-data-disk,id=data-disk,bootindex=3 -k en-us -boot menu=on -qmp tcp:0:4444,server,nowait -serial unix:/tmp/ttyS0,server,nowait -vnc :1 -spice port=5931,disable-ticketing -monitor stdio

The full bt log as following:
(gdb) bt full
#0  0x0000000000000000 in ?? ()
No symbol table info available.
#1  0x00007f88f6178d6a in qemu_chr_be_write (chan=<value optimized out>, cond=<value optimized out>, 
    opaque=0x7f88f6bc7740) at /usr/src/debug/qemu-kvm-0.12.1.2/qemu-char.c:191
No locals.
#2  tcp_chr_read (chan=<value optimized out>, cond=<value optimized out>, opaque=0x7f88f6bc7740)
    at /usr/src/debug/qemu-kvm-0.12.1.2/qemu-char.c:2349
        chr = 0x7f88f6bc7740
        s = 0x7f88f6bc7800
        buf = '\000' <repeats 4095 times>
        len = <value optimized out>
        size = <value optimized out>
#3  0x00007f88f5780eb2 in g_main_context_dispatch () from /lib64/libglib-2.0.so.0
No symbol table info available.
#4  0x00007f88f60f4fba in glib_select_poll (timeout=1000) at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:3993
        context = 0x7f88f6bc9900
#5  main_loop_wait (timeout=1000) at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:4066
        ioh = <value optimized out>
        rfds = {fds_bits = {8796093022209, 0 <repeats 15 times>}}
        wfds = {fds_bits = {0 <repeats 16 times>}}
        xfds = {fds_bits = {0 <repeats 16 times>}}
        ret = <value optimized out>
        nfds = 43
        tv = {tv_sec = 0, tv_usec = 999984}
#6  0x00007f88f611785a in kvm_main_loop () at /usr/src/debug/qemu-kvm-0.12.1.2/qemu-kvm.c:2244
        fds = {29, 30}
        mask = {__val = {268443712, 0 <repeats 15 times>}}
        sigfd = 31
#7  0x00007f88f60f8938 in main_loop (argc=60, argv=<value optimized out>, envp=<value optimized out>)
    at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:4260
        r = <value optimized out>
#8  main (argc=60, argv=<value optimized out>, envp=<value optimized out>) at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:6627
        gdbstub_dev = 0x0
        i = <value optimized out>
        snapshot = 0
        linux_boot = 0
        initrd_filename = 0x0
        kernel_filename = 0x0
        kernel_cmdline = 0x7f88f62aec0f ""
        boot_devices = "cad", '\000' <repeats 29 times>
        ds = <value optimized out>
        dcl = <value optimized out>
        cyls = 0
        heads = 0
        secs = 0
        translation = 0
        hda_opts = 0x7f88f660b210
        opts = <value optimized out>
        olist = <value optimized out>
        optind = 60
        optarg = 0x7ffffb89f8a1 "stdio"
        loadvm = 0x0
        machine = 0x7f88f6604a20
        cpu_model = 0x7ffffb89f29b "SandyBridge"
        fds = {-167280640, 32648}
        tb_size = 0
        pid_file = 0x0
        incoming = 0x0
        fd = 0
        pwd = 0x0
        chroot_dir = 0x0
        run_as = 0x0
        env = <value optimized out>
        show_vnc_port = 0
        defconfig = <value optimized out>
        defconfig_verbose = <value optimized out>
(gdb)

Cpu info of my host:
processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 42
model name	: Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz
stepping	: 7
cpu MHz		: 1600.000
cache size	: 8192 KB
physical id	: 0
siblings	: 8
core id		: 3
cpu cores	: 4
apicid		: 7
initial apicid	: 7
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm ida arat epb xsaveopt pln pts dts tpr_shadow vnmi flexpriority ept vpid
bogomips	: 6784.27
clflush size	: 64
cache_alignment	: 64
address sizes	: 36 bits physical, 48 bits virtual
power management:
Comment 2 Qunfang Zhang 2013-07-17 07:23:20 EDT
Is this a regression or new issue? We had a similar bug before:
Bug 745758 - Segmentation fault occurs after hot unplug virtio-serial-pci while virtio-serial-port in use
Comment 3 Sibiao Luo 2013-07-17 07:38:47 EDT
Two-way stream socket to create, if path is specified, a Unix socket is created and if port is also specified, a TCP socket is created. 

Sorry for my mistake during all my description, just update a comment here to make clear for it.

Best Regards,
sluo
Comment 12 Amit Shah 2013-08-28 01:17:09 EDT
Patches submitted upstream

http://thread.gmane.org/gmane.comp.emulators.qemu/230226
Comment 18 mazhang 2013-10-14 02:34:39 EDT
Reproduce this bug with qemu-kvm-0.12.1.2-2.398.el6.x86_64


host:
RHEL6.5-Snapshot-2.0
qemu-img-0.12.1.2-2.398.el6.x86_64
gpxe-roms-qemu-0.9.7-6.10.el6.noarch
qemu-kvm-tools-0.12.1.2-2.398.el6.x86_64
qemu-kvm-debuginfo-0.12.1.2-2.398.el6.x86_64
qemu-kvm-0.12.1.2-2.398.el6.x86_64
kernel-2.6.32-422.el6.x86_64

guest:
kernel-2.6.32-358.el6.x86_64

steps:
1 cli:
gdb --args /usr/libexec/qemu-kvm \
-M pc \
-cpu Nehalem \
-m 4096 \
-smp 4,sockets=2,cores=2,threads=1,maxcpus=16 \
-enable-kvm \
-name rhel6u5 \
-uuid 990ea161-6b67-47b2-b803-19fb01d30d12 \
-smbios type=1,manufacturer='Red Hat',product='RHEV Hypervisor',version=el6,serial=koTUXQrb,uuid=feebc8fd-f8b0-4e75-abc3-e63fcdb67170 \
-k en-us \
-rtc base=localtime,clock=host,driftfix=slew \
-no-kvm-pit-reinjection \
-monitor stdio \
-boot menu=on \
-bios /usr/share/seabios/bios.bin \
-qmp tcp:0:6666,server,nowait \
-nodefconfig \
-nodefaults \
-global PIIX4_PM.disable_s3=0 \
-global PIIX4_PM.disable_s4=0 \
-vga qxl \
-spice port=5900,disable-ticketing \
-drive file=/home/rhel6u5.qcow2,if=none,id=gfs0,cache=none,aio=native \
-device virtio-blk-pci,drive=gfs0,bootindex=0 \
-device virtio-serial-pci,id=virtio-serial0,max_ports=16,vectors=0,bus=pci.0 \
-chardev socket,id=channel1,host=localhost,port=12345,server,nowait \
-device virtserialport,chardev=channel1,name=com.redhat.rhevm.vdsm,bus=virtio-serial0.0,id=port1 \
-chardev socket,id=channel2,host=localhost,port=12346,server,nowait \
-device virtserialport,chardev=channel2,name=com.redhat.rhevm.vdsm,bus=virtio-serial0.0,id=port2 \

2.dd a 2G file in the host.
(host)# dd if=/dev/zero of=test-file bs=1M count=2000

3.try to do hot-unplug the port device during transfering big size data from host.
(host)# cat test-file | nc localhost 12346
(guest)# cat /dev/vport0p2 >> aaa
(qemu) device_del port2

Actual results:
after step 3, qemu will core dumped.
(gdb) bt full
#0  0x0000000000000000 in ?? ()
No symbol table info available.
#1  0x00007ffff7e53c5a in qemu_chr_be_write (chan=<value optimized out>, cond=<value optimized out>, 
    opaque=0x7ffff86e61d0) at /usr/src/debug/qemu-kvm-0.12.1.2/qemu-char.c:192
No locals.
#2  tcp_chr_read (chan=<value optimized out>, cond=<value optimized out>, opaque=0x7ffff86e61d0)
    at /usr/src/debug/qemu-kvm-0.12.1.2/qemu-char.c:2303
        chr = 0x7ffff86e61d0
        s = 0x7ffff86e6290
        buf = '\000' <repeats 4095 times>
        len = <value optimized out>
        size = <value optimized out>
#3  0x00007ffff7232eb2 in g_main_context_dispatch () from /lib64/libglib-2.0.so.0
No symbol table info available.
#4  0x00007ffff7dc9faa in glib_select_poll (timeout=1000) at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:3999
        context = 0x7ffff86e4360
#5  main_loop_wait (timeout=1000) at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:4072
        ioh = <value optimized out>
        rfds = {fds_bits = {137438953473, 0 <repeats 15 times>}}
        wfds = {fds_bits = {0 <repeats 16 times>}}
        xfds = {fds_bits = {0 <repeats 16 times>}}
        ret = <value optimized out>
        nfds = 37
        tv = {tv_sec = 0, tv_usec = 999988}
#6  0x00007ffff7decd3a in kvm_main_loop () at /usr/src/debug/qemu-kvm-0.12.1.2/qemu-kvm.c:2245
        fds = {29, 30}
        mask = {__val = {268443712, 0 <repeats 15 times>}}
        sigfd = 31
#7  0x00007ffff7dcccf9 in main_loop (argc=53, argv=<value optimized out>, envp=<value optimized out>)
    at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:4266
        r = <value optimized out>
#8  main (argc=53, argv=<value optimized out>, envp=<value optimized out>)
Comment 19 mazhang 2013-10-14 02:44:45 EDT
Verify this bug with qemu-kvm-0.12.1.2-2.412.el6.x86_64

host:
RHEL6.5-Snapshot-2.0
qemu-kvm-0.12.1.2-2.412.el6.x86_64
gpxe-roms-qemu-0.9.7-6.10.el6.noarch
qemu-kvm-tools-0.12.1.2-2.412.el6.x86_64
qemu-kvm-debuginfo-0.12.1.2-2.412.el6.x86_64
qemu-img-0.12.1.2-2.412.el6.x86_64

guest:
kernel-2.6.32-358.el6.x86_64

steps refer to comment#8

result:
Remove virtserialport during transfer data, guest works well and prompt "No such device", no crash or segmentation fault.

so this bug has been fixed.
Comment 21 errata-xmlrpc 2013-11-21 02:02:33 EST
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHSA-2013-1553.html

Note You need to log in before you can comment on or make changes to this bug.