Bug 880139
| Summary: | guest abort when transfer file with virtio-serial from guest to host | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 6 | Reporter: | langfang <flang> | ||||
| Component: | qemu-kvm | Assignee: | Amit Shah <amit.shah> | ||||
| Status: | CLOSED WORKSFORME | QA Contact: | Virtualization Bugs <virt-bugs> | ||||
| Severity: | medium | Docs Contact: | |||||
| Priority: | high | ||||||
| Version: | 6.4 | CC: | acathrow, areis, bsarathy, chayang, flang, juzhang, lnovich, mdeng, mkenneth, qzhang, rhod, sluo, sradvan, virt-maint, xfu, xigao | ||||
| Target Milestone: | rc | ||||||
| Target Release: | --- | ||||||
| Hardware: | Unspecified | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | Doc Type: | Bug Fix | |||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2014-06-12 12:15:31 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Attachments: |
|
||||||
my CLI: Starting program: /usr/libexec/qemu-kvm -M rhel6.4.0 -cpu Opteron_G3 -enable-kvm -uuid `uuidgen` -rtc base=localtime,clock=host,driftfix=slew -m 2G -smp 4,sockets=2,cores=2,threads=1 -name rhel6.3.z -drive file=/home/RHEL6.3.z-64,if=none,id=drive-virtio-disk0,format=qcow2,cache=none,aio=native,media=disk,werror=stop,rerror=stop -device virtio-scsi-pci,id=bus1,addr=0x3 -device scsi-hd,bus=bus1.0,drive=drive-virtio-disk0,id=virtio-scsi-pci0 -monitor stdio -boot c -usb -device usb-tablet,id=input1,bus=usb.0 -vnc :12 -netdev tap,id=hostnet0,vhost=on,script=/etc/qemu-ifup -device virtio-net-pci,netdev=hostnet0,id=net0,mac=44:37:E6:97:58:88,addr=0x4 -serial unix:/tmp/tty0,server,nowait -device virtio-balloon-pci,bus=pci.0,id=balloon0,addr=0x5 -qmp tcp:0:4444,server,nowait -drive file=/home/RHEL6.3-20120613.2-Server-x86_64-DVD1.iso,if=none,media=cdrom,id=ide1-cd0,format=raw -device ide-drive,bus=ide.1,unit=0,drive=ide1-cd0,id=ide1-cd0-1 -device virtio-serial-pci,id=virtio-serial0,max_ports=16,bus=pci.0,addr=0x6 -chardev socket,id=channel0,path=/tmp/serial-socket-1,server,nowait -device virtserialport,chardev=channel0,name=org.linux-kvm.port.1,bus=virtio-serial0.0,id=port1 -chardev socket,id=channel1,path=/tmp/serial-socket-2,server,nowait -device virtserialport,chardev=channel1,name=org.linux-kvm.port.2,bus=virtio-serial0.0,id=port2 tried this bug on release version: # uname -r 2.6.32-279.el6.x86_64 [root@lo# uname -r 2.6.32-279.el6.x86_64 [root@localhost ~]# rpm -q qemu-kvm qemu-kvm-0.12.1.2-2.295.el6.x86_64 calhost ~]# rpm -q qemu-kvm qemu-kvm-0.12.1.2-2.295.el6.x86_64 steps as same as description results:seem hit the same problem Program received signal SIGPIPE, Broken pipe. 0x00007ffff77514ed in write () from /lib64/libpthread.so.0 (gdb) bt #0 0x00007ffff77514ed in write () from /lib64/libpthread.so.0 #1 0x00007ffff7e67ee9 in ?? () #2 0x00007ffff7e68054 in ?? () #3 0x00007ffff7f2fa18 in ?? () #4 0x00007ffff7e056b7 in ?? () #5 0x00007ffff7e1e921 in ?? () #6 0x00007ffff7dec319 in ?? () #7 0x00007ffff7e0da4a in ?? () #8 0x00007ffff7deecec in main () addinfo:need to tried more time to reproduce You need the debug version of qemu-kvm to get the proper backtrace (in comment 3). From comment 1: > Guest: #echo guest-file > /dev/vport0p2 This isn't right, you have to use 'cat guest-file > /dev/vport0p2' to transfer the file, echo will just transmit the string "guest-file" to the host. Can you check if this reproduces with socat instead of nc as well? Use socat like this: socat unix-connect:/tmp/serial-socket-2 gopen:/tmp/received-file Hi amit This bug can not easy to reproduce.Later time to paste the results. (In reply to comment #4) > You need the debug version of qemu-kvm to get the proper backtrace (in > comment 3). > Reproduce on release version: # uname -r 2.6.32-279.el6.x86_64 [root@lo# uname -r 2.6.32-279.el6.x86_64 [root@localhost ~]# rpm -q qemu-kvm qemu-kvm-0.12.1.2-2.295.el6.x86_64 calhost ~]# rpm -q qemu-kvm qemu-kvm-0.12.1.2-2.295.el6.x86_64 Results: hit same problem Program received signal SIGPIPE, Broken pipe. 0x00007ffff77514ed in write () from /lib64/libpthread.so.0 (gdb) (gdb) bt #0 0x00007ffff77514ed in write () from /lib64/libpthread.so.0 #1 0x00007ffff7e67ee9 in do_send (chr=0x7ffff86d90d0, fd=27, _buf=<value optimized out>, len1=32768) at qemu-char.c:572 #2 send_all (chr=0x7ffff86d90d0, fd=27, _buf=<value optimized out>, len1=32768) at qemu-char.c:627 #3 0x00007ffff7e68054 in tcp_chr_write (chr=0x7ffff86d90d0, buf=<value optimized out>, len=32768) at qemu-char.c:2051 #4 0x00007ffff7f2fa18 in flush_buf (port=<value optimized out>, buf=<value optimized out>, len=<value optimized out>) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/virtio-console.c:43 #5 0x00007ffff7e056b7 in do_flush_queued_data (port=0x7ffff930a430, vq=0x7ffff93081a0, vdev=0x7ffff88d4940) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/virtio-serial-bus.c:193 #6 0x00007ffff7e1e921 in qemu_bh_poll () at async.c:70 #7 0x00007ffff7dec319 in main_loop_wait (timeout=1000) at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:4032 #8 0x00007ffff7e0da4a in kvm_main_loop () at /usr/src/debug/qemu-kvm-0.12.1.2/qemu-kvm.c:2244 #9 0x00007ffff7deecec in main_loop (argc=20, argv=<value optimized out>, envp=<value optimized out>) at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:4202 #10 main (argc=20, argv=<value optimized out>, envp=<value optimized out>) at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:6427 > From comment 1: > > > Guest: #echo guest-file > /dev/vport0p2 > > This isn't right, you have to use 'cat guest-file > /dev/vport0p2' to > transfer the file, echo will just transmit the string "guest-file" to the > host. > > Can you check if this reproduces with socat instead of nc as well? Use > socat like this: > > socat unix-connect:/tmp/serial-socket-2 gopen:/tmp/received-file Steps: 1.Boot guest with virtio-serial-pci . 2.Guest:#cat lang1.txt >/dev/vport0p2 Results:Can not wait me input as follow command in host ,guest abort #Host:#socat unix-connect:/tmp/serial-socket-2 gopen:/tmp/received-file Program received signal SIGPIPE, Broken pipe. 0x00007ffff77434ed in write () from /lib64/libpthread.so.0 (gdb) bt #0 0x00007ffff77434ed in write () from /lib64/libpthread.so.0 #1 0x00007ffff7e623a9 in do_send (chr=0x7ffff86db720, fd=23, _buf=<value optimized out>, len1=32768) at qemu-char.c:572 #2 send_all (chr=0x7ffff86db720, fd=23, _buf=<value optimized out>, len1=32768) at qemu-char.c:627 #3 0x00007ffff7e62514 in tcp_chr_write (chr=0x7ffff86db720, buf=<value optimized out>, len=32768) at qemu-char.c:2045 #4 0x00007ffff7f2b698 in flush_buf (port=<value optimized out>, buf=<value optimized out>, len=<value optimized out>) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/virtio-console.c:43 #5 0x00007ffff7dfb227 in do_flush_queued_data (port=0x7ffff92e8a40, vq=0x7ffff92da760, vdev=0x7ffff88ba800) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/virtio-serial-bus.c:193 #6 0x00007ffff7e16681 in qemu_bh_poll () at async.c:70 #7 0x00007ffff7de1b49 in main_loop_wait (timeout=1000) at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:4017 #8 0x00007ffff7e03b9a in kvm_main_loop () at /usr/src/debug/qemu-kvm-0.12.1.2/qemu-kvm.c:2244 #9 0x00007ffff7de4738 in main_loop (argc=46, argv=<value optimized out>, envp=<value optimized out>) at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:4187 #10 main (argc=46, argv=<value optimized out>, envp=<value optimized out>) at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:6525 my CLI:/usr/libexec/qemu-kvm -M rhel6.4.0 -cpu Penryn -enable-kvm -uuid `uuidgen` -rtc base=localtime,clock=host,driftfix=slew -m 2G -smp 2,sockets=1,cores=2,threads=1 -name rhel6.4-64 -drive file=/home/RHEL6.3.z-64,format=qcow2,if=none,id=drive-ide0-0-0,werror=stop,rerror=stop,cache=none -device virtio-blk-pci,drive=drive-ide0-0-0,id=ide0-0-0 -netdev tap,id=hostnet0,script=/etc/qemu-ifup -device virtio-net-pci,netdev=hostnet0,id=net0,mac=04:17:21:a2:17:00,bus=pci.0,addr=0x6 -monitor stdio -boot c -global PIIX4_PM.disable_s3=0 -global PIIX4_PM.disable_s4=0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 -vnc :11 -device virtio-serial-pci,id=virtio-serial0,max_ports=16,bus=pci.0,addr=0x7 -chardev socket,id=channel0,path=/tmp/serial-socket-1,server,nowait -device virtserialport,chardev=channel0,name=org.linux-kvm.port.1,bus=virtio-serial0.0,id=port1 -chardev socket,id=channel1,path=/tmp/serial-socket-2,server,nowait -device virtserialport,chardev=channel1,name=org.linux-kvm.port.2,bus=virtio-serial0.0,id=port2 addinfo:use large to reproce The steps to reproduce this bug: 1.Boot guest with ... -device virtio-serial-pci,id=virtio-serial0,max_ports=16,bus=pci.0,addr=0x7 -chardev socket,id=channel0,path=/tmp/serial-socket-1,server,nowait -device virtserialport,chardev=channel0,name=org.linux-kvm.port.1,bus=virtio-serial0.0,id=port1 -chardev socket,id=channel1,path=/tmp/serial-socket-2,server,nowait -device virtserialport,chardev=channel1,name=org.linux-kvm.port.2,bus=virtio-serial0.0,id=port2.... 2.Guest write with port2 #dd a big file in guest: #dd if=/dev/zero of=guest-file bs=1M count=2048 or prepare a large file(more than 700M) : #hexdump -C /dev/sda2 > lang1.txt 3.Prepare receive data Guest: #cat lang1.txt > /dev/vport0p2 Host: #socat unix-connect:/tmp/serial-socket-2 gopen:/tmp/received-file Results: Guest abort . Test on latest version,hit the problem
Host:
# uname -r
2.6.32-371.el6.x86_64
# rpm -q qemu-kvm
qemu-kvm-0.12.1.2-2.361.el6.x86_64
Guest:
2.6.32-371.el6.x86_64
Steps
1.Boot guest with serial device
2.Transfer data from guest to host
In guest
#hexdump -c /dev/vda >lant.txt(258M)
#cat lant.txt>/dev/vport0p1
3.In host
#nc -U /tmp/serial-socket-1
4.When during transfer data from guest to host,"Ctrl+c" to quit host process(#nc -U /tmp/serial-socket-1)
Results:
Program received signal SIGPIPE, Broken pipe.
0x00007ffff773f4ed in write () from /lib64/libpthread.so.0
(gdb) bt
#0 0x00007ffff773f4ed in write () from /lib64/libpthread.so.0
#1 0x00007ffff7e61219 in do_send (chr=0x7ffff86dda70, fd=39, _buf=<value optimized out>, len1=16768)
at /usr/src/debug/qemu-kvm-0.12.1.2/qemu-char.c:572
#2 send_all (chr=0x7ffff86dda70, fd=39, _buf=<value optimized out>, len1=16768) at /usr/src/debug/qemu-kvm-0.12.1.2/qemu-char.c:627
#3 0x00007ffff7e61384 in tcp_chr_write (chr=0x7ffff86dda70, buf=<value optimized out>, len=16768)
at /usr/src/debug/qemu-kvm-0.12.1.2/qemu-char.c:2045
#4 0x00007ffff7f2ab88 in flush_buf (port=<value optimized out>, buf=<value optimized out>, len=<value optimized out>)
at /usr/src/debug/qemu-kvm-0.12.1.2/hw/virtio-console.c:43
#5 0x00007ffff7df81b7 in do_flush_queued_data (port=0x7ffff92ea690, vq=0x7ffff92e8400, vdev=0x7ffff88db190)
at /usr/src/debug/qemu-kvm-0.12.1.2/hw/virtio-serial-bus.c:193
#6 0x00007ffff7e15301 in qemu_bh_poll () at /usr/src/debug/qemu-kvm-0.12.1.2/async.c:70
#7 0x00007ffff7dde509 in main_loop_wait (timeout=1000) at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:4017
#8 0x00007ffff7e00b8a in kvm_main_loop () at /usr/src/debug/qemu-kvm-0.12.1.2/qemu-kvm.c:2244
#9 0x00007ffff7de1128 in main_loop (argc=61, argv=<value optimized out>, envp=<value optimized out>)
at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:4187
#10 main (argc=61, argv=<value optimized out>, envp=<value optimized out>) at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:6530
It's mentioned in bug 909059 that this is reproduced with latest qemu version with the new chardev patches. Please confirm this happens with latest qemu. Also, please mention what happens with upstream qemu. It was mentioned RHEL6 qemu receives SIGPIPE but upstream qemu freezes for the same steps. Please confirm. (In reply to Amit Shah from comment #10) > It's mentioned in bug 909059 that this is reproduced with latest qemu > version with the new chardev patches. Please confirm this happens with > latest qemu. > Same steps as comment9,test on latest version: Host: # uname -r 2.6.32-380.el6.x86_64 # rpm -q qemu-kvm qemu-kvm-0.12.1.2-2.369.el6.x86_64 Guest: Results: Program received signal SIGPIPE, Broken pipe. 0x00007ffff773e4ed in write () from /lib64/libpthread.so.0 (gdb) bt #0 0x00007ffff773e4ed in write () from /lib64/libpthread.so.0 #1 0x00007ffff74b9661 in ?? () from /lib64/libglib-2.0.so.0 #2 0x00007ffff74799b7 in g_io_channel_write_chars () from /lib64/libglib-2.0.so.0 #3 0x00007ffff7e5f95a in io_channel_send (fd=0x7ffff88d95e0, buf=0x7fffd7e57d00, len=768) at /usr/src/debug/qemu-kvm-0.12.1.2/qemu-char.c:726 #4 0x00007ffff7f2ab24 in flush_buf (port=<value optimized out>, buf=0x7fffd7e57d00 " 1\n087f8f0 1 0 1 b 9 0 t b l k _ a d d\n087f900 _ t r a c e _ r q _ r e q u e u\n087f910 e \\n f f f f f f f f 8 "..., len=768) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/virtio-console.c:58 #5 0x00007ffff7df777c in do_flush_queued_data (port=0x7ffff92d0520, vq=0x7ffff92c2240, vdev=0x7ffff92b1d20) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/virtio-serial-bus.c:193 #6 0x00007ffff7e14f41 in qemu_bh_poll () at /usr/src/debug/qemu-kvm-0.12.1.2/async.c:70 #7 0x00007ffff7dde079 in main_loop_wait (timeout=1000) at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:4064 #8 0x00007ffff7e007ca in kvm_main_loop () at /usr/src/debug/qemu-kvm-0.12.1.2/qemu-kvm.c:2244 #9 0x00007ffff7de0ce8 in main_loop (argc=45, argv=<value optimized out>, envp=<value optimized out>) at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:4234 #10 main (argc=45, argv=<value optimized out>, envp=<value optimized out>) at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:6590 (gdb) > Also, please mention what happens with upstream qemu. > > It was mentioned RHEL6 qemu receives SIGPIPE but upstream qemu freezes for > the same steps. Please confirm. Test the same steps use qemu upstream Results:The detail see attachment "Program received signal SIGPIPE" Created attachment 750770 [details]
qemu upstream test
So both RHEL6.5 and upstream receive SIGPIPE. Thanks for confirming. Backtrace is same as bug 808295, https://bugzilla.redhat.com/show_bug.cgi?id=808295#c39 and https://bugzilla.redhat.com/show_bug.cgi?id=808295#c40 There were bugs where SIGPIPE was being received because qemu was invoked with gdb. Can you confirm if this is the case? And if so, please confirm also that w/o gdb, qemu runs fine? When using with gdb, the gdb commands, 'handle SIGPIPE nostop' and 'handle SIGPIPE noprint' help. (In reply to Amit Shah from comment #21) > There were bugs where SIGPIPE was being received because qemu was invoked > with gdb. Can you confirm if this is the case? And if so, please confirm > also that w/o gdb, qemu runs fine? > > When using with gdb, the gdb commands, > > 'handle SIGPIPE nostop' and > 'handle SIGPIPE noprint' > > help. Hi, Min As flang is taking leave, could you help have a look about Amit's question as you are the feature owner? Thanks, Qunfang I tried the bug but I still could not reproduce the issue. host, kernel-2.6.32-470.el6.x86_64 qemu-kvm-rhev-0.12.1.2-2.427.el6.x86_64 Guest, kernel-2.6.32-431.el6.x86_64 Will try more times later.Thanks. Thanks, will close for now, then. |
Description of problem: guest abort when transfer file with virtio-serial from guest to host, transfer file from guest to host ,not hit the probelm Version-Release number of selected component (if applicable): host: # uname -r 2.6.32-344.el6.x86_64 [root@amd-2427-32-2 ~]# rpm -q qemu-kvm qemu-kvm-0.12.1.2-2.337.el6.x86_64 guest :rhel6.3.z kernel-2.6.32-279.19.1.el6 How reproducible: only hit once Steps to Reproduce: 1.boot guest virtio-serial-pci and two ports .... -device virtio-serial-pci,id=virtio-serial0,max_ports=16,bus=pci.0,addr=0x6 -chardev socket,id=channel0,path=/tmp/serial-socket-1,server,nowait -device virtserialport,chardev=channel0,name=org.linux-kvm.port.1,bus=virtio-serial0.0,id=port1 -chardev socket,id=channel1,path=/tmp/serial-socket-2,server,nowait -device virtserialport,chardev=channel1,name=org.linux-kvm.port.2,bus=virtio-serial0.0,id=port2... 2. Guest read with port1--->sucess dd a big file in host: #dd if=/dev/zero of=host-file bs=1M count=500 Host: # cat host-file | nc -U /tmp/serial-socket-1 Guest: #cat /dev/vport0p1 > received-file; 3.Guest write with port2-->abort dd a big file in guest: #dd if=/dev/zero of=guest-file bs=1M count=2048 Host: # nc -U /tmp/serial-socket-2 > received-file Guest: #echo guest-file > /dev/vport0p2 Actual results: guest abort Program received signal SIGPIPE, Broken pipe. 0x00007ffff77434ed in write () from /lib64/libpthread.so.0 (gdb) bt #0 0x00007ffff77434ed in write () from /lib64/libpthread.so.0 #1 0x00007ffff7e623a9 in do_send (chr=0x7ffff86dc650, fd=35, _buf=<value optimized out>, len1=11) at qemu-char.c:572 #2 send_all (chr=0x7ffff86dc650, fd=35, _buf=<value optimized out>, len1=11) at qemu-char.c:627 #3 0x00007ffff7e62514 in tcp_chr_write (chr=0x7ffff86dc650, buf=<value optimized out>, len=11) at qemu-char.c:2045 #4 0x00007ffff7f2b698 in flush_buf (port=<value optimized out>, buf=<value optimized out>, len=<value optimized out>) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/virtio-console.c:43 #5 0x00007ffff7dfb227 in do_flush_queued_data (port=0x7ffff9327a40, vq=0x7ffff9319760, vdev=0x7ffff88dd7d0) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/virtio-serial-bus.c:193 #6 0x00007ffff7e16681 in qemu_bh_poll () at async.c:70 #7 0x00007ffff7de1b49 in main_loop_wait (timeout=1000) at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:4017 #8 0x00007ffff7e03b9a in kvm_main_loop () at /usr/src/debug/qemu-kvm-0.12.1.2/qemu-kvm.c:2244 #9 0x00007ffff7de4738 in main_loop (argc=55, argv=<value optimized out>, envp=<value optimized out>) at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:4187 #10 main (argc=55, argv=<value optimized out>, envp=<value optimized out>) at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:6525 Expected results: guest can be transfer successfully Additional info: