Bug 1478227 - [NBD] qemu-kvm hit Segmentation fault if guest is writing to the NBD data disk and meanwhile unexport this data disk
[NBD] qemu-kvm hit Segmentation fault if guest is writing to the NBD data dis...
Status: NEW
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: qemu-kvm (Show other bugs)
7.4-Alt
All Linux
medium Severity high
: rc
: ---
Assigned To: Ademar Reis
yilzhang
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2017-08-03 23:09 EDT by yilzhang
Modified: 2017-08-05 02:02 EDT (History)
6 users (show)

See Also:
Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description yilzhang 2017-08-03 23:09:52 EDT
Description of problem:
When guest is writing to the NBD data disk, unexport this NBD data disk on NBD server side, after that, qemu-kvm process will abort with Segmentation fault.

Version-Release number of selected component (if applicable):
host: 4.11.0-19.el7a.ppc64le
      qemu-kvm-2.9.0-19.el7a.ppc64le
      SLOF-20170303-4.git66d250e.el7.noarch
guest kernel: 4.11.0-14.el7a.ppc64le

How reproducible: 100%


Steps to Reproduce:
1. Create disk image on NBD server
# qemu-img create -f qcow2  -o preallocation=full   nbd_dataimage_0.qcow2  4G
2. Export image file on NBD server side
# qemu-nbd -f raw  /home/yilzhang/nbd_dataimage_0.qcow2  -p 9001 -t &
3. Boot up guest on NBD client, using the above NBD disk image as one data disk:
/usr/libexec/qemu-kvm \
-name yilzhang_virt8_guest \
 -smp 8,sockets=2,cores=4,threads=1 -m 8192 \
-serial unix:/tmp/nbd-serial.log,server,nowait \
-nodefaults \
 -rtc base=localtime,clock=host \
 -boot menu=on \
 -monitor stdio \
 -vnc :88 \
 -qmp tcp:0:9990,server,nowait \
\
-device pci-bridge,id=bridge1,chassis_nr=1,bus=pci.0 \
 -device virtio-scsi-pci,bus=bridge1,addr=0x1,id=scsi0 \
-drive file=/home/yilzhang/rhel7.4-alt.qcow2,if=none,cache=none,id=drive_sysdisk,snapshot=off,aio=native,format=qcow2,werror=stop,rerror=stop \
-device scsi-hd,drive=drive_sysdisk,bus=scsi0.0,id=sysdisk,bootindex=0 \
\
-drive file=nbd://10.0.1.20:9001,if=none,cache=none,id=drive_datadisk1,aio=native,format=qcow2,werror=stop,rerror=stop \
-device virtio-blk-pci,drive=drive_datadisk1,bus=bridge1,addr=0x2,id=datadisk1 \
 -netdev tap,id=net0,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown,vhost=on \
 -device virtio-net-pci,netdev=net0,id=nic0,mac=52:54:00:c3:e7:84 \

4. Start QMP on host: # telnet localhost 9990
                        {"execute": "qmp_capabilities"}
5. Login guest, and write data to the above data disk exported from NBD server
   [guest]# dd if=/dev/zero  of=/dev/vda  bs=1M count=2000 oflag=sync
6. During "dd" is still ongoing, unexport the NBD disk image
[NBD server]# kill -9 6746
[5]+  Killed                  qemu-nbd -f raw /home/yilzhang/nbd_dataimage_0.qcow2 -p 9001 -t
7. QMP emits "BLOCK_IO_ERROR" event:
{"timestamp": {"seconds": 1494090476, "microseconds": 553531}, "event": "BLOCK_IO_ERROR", "data": {"device": "drive_datadisk1", "nospace": false, "__com.redhat_reason": "eio", "node-name": "#block349", "reason": "Input/output error", "operation": "write", "action": "stop"}}


Actual results:
After a short while, qemu-kvm aborted with Segmentation fault

Expected results:
qemu-kvm should not abort abnormally


Additional info:
1. Power8+qemu-kvm-rhev-2.9.0-14.el7.ppc64le and x86 platform also have this issue
2. gdb  /usr/libexec/qemu-kvm  core.9638
warning: exec file is newer than core file.
[New LWP 9638]
[New LWP 9680]
[New LWP 9682]
[New LWP 9681]
[New LWP 9683]
[New LWP 9685]
[New LWP 9684]
[New LWP 9689]
[New LWP 9639]
[New LWP 9687]
[New LWP 9686]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Core was generated by `/usr/libexec/qemu-kvm -name yilzhang_virt8_guest -smp 8,sockets=2,cores=4,threa'.
Program terminated with signal 11, Segmentation fault.
#0  0x000000002cd67bb4 in aio_co_schedule ()
Missing separate debuginfos, use: debuginfo-install qemu-kvm-2.9.0-19.el7a.ppc64le
(gdb) bt
#0  0x000000002cd67bb4 in aio_co_schedule ()
#1  0x000000002ccd1c9c in nbd_client_attach_aio_context ()
#2  0x000000002cccfce8 in nbd_attach_aio_context ()
#3  0x000000002cc72ac0 in bdrv_attach_aio_context ()
#4  0x000000002cc72a8c in bdrv_attach_aio_context ()
#5  0x000000002cc72c38 in bdrv_set_aio_context ()
#6  0x000000002ccbaab4 in blk_set_aio_context ()
#7  0x000000002c9ff9d0 in virtio_blk_data_plane_stop ()
#8  0x000000002cbf7020 in virtio_bus_stop_ioeventfd ()
#9  0x000000002cbf2598 in virtio_pci_vmstate_change ()
#10 0x000000002ca2e91c in virtio_vmstate_change ()
#11 0x000000002cb5a6b4 in vm_state_notify ()
#12 0x000000002c9bd9c0 in vm_stop ()
#13 0x000000002c95af10 in main ()
Comment 2 yilzhang 2017-08-04 03:53:10 EDT
Power8+qemu-kvm-rhev-2.9.0-14.el7.ppc64le and x86+qemu-kvm-rhev-2.9.0-19.el7a.x86_64   also have this issue

Note You need to log in before you can comment on or make changes to this bug.