Bug 1658974 - [data plane] Qemu core dump for 'qemu_mutex_unlock_impl: Operation not permitted' if hot unplug then replug the frontend virtio scsi disk
Summary: [data plane] Qemu core dump for 'qemu_mutex_unlock_impl: Operation not permit...
Keywords:
Status: CLOSED DUPLICATE of bug 1656276
Alias: None
Product: Red Hat Enterprise Linux Advanced Virtualization
Classification: Red Hat
Component: qemu-kvm
Version: 8.0
Hardware: Unspecified
OS: Unspecified
medium
high
Target Milestone: pre-dev-freeze
: ---
Assignee: Ademar Reis
QA Contact: Xueqiang Wei
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-12-13 10:03 UTC by Gu Nini
Modified: 2019-02-22 22:11 UTC (History)
10 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-12-19 07:16:22 UTC
Type: Bug
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
gdb_debug_info_all_threads-12132018 (35.46 KB, text/plain)
2018-12-13 10:03 UTC, Gu Nini
no flags Details
gdb_debug_info_all_threads-x86-12142018 (29.62 KB, text/plain)
2018-12-14 03:11 UTC, Gu Nini
no flags Details

Description Gu Nini 2018-12-13 10:03:36 UTC
Created attachment 1513973 [details]
gdb_debug_info_all_threads-12132018

Description of problem:
After hot plug the backend with 'blockdev-add', then hot plug the frontend with 'device_add'(virtio-scsi/data plane used); hot unplug the frontend with 'device_del', and then hot plug it again with 'device_add', it's found the guest core dump:
  
QEMU 3.0.95 monitor - type 'help' for more information
(qemu) qemu: qemu_mutex_unlock_impl: Operation not permitted
./vm1c.sh: line 23: 55547 Aborted                 (core dumped) /usr/libexec/qemu-kvm -name 'avocado-vt-vm1' -sandbox off -machine pseries,max-cpu-compat=power8 -object iothread,id=iothread1 -chardev socket,id=qmp_id_qmpmonitor1,path=/var/tmp/avocado1,server,nowait -mon chardev=qmp_id_qmpmonitor1,mode=control -device nec-usb-xhci,id=usb1,bus=pci.0,addr=0x4 -device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pci.0,addr=0x6,iothread=iothread1 -blockdev node-name=disk0,file.driver=host_device,driver=qcow2,file.filename=/dev/vg1/lv1,cache.no-flush=off,cache.direct=off -device scsi-hd,drive=disk0,id=image0,werror=stop,rerror=stop,bootindex=0,bus=virtio_scsi_pci0.0 -device virtio-net-pci,mac=9a:78:79:7a:7b:7a,id=id8e5D72,vectors=4,netdev=idrYUYaH,bus=pci.0,addr=0x3 -netdev tap,id=idrYUYaH,vhost=on -m 1024 -smp 2,maxcpus=2,cores=2,threads=1,sockets=1 -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 -cpu host -vnc :20 -rtc base=localtime,clock=host,driftfix=slew -boot menu=off,strict=off,order=cdn,once=c -enable-kvm -monitor stdio
[root@ibm-p9b-21 ngu]# [root@ibm-p9b-21 ngu]# [root@ibm-p9b-21 ngu]# 


Version-Release number of selected component (if applicable):
[root@ibm-p9b-21 ~]# uname -r
4.18.0-50.el8.ppc64le
[root@ibm-p9b-21 ~]# rpm -qa|grep qemu
qemu-img-debuginfo-3.1.0-0.el8.rc5.ppc64le
qemu-kvm-common-3.1.0-0.el8.rc5.ppc64le
qemu-kvm-3.1.0-0.el8.rc5.ppc64le
qemu-kvm-debuginfo-3.1.0-0.el8.rc5.ppc64le
qemu-kvm-common-debuginfo-3.1.0-0.el8.rc5.ppc64le
qemu-kvm-debugsource-3.1.0-0.el8.rc5.ppc64le
qemu-kvm-block-ssh-debuginfo-3.1.0-0.el8.rc5.ppc64le
qemu-kvm-block-rbd-debuginfo-3.1.0-0.el8.rc5.ppc64le
qemu-kvm-core-3.1.0-0.el8.rc5.ppc64le
qemu-kvm-block-iscsi-debuginfo-3.1.0-0.el8.rc5.ppc64le
qemu-kvm-block-rbd-3.1.0-0.el8.rc5.ppc64le
qemu-kvm-block-curl-3.1.0-0.el8.rc5.ppc64le
qemu-kvm-block-ssh-3.1.0-0.el8.rc5.ppc64le
qemu-kvm-block-iscsi-3.1.0-0.el8.rc5.ppc64le
qemu-guest-agent-3.1.0-0.el8.rc5.ppc64le
qemu-kvm-block-curl-debuginfo-3.1.0-0.el8.rc5.ppc64le
qemu-kvm-core-debuginfo-3.1.0-0.el8.rc5.ppc64le
qemu-guest-agent-debuginfo-3.1.0-0.el8.rc5.ppc64le
ipxe-roms-qemu-20170710-6.git0600d3ae.el8.noarch
qemu-img-3.1.0-0.el8.rc5.ppc64le

How reproducible:
100%

Steps to Reproduce:
1. Boot up a guest using virtio-scsi/data-plane on the system disk:

/usr/libexec/qemu-kvm \
    -name 'avocado-vt-vm1'  \
    -sandbox off  \
    -machine pseries,max-cpu-compat=power8 \
    -object iothread,id=iothread1 \
    -chardev socket,id=qmp_id_qmpmonitor1,path=/var/tmp/avocado1,server,nowait \
    -mon chardev=qmp_id_qmpmonitor1,mode=control  \
    -device nec-usb-xhci,id=usb1,bus=pci.0,addr=0x4\
    -device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pci.0,addr=0x6,iothread=iothread1 \
    -blockdev node-name=disk0,file.driver=host_device,driver=qcow2,file.filename=/dev/vg1/lv1,cache.no-flush=off,cache.direct=off \
    -device scsi-hd,drive=disk0,id=image0,werror=stop,rerror=stop,bootindex=0,bus=virtio_scsi_pci0.0 \
    -device virtio-net-pci,mac=9a:78:79:7a:7b:7a,id=id8e5D72,vectors=4,netdev=idrYUYaH,bus=pci.0,addr=0x3  \
    -netdev tap,id=idrYUYaH,vhost=on \
    -m 1024  \
    -smp 2,maxcpus=2,cores=2,threads=1,sockets=1  \
    -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1  \
    -cpu host \
    -vnc :20  \
    -rtc base=localtime,clock=host,driftfix=slew  \
    -boot menu=off,strict=off,order=cdn,once=c \
    -enable-kvm \
    -monitor stdio

2. Create an image with qemu-img:
# qemu-img create -f qcow2 /dev/vg1/lv2 20G

3. Hot plug the image with 'blockdev-add' and 'device_add':
# {"execute": "blockdev-add","arguments": {"node-name": "disk2","driver": "raw", "file":    {"driver":"host_device", "filename": "/dev/vg1/lv2"}}}

# {"execute":"device_add","arguments":{"driver":"scsi-hd","drive":"disk2","id":"data-disk2"}}

4. Hot unplug the frontend, then replug it:
# {"execute":"device_del","arguments":{"id":data-disk2"}}
# {"execute":"device_add","arguments":{"driver":"scsi-hd","drive":"disk2","id":"data-disk2"}}


Actual results:
Qemu core dump as that in the description part

Expected results:
No core dump

Additional info:
1. Not met the problem when using virtio-blk and data plane
2. When data plane disabled, no the problem
3. I guess the bug should be platform unrelated, will check it tomorrow once I have a usable x86 host.

Comment 1 Gu Nini 2018-12-14 03:10:31 UTC
Reproduced the bug on x86 platform:
[root@lenovo-sr630-02 coredump]# uname -r
4.18.0-50.el8.x86_64
[root@lenovo-sr630-02 coredump]# rpm -qa|grep qemu
qemu-kvm-block-gluster-3.1.0-0.el8.rc5.x86_64
ipxe-roms-qemu-20170710-6.git0600d3ae.el8.noarch
qemu-kvm-block-gluster-debuginfo-3.1.0-0.el8.rc5.x86_64
qemu-kvm-3.1.0-0.el8.rc5.x86_64
qemu-kvm-debuginfo-3.1.0-0.el8.rc5.x86_64
qemu-kvm-block-ssh-debuginfo-3.1.0-0.el8.rc5.x86_64
qemu-kvm-debugsource-3.1.0-0.el8.rc5.x86_64
qemu-img-debuginfo-3.1.0-0.el8.rc5.x86_64
qemu-kvm-block-rbd-debuginfo-3.1.0-0.el8.rc5.x86_64
qemu-guest-agent-3.1.0-0.el8.rc5.x86_64
qemu-kvm-common-3.1.0-0.el8.rc5.x86_64
qemu-kvm-block-rbd-3.1.0-0.el8.rc5.x86_64
qemu-kvm-block-curl-3.1.0-0.el8.rc5.x86_64
qemu-guest-agent-debuginfo-3.1.0-0.el8.rc5.x86_64
qemu-kvm-block-iscsi-debuginfo-3.1.0-0.el8.rc5.x86_64
qemu-kvm-core-debuginfo-3.1.0-0.el8.rc5.x86_64
qemu-kvm-block-ssh-3.1.0-0.el8.rc5.x86_64
qemu-img-3.1.0-0.el8.rc5.x86_64
qemu-kvm-core-3.1.0-0.el8.rc5.x86_64
qemu-kvm-common-debuginfo-3.1.0-0.el8.rc5.x86_64
qemu-kvm-block-iscsi-3.1.0-0.el8.rc5.x86_64
qemu-kvm-block-curl-debuginfo-3.1.0-0.el8.rc5.x86_64

And in the bug description part, I have used the host_device(lv) to do the test, but now it's found the bug has no relation with it, I can reproduce it more easily:
1. Boot up a guest with a system disk and a data disk:
    -blockdev node-name=disk0,file.driver=file,driver=qcow2,file.filename=/home/guest.qcow2,cache.no-flush=off,cache.direct=off \
    -device scsi-hd,drive=disk0,id=image0,werror=stop,rerror=stop,bootindex=0,bus=virtio_scsi_pci0.0 \
    -blockdev node-name=disk1,file.driver=file,driver=qcow2,file.filename=/home/hd1,cache.no-flush=off,cache.direct=off \
    -device scsi-hd,drive=disk1,id=image1,werror=stop,rerror=stop,bootindex=1 \

2. Hot unplug the frontend of data disk, then hot plug it again, the bug is reproduced till now:
{"execute":"device_del","arguments":{"id":"image1"}}
{"execute":"device_add","arguments":{"driver":"scsi-hd","drive":"disk1","id":"image1"}}
Ncat: Connection reset by peer.

Comment 2 Gu Nini 2018-12-14 03:11:30 UTC
Created attachment 1514284 [details]
gdb_debug_info_all_threads-x86-12142018

Comment 3 CongLi 2018-12-19 07:16:22 UTC

*** This bug has been marked as a duplicate of bug 1656276 ***


Note You need to log in before you can comment on or make changes to this bug.