Bug 1607768
Summary: | qemu aborted when start guest with a big iothreads | ||||||
---|---|---|---|---|---|---|---|
Product: | Red Hat Enterprise Linux 7 | Reporter: | yafu <yafu> | ||||
Component: | qemu-kvm-rhev | Assignee: | Stefan Hajnoczi <stefanha> | ||||
Status: | CLOSED ERRATA | QA Contact: | Xueqiang Wei <xuwei> | ||||
Severity: | medium | Docs Contact: | |||||
Priority: | medium | ||||||
Version: | 7.6 | CC: | aliang, chayang, coli, ddepaula, juzhang, mrezanin, ngu, stefanha, virt-maint, yafu | ||||
Target Milestone: | rc | ||||||
Target Release: | --- | ||||||
Hardware: | Unspecified | ||||||
OS: | Unspecified | ||||||
Whiteboard: | |||||||
Fixed In Version: | qemu-kvm-rhev-2.12.0-20.el7 | Doc Type: | If docs needed, set a value | ||||
Doc Text: | Story Points: | --- | |||||
Clone Of: | |||||||
: | 1687541 (view as bug list) | Environment: | |||||
Last Closed: | 2019-08-22 09:18:48 UTC | Type: | Bug | ||||
Regression: | --- | Mount Type: | --- | ||||
Documentation: | --- | CRM: | |||||
Verified Versions: | Category: | --- | |||||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
Cloudforms Team: | --- | Target Upstream Version: | |||||
Embargoed: | |||||||
Bug Depends On: | |||||||
Bug Blocks: | 1649160, 1687541 | ||||||
Attachments: |
|
Description
yafu
2018-07-24 08:47:09 UTC
Hi, yafu I tested with 1000 iothreads, don't hit your issue. kernel version:3.10.0-918.el7.x86_64 qemu-kvm-rhev version:qemu-kvm-rhev-2.12.0-8.el7.x86_64 Need to confirm info bellow: 1. your testing scenarios: iommu + iothreads? or only big iothreads? 2. if your testing scenarios is iommu, do you configure iommu in both host and guest cmdline? (In reply to aihua liang from comment #3) > Hi, yafu > > I tested with 1000 iothreads, don't hit your issue. > kernel version:3.10.0-918.el7.x86_64 > qemu-kvm-rhev version:qemu-kvm-rhev-2.12.0-8.el7.x86_64 > > Need to confirm info bellow: > 1. your testing scenarios: iommu + iothreads? or only big iothreads? > 2. if your testing scenarios is iommu, do you configure iommu in both > host and guest cmdline? Iommu is not enabled. The issue can be reproduced with 10000 iothreads. Yes, can reproduce it with 10000 iothreads. Test env: kernel version: 3.10.0-926.el7.x86_64 qemu-kvm-rhev version:qemu-kvm-rhev-2.12.0-8.el7.x86_64 Test steps: 1.Start rhel7.6 guest with 10000 iothreads, cmds as bellow: /usr/libexec/qemu-kvm \ -name 'avocado-vt-vm1' \ -sandbox off \ -machine pc \ -nodefaults \ -device VGA,bus=pci.0,addr=0x2 \ -chardev socket,id=qmp_id_qmpmonitor1,path=/var/tmp/monitor-qmpmonitor1-20180730-052157-8R2ZuIGd,server,nowait \ -mon chardev=qmp_id_qmpmonitor1,mode=control \ -chardev socket,id=qmp_id_catch_monitor,path=/var/tmp/monitor-catch_monitor-20180730-052157-8R2ZuIGd,server,nowait \ -mon chardev=qmp_id_catch_monitor,mode=control \ -device pvpanic,ioport=0x505,id=idPX2yVp \ -chardev socket,id=serial_id_serial0,path=/var/tmp/serial-serial0-20180730-052157-8R2ZuIGd,server,nowait \ -device isa-serial,chardev=serial_id_serial0 \ -chardev socket,id=seabioslog_id_20180730-052157-8R2ZuIGd,path=/var/tmp/seabios-20180730-052157-8R2ZuIGd,server,nowait \ -device isa-debugcon,chardev=seabioslog_id_20180730-052157-8R2ZuIGd,iobase=0x402 \ -device nec-usb-xhci,id=usb1,bus=pci.0,addr=0x3 \ -drive id=drive_image1,if=none,snapshot=off,aio=threads,cache=none,format=qcow2,file=/home/rhel76-64-virtio.qcow2 \ -device virtio-blk-pci,id=image1,drive=drive_image1,bus=pci.0 \ device virtio-net-pci,mac=9a:be:bf:c0:c1:c2,id=idZKQtxs,vectors=4,netdev=idvsIEDa,bus=pci.0 \ -netdev tap,id=idvsIEDa,vhost=on \ -m 4096 \ -smp 2,maxcpus=2,cores=1,threads=1,sockets=2 \ -cpu 'Penryn' \ -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 \ -vnc :0 \ -rtc base=utc,clock=host,driftfix=slew \ -boot menu=on,strict=off,order=cdn \ -no-shutdown \ -enable-kvm \ -monitor stdio \ -object iothread,id=iothread0 \ -object iothread,id=iothread1 \ -object iothread,id=iothread2 \ -object iothread,id=iothread3 \ -object iothread,id=iothread4 \ -object iothread,id=iothread5 \ .... -object iothread,id=iothread9999 \ Test reuslt: Qemu core dump: [root@intel-3323-24-2 home]# sh ta.txt Failed to create epoll instance: Too many open filesqemu-kvm: util/qemu-thread-posix.c:131: qemu_cond_destroy: Assertion `cond->initialized' failed. ta.txt: line 10031: 10845 Aborted (core dumped) /usr/libexec/qemu-kvm -name 'avocado-vt-vm1' -sandbox off .... Attachment it the coredump file. Created attachment 1472254 [details]
coredump file
The minimum iothreads loaded that will cause qemu core dump is 4085. So when you start guest with 4085 iothreads, qemu will core dump. I will backport the following fix: commit 14a2d11825ddc37d6547a80704ae6450e9e376c7 Author: Marc-André Lureau <marcandre.lureau> Date: Tue Aug 21 12:07:16 2018 +0200 iothread: fix crash with invalid properties Fix included in qemu-kvm-rhev-2.12.0-20.el7 Tested on qemu-kvm-rhev-2.12.0-20.el7, not hit this issue. So set bug status to VERIFIED. Versions: Host: kernel-3.10.0-957.el7.x86_64 qemu-kvm-rhev-2.12.0-20.el7 Guest: kernel-3.10.0-957.el7.x86_64 # ulimit -n 102400 start guest with 10000 iothreads, it works well. # sh bug_1607768.sh QEMU 2.12.0 monitor - type 'help' for more information (qemu) c (qemu) system_reset (qemu) system_powerdown *** Bug 1622963 has been marked as a duplicate of this bug. *** * my last comment is related to the slow train. There will be a fast train update for 8.0.1. The branch is ready anyway. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2019:2553 |