Note: This bug is displayed in read-only format because
the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Description of problem:
Qemu core dump when block jobs on nbd node were blocked by firewall or server down
Version-Release number of selected component (if applicable):
kernel version:5.14.0-30.el9.x86_64
qemu-kvm version:qemu-kvm-6.2.0-1.el9
How reproducible:
100% by auto
Steps to Reproduce:
1.Create data disk image.
#qemu-img create -f qcow2 /root/avocado/data/avocado-vt/data.qcow2 2G
2.Expose data image
#/usr/bin/qemu-nbd -f qcow2 -t -p 10810 --fork --pid-file /tmp/avocado_0e1jqoko/data_nbd_server-20211217-075948-p12l.pid /root/avocado/data/avocado-vt/data.qcow2
3.Start guest with the exposed data disk:
/usr/libexec/qemu-kvm \
-S \
-name 'avocado-vt-vm1' \
-sandbox on \
-machine q35,memory-backend=mem-machine_mem \
-device pcie-root-port,id=pcie-root-port-0,multifunction=on,bus=pcie.0,addr=0x1,chassis=1 \
-device pcie-pci-bridge,id=pcie-pci-bridge-0,addr=0x0,bus=pcie-root-port-0 \
-nodefaults \
-device VGA,bus=pcie.0,addr=0x2 \
-m 30720 \
-object memory-backend-ram,size=30720M,id=mem-machine_mem \
-smp 10,maxcpus=10,cores=5,threads=1,dies=1,sockets=2 \
-cpu 'Cascadelake-Server-noTSX',+kvm_pv_unhalt \
-chardev socket,wait=off,path=/tmp/avocado_0e1jqoko/monitor-qmpmonitor1-20211217-075949-ZwO9C7pE,server=on,id=qmp_id_qmpmonitor1 \
-mon chardev=qmp_id_qmpmonitor1,mode=control \
-chardev socket,wait=off,path=/tmp/avocado_0e1jqoko/monitor-catch_monitor-20211217-075949-ZwO9C7pE,server=on,id=qmp_id_catch_monitor \
-mon chardev=qmp_id_catch_monitor,mode=control \
-device pvpanic,ioport=0x505,id=id9Na7Yp \
-chardev socket,wait=off,path=/tmp/avocado_0e1jqoko/serial-serial0-20211217-075949-ZwO9C7pE,server=on,id=chardev_serial0 \
-device isa-serial,id=serial0,chardev=chardev_serial0 \
-chardev socket,id=seabioslog_id_20211217-075949-ZwO9C7pE,path=/tmp/avocado_0e1jqoko/seabios-20211217-075949-ZwO9C7pE,server=on,wait=off \
-device isa-debugcon,chardev=seabioslog_id_20211217-075949-ZwO9C7pE,iobase=0x402 \
-device pcie-root-port,id=pcie-root-port-1,port=0x1,addr=0x1.0x1,bus=pcie.0,chassis=2 \
-device qemu-xhci,id=usb1,bus=pcie-root-port-1,addr=0x0 \
-device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 \
-object iothread,id=iothread0 \
-device pcie-root-port,id=pcie-root-port-2,port=0x2,addr=0x1.0x2,bus=pcie.0,chassis=3 \
-device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pcie-root-port-2,addr=0x0,iothread=iothread0 \
-blockdev node-name=file_image1,driver=file,auto-read-only=on,discard=unmap,aio=threads,filename=/home/kvm_autotest_root/images/rhel900-64-virtio-scsi.qcow2,cache.direct=on,cache.no-flush=off \
-blockdev node-name=drive_image1,driver=qcow2,read-only=off,cache.direct=on,cache.no-flush=off,file=file_image1 \
-device scsi-hd,id=image1,drive=drive_image1,write-cache=on \
-blockdev node-name=nbd_nbddata,driver=nbd,auto-read-only=on,discard=unmap,server.type=inet,server.host=dell-per440-09.lab.eng.pek2.redhat.com,server.port=10810,reconnect-delay=30,cache.direct=on,cache.no-flush=off \
-blockdev node-name=drive_nbddata,driver=raw,read-only=off,cache.direct=on,cache.no-flush=off,file=nbd_nbddata \
-device scsi-hd,id=nbddata,drive=drive_nbddata,write-cache=on \
-device pcie-root-port,id=pcie-root-port-3,port=0x3,addr=0x1.0x3,bus=pcie.0,chassis=4 \
-device virtio-net-pci,mac=9a:ed:fc:56:77:68,id=idY9yr3Q,netdev=id1qEUjK,bus=pcie-root-port-3,addr=0x0 \
-netdev tap,id=id1qEUjK,vhost=on,vhostfd=22,fd=19 \
-vnc :0 \
-rtc base=utc,clock=host,driftfix=slew \
-boot menu=off,order=cdn,once=c,strict=off \
-enable-kvm \
-device pcie-root-port,id=pcie_extra_root_port_0,multifunction=on,bus=pcie.0,addr=0x3,chassis=5
4. Continue vm
{'execute': 'cont', 'id': 'CE1lLVef'}
Actual results:
After step4, qemu core dump with info:
[qemu output] qemu-kvm: ../io/channel.c:484: void qio_channel_restart_read(void *): Assertion `qemu_get_current_aio_context() == qemu_coroutine_get_aio_context(co)' failed.
[qemu output] /tmp/aexpect_SwITdppJ/aexpect-y1zy2kcy.sh: line 1: 236656 Aborted (core dumped) MALLOC_PERTURB_=1 /usr/libexec/qemu-kvm -S -name 'avocado-vt-vm1' -sandbox on -machine q35,memory-backend=mem-machine_mem -device pcie-root-port,id=pcie-root-port-0,multifunction=on,bus=pcie.0,addr=0x1,chassis=1 -device pcie-pci-bridge,id=pcie-pci-bridge-0,addr=0x0,bus=pcie-root-port-0 ...
Detailed core dump info:
coredumpctl debug 236656
PID: 236656 (qemu-kvm)
UID: 0 (root)
GID: 0 (root)
Signal: 6 (ABRT)
Timestamp: Fri 2021-12-17 08:00:02 EST (8min ago)
Command Line: /usr/libexec/qemu-kvm -S -name avocado-vt-vm1 -sandbox on -machine q35,memory-backend=mem-machine_mem -device pcie-root-port,id=pcie-root-port-0,multifunction=on,bus=pcie.0,addr=0x1,chassis=1 -device pcie-pci-bridge,id=pcie-pci-bridge-0,addr=0x0,bus=pcie-root-port-0 -nodefaults -device VGA,bus=pcie.0,addr=0x2 -m 30720 -object memory-backend-ram,size=30720M,id=mem-machine_mem -smp 10,maxcpus=10,cores=5,threads=1,dies=1,sockets=2 -cpu Cascadelake-Server-noTSX,+kvm_pv_unhalt -chardev socket,wait=off,path=/tmp/avocado_0e1jqoko/monitor-qmpmonitor1-20211217-075949-ZwO9C7pE,server=on,id=qmp_id_qmpmonitor1 -mon chardev=qmp_id_qmpmonitor1,mode=control -chardev socket,wait=off,path=/tmp/avocado_0e1jqoko/monitor-catch_monitor-20211217-075949-ZwO9C7pE,server=on,id=qmp_id_catch_monitor -mon chardev=qmp_id_catch_monitor,mode=control -device pvpanic,ioport=0x505,id=id9Na7Yp -chardev socket,wait=off,path=/tmp/avocado_0e1jqoko/serial-serial0-20211217-075949-ZwO9C7pE,server=on,id=chardev_serial0 -device isa-serial,id=serial0,chardev=chardev_serial0 -chardev socket,id=seabioslog_id_20211217-075949-ZwO9C7pE,path=/tmp/avocado_0e1jqoko/seabios-20211217-075949-ZwO9C7pE,server=on,wait=off -device isa-debugcon,chardev=seabioslog_id_20211217-075949-ZwO9C7pE,iobase=0x402 -device pcie-root-port,id=pcie-root-port-1,port=0x1,addr=0x1.0x1,bus=pcie.0,chassis=2 -device qemu-xhci,id=usb1,bus=pcie-root-port-1,addr=0x0 -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 -object iothread,id=iothread0 -device pcie-root-port,id=pcie-root-port-2,port=0x2,addr=0x1.0x2,bus=pcie.0,chassis=3 -device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pcie-root-port-2,addr=0x0,iothread=iothread0 -blockdev node-name=file_image1,driver=file,auto-read-only=on,discard=unmap,aio=threads,filename=/home/kvm_autotest_root/images/rhel900-64-virtio-scsi.qcow2,cache.direct=on,cache.no-flush=off -blockdev node-name=drive_image1,driver=qcow2,read-only=off,cache.direct=on,cache.no-flush=off,file=file_image1 -device scsi-hd,id=image1,drive=drive_image1,write-cache=on -blockdev node-name=nbd_nbddata,driver=nbd,auto-read-only=on,discard=unmap,server.type=inet,server.host=dell-per440-09.lab.eng.pek2.redhat.com,server.port=10810,reconnect-delay=30,cache.direct=on,cache.no-flush=off -blockdev node-name=drive_nbddata,driver=raw,read-only=off,cache.direct=on,cache.no-flush=off,file=nbd_nbddata -device scsi-hd,id=nbddata,drive=drive_nbddata,write-cache=on -device pcie-root-port,id=pcie-root-port-3,port=0x3,addr=0x1.0x3,bus=pcie.0,chassis=4 -device virtio-net-pci,mac=9a:ed:fc:56:77:68,id=idY9yr3Q,netdev=id1qEUjK,bus=pcie-root-port-3,addr=0x0 -netdev tap,id=id1qEUjK,vhost=on,vhostfd=22,fd=19 -vnc :0 -rtc base=utc,clock=host,driftfix=slew -boot menu=off,order=cdn,once=c,strict=off -enable-kvm -device pcie-root-port,id=pcie_extra_root_port_0,multifunction=on,bus=pcie.0,addr=0x3,chassis=5
Executable: /usr/libexec/qemu-kvm
Control Group: /user.slice/user-0.slice/session-1.scope
Unit: session-1.scope
Slice: user-0.slice
Session: 1
Owner UID: 0 (root)
Boot ID: 0107346256b84e8ebd9076ae0ca431c0
Machine ID: 1437cd218cca45cbb2059307f6958ac6
Hostname: dell-per440-09.lab.eng.pek2.redhat.com
Storage: /var/lib/systemd/coredump/core.qemu-kvm.0.0107346256b84e8ebd9076ae0ca431c0.236656.1639746002000000.zst (present)
Disk Size: 123.9M
Message: Process 236656 (qemu-kvm) of user 0 dumped core.
Found module linux-vdso.so.1 with build-id: 0417c877078abdfaf5ca0d3d449ce6699e5a520c
Found module libpcre2-8.so.0 with build-id: a5dafbac0243bdc2e3b5ab9a07d61d4b32f86927
Found module liblzma.so.5 with build-id: 66b7fca8b63a61d19c5ded440ee6a8bf05dcafa5
Found module libzstd.so.1 with build-id: 415c738420cd82e182b99fc4c6e6821098e80ce4
Found module libselinux.so.1 with build-id: c1d23d40bcf1fae271256fcbe90a46ff6dbcce20
Found module libkmod.so.2 with build-id: 722d3027106c561018756766be8bed5215ed0f31
Found module libuuid.so.1 with build-id: da656ae81596c151559892375d730bd35f76048c
Found module libudev.so.1 with build-id: 7c275efc762b4b5a63016164ffd5a53962effbb1
Found module libcrypto.so.3 with build-id: e3a618644eec4fb65c7cd059c42592a686191f72
Found module libkeyutils.so.1 with build-id: 83c6539bd0d3140678ba836b8baa1b215efa2632
Found module libkrb5support.so.0 with build-id: 22e9f06a5b088d401df9153948492077b4e198af
Found module libffi.so.8 with build-id: 48e3675db4765a2e42729140922e11a10016f7ab
Found module libpcre.so.1 with build-id: cffb947bcc416dca3cd249cdb0a1c6f614549c30
Found module libnl-route-3.so.200 with build-id: 2f297144194e194e379c1c31dcd245220764a488
Found module libnl-3.so.200 with build-id: 5690fe17968270ad4ee00fdafe15cf0a4db9c7ed
Found module libdaxctl.so.1 with build-id: c46dc4ef727b75a311e76e41940451a90b9f1080
Found module libndctl.so.6 with build-id: 8c52e205e3982b331bb9683f1a35d53bb1c11c50
Found module libresolv.so.2 with build-id: eaebc6c6a15dcc8af04ea904eb33aeb5557fcafd
Found module libcom_err.so.2 with build-id: deb039f83bae82e65be49dd97ac3d900a053e265
Found module libk5crypto.so.3 with build-id: eca094d4bc539db337447b773462de7b592e785b
Found module libkrb5.so.3 with build-id: a3fec09569ad99121e79e2d73b0d446169965d26
Found module libgssapi_krb5.so.2 with build-id: a2daa1c10270547f24b82fcb0034e84853d35344
Found module libcrypt.so.2 with build-id: 6ce4e5eb200e61d07398af52f8bcb316cf8466e0
Found module libgmp.so.10 with build-id: 754b87178e725c166a68353f043ba7638e0723e1
Found module libhogweed.so.6 with build-id: 21c32760f1f50b9d69b43e966b93fb3de8baa076
Found module libnettle.so.8 with build-id: 3eda844d5a5df2443a281158b0d7ca6e81483b59
Found module libtasn1.so.6 with build-id: 3d3a2f6f0d4a70919496afe25e329abd189b7882
Found module libunistring.so.2 with build-id: 15e34cdfafa3547f9c700489b842ceb86f6fb73e
Found module libidn2.so.0 with build-id: 958c50fc94ecb196b24f3619762e7ec3f28a5b40
Found module libp11-kit.so.0 with build-id: b03163bf5b98313c7347e7270627a6b2dcd35060
Found module libstdc++.so.6 with build-id: 66b742a0a51ccd8a67fdf59a31663b0cd28dfcf2
Found module ld-linux-x86-64.so.2 with build-id: 2b7f1f415256e40ab15e5808c73eb7acf3c73ea3
Found module libc.so.6 with build-id: 723a5d3072f3b79f3716880303439e11f7661fc5
Found module libgcc_s.so.1 with build-id: a1a28a65f942690a11041922ba11c16f716565e2
Found module libm.so.6 with build-id: c58412f6a7fffa4068614708cdb5b15f2ba49c0a
Found module libaio.so.1 with build-id: 9b75d21282f8e17ddfa06aff78dae4f8dcce4106
Found module libgmodule-2.0.so.0 with build-id: 952459b04fb217bc3e4a17b902759cf66c89a95d
Found module libglib-2.0.so.0 with build-id: 0006a4eed3e835c7d7812e36e5d720c1a6eb8689
Found module libslirp.so.0 with build-id: fa9d492d6d050c6f5cb906a3f087a8a63e4201e2
Found module libibverbs.so.1 with build-id: 3e19d7349fe9ea4c79debeb8716df7000ab13159
Found module librdmacm.so.1 with build-id: 4f6798f3f4164a4aa4d75d8a890c760e2835c495
Found module libnuma.so.1 with build-id: a0c23354fdb39f51ec9e31aeae27a9f8ee034500
Found module libfdt.so.1 with build-id: 31a56e0009a8824c7a09267c8205034c91cb4095
Found module libseccomp.so.2 with build-id: 5a5b4ee3bb99ee0378bd937fdef7a7b7255d0428
Found module libpmem.so.1 with build-id: 084e72da2642981fe75b446293655ae4a68c0663
Found module libsasl2.so.3 with build-id: 2864d8e1600f3b46016fd62c44269e92c5ec0ca1
Found module libgnutls.so.30 with build-id: b08009ada925278615f5592e982e0776c06e065e
Found module libpng16.so.16 with build-id: 698f67f36c58fa443c6ba891e4a0f106553a0452
Found module libz.so.1 with build-id: 62196acf9f75625acb09840435bb396c62829e12
Found module liblzo2.so.2 with build-id: c42a1f1fd2f73000efcbe3d2fb6ad196170fd1cf
Found module libsnappy.so.1 with build-id: 83553ef5fb4b9e530a3f05b68d92f320a6d53772
Found module libpixman-1.so.0 with build-id: 4420cf633fcf726fc8cef2dd84d817083afe13e5
Found module qemu-kvm with build-id: debf9ea821f74b76b75f78c5558eba8ca491b1b5
Stack trace of thread 236656:
#0 0x00007fef0ee7b7fc __pthread_kill_implementation (libc.so.6 + 0x8f7fc)
#1 0x00007fef0ee2e676 raise (libc.so.6 + 0x42676)
#2 0x00007fef0ee187d3 abort (libc.so.6 + 0x2c7d3)
#3 0x00007fef0ee186fb __assert_fail_base.cold (libc.so.6 + 0x2c6fb)
#4 0x00007fef0ee27396 __assert_fail (libc.so.6 + 0x3b396)
#5 0x0000556d867a4791 qio_channel_restart_read (qemu-kvm + 0x760791)
#6 0x0000556d869a147f aio_dispatch_handler (qemu-kvm + 0x95d47f)
#7 0x0000556d869a132c aio_dispatch (qemu-kvm + 0x95d32c)
#8 0x0000556d869bb522 aio_ctx_dispatch (qemu-kvm + 0x977522)
#9 0x00007fef0f14df6f g_main_context_dispatch (libglib-2.0.so.0 + 0x54f6f)
#10 0x0000556d869c81e3 main_loop_wait (qemu-kvm + 0x9841e3)
#11 0x0000556d86647d37 qemu_main_loop (qemu-kvm + 0x603d37)
#12 0x0000556d863a06f2 main (qemu-kvm + 0x35c6f2)
#13 0x00007fef0ee19560 __libc_start_call_main (libc.so.6 + 0x2d560)
#14 0x00007fef0ee1960c __libc_start_main@@GLIBC_2.34 (libc.so.6 + 0x2d60c)
#15 0x0000556d8639fdb5 _start (qemu-kvm + 0x35bdb5)
Stack trace of thread 236664:
#0 0x00007fef0ee76870 __GI___lll_lock_wait (libc.so.6 + 0x8a870)
#1 0x00007fef0ee7cea2 __pthread_mutex_lock.5 (libc.so.6 + 0x90ea2)
#2 0x0000556d869a521f qemu_mutex_lock_impl (qemu-kvm + 0x96121f)
#3 0x0000556d8663c78a flatview_read_continue (qemu-kvm + 0x5f878a)
#4 0x0000556d8663c62e flatview_read (qemu-kvm + 0x5f862e)
#5 0x0000556d8663f621 address_space_read_full (qemu-kvm + 0x5fb621)
#6 0x0000556d8674c695 kvm_cpu_exec (qemu-kvm + 0x708695)
#7 0x0000556d8674e5fa kvm_vcpu_thread_fn (qemu-kvm + 0x70a5fa)
#8 0x0000556d869a609a qemu_thread_start (qemu-kvm + 0x96209a)
#9 0x00007fef0ee79aaf start_thread (libc.so.6 + 0x8daaf)
#10 0x00007fef0eefe740 __clone3 (libc.so.6 + 0x112740)
Stack trace of thread 236668:
#0 0x00007fef0eef33fb ioctl (libc.so.6 + 0x1073fb)
#1 0x0000556d8674cd77 kvm_vcpu_ioctl (qemu-kvm + 0x708d77)
#2 0x0000556d8674c601 kvm_cpu_exec (qemu-kvm + 0x708601)
#3 0x0000556d8674e5fa kvm_vcpu_thread_fn (qemu-kvm + 0x70a5fa)
#4 0x0000556d869a609a qemu_thread_start (qemu-kvm + 0x96209a)
#5 0x00007fef0ee79aaf start_thread (libc.so.6 + 0x8daaf)
#6 0x00007fef0eefe740 __clone3 (libc.so.6 + 0x112740)
Stack trace of thread 236665:
#0 0x00007fef0eef33fb ioctl (libc.so.6 + 0x1073fb)
#1 0x0000556d8674cd77 kvm_vcpu_ioctl (qemu-kvm + 0x708d77)
#2 0x0000556d8674c601 kvm_cpu_exec (qemu-kvm + 0x708601)
#3 0x0000556d8674e5fa kvm_vcpu_thread_fn (qemu-kvm + 0x70a5fa)
#4 0x0000556d869a609a qemu_thread_start (qemu-kvm + 0x96209a)
#5 0x00007fef0ee79aaf start_thread (libc.so.6 + 0x8daaf)
#6 0x00007fef0eefe740 __clone3 (libc.so.6 + 0x112740)
Stack trace of thread 236670:
#0 0x00007fef0eef33fb ioctl (libc.so.6 + 0x1073fb)
#1 0x0000556d8674cd77 kvm_vcpu_ioctl (qemu-kvm + 0x708d77)
#2 0x0000556d8674c601 kvm_cpu_exec (qemu-kvm + 0x708601)
#3 0x0000556d8674e5fa kvm_vcpu_thread_fn (qemu-kvm + 0x70a5fa)
#4 0x0000556d869a609a qemu_thread_start (qemu-kvm + 0x96209a)
#5 0x00007fef0ee79aaf start_thread (libc.so.6 + 0x8daaf)
#6 0x00007fef0eefe740 __clone3 (libc.so.6 + 0x112740)
Stack trace of thread 236657:
#0 0x00007fef0eef6fbd syscall (libc.so.6 + 0x10afbd)
#1 0x0000556d869a5def qemu_event_wait (qemu-kvm + 0x961def)
#2 0x0000556d869b1eb1 call_rcu_thread (qemu-kvm + 0x96deb1)
#3 0x0000556d869a609a qemu_thread_start (qemu-kvm + 0x96209a)
#4 0x00007fef0ee79aaf start_thread (libc.so.6 + 0x8daaf)
#5 0x00007fef0eefe740 __clone3 (libc.so.6 + 0x112740)
Stack trace of thread 236667:
#0 0x00007fef0eef33fb ioctl (libc.so.6 + 0x1073fb)
#1 0x0000556d8674cd77 kvm_vcpu_ioctl (qemu-kvm + 0x708d77)
#2 0x0000556d8674c601 kvm_cpu_exec (qemu-kvm + 0x708601)
#3 0x0000556d8674e5fa kvm_vcpu_thread_fn (qemu-kvm + 0x70a5fa)
#4 0x0000556d869a609a qemu_thread_start (qemu-kvm + 0x96209a)
#5 0x00007fef0ee79aaf start_thread (libc.so.6 + 0x8daaf)
#6 0x00007fef0eefe740 __clone3 (libc.so.6 + 0x112740)
Stack trace of thread 236663:
#0 0x00007fef0eef187f __poll (libc.so.6 + 0x10587f)
#1 0x00007fef0f1a256c g_main_context_iterate.constprop.0 (libglib-2.0.so.0 + 0xa956c)
#2 0x00007fef0f14d533 g_main_loop_run (libglib-2.0.so.0 + 0x54533)
#3 0x0000556d867cc924 iothread_run (qemu-kvm + 0x788924)
#4 0x0000556d869a609a qemu_thread_start (qemu-kvm + 0x96209a)
#5 0x00007fef0ee79aaf start_thread (libc.so.6 + 0x8daaf)
#6 0x00007fef0eefe740 __clone3 (libc.so.6 + 0x112740)
Stack trace of thread 236658:
#0 0x00007fef0eef197e ppoll (libc.so.6 + 0x10597e)
#1 0x0000556d869a2852 fdmon_poll_wait (qemu-kvm + 0x95e852)
#2 0x0000556d869a1fe4 aio_poll (qemu-kvm + 0x95dfe4)
#3 0x0000556d867cc906 iothread_run (qemu-kvm + 0x788906)
#4 0x0000556d869a609a qemu_thread_start (qemu-kvm + 0x96209a)
#5 0x00007fef0ee79aaf start_thread (libc.so.6 + 0x8daaf)
#6 0x00007fef0eefe740 __clone3 (libc.so.6 + 0x112740)
Stack trace of thread 236671:
#0 0x00007fef0eef33fb ioctl (libc.so.6 + 0x1073fb)
#1 0x0000556d8674cd77 kvm_vcpu_ioctl (qemu-kvm + 0x708d77)
#2 0x0000556d8674c601 kvm_cpu_exec (qemu-kvm + 0x708601)
#3 0x0000556d8674e5fa kvm_vcpu_thread_fn (qemu-kvm + 0x70a5fa)
#4 0x0000556d869a609a qemu_thread_start (qemu-kvm + 0x96209a)
#5 0x00007fef0ee79aaf start_thread (libc.so.6 + 0x8daaf)
#6 0x00007fef0eefe740 __clone3 (libc.so.6 + 0x112740)
Stack trace of thread 236731:
#0 0x00007fef0ee7670a __futex_abstimed_wait_common (libc.so.6 + 0x8a70a)
#1 0x00007fef0ee81813 __new_sem_wait_slow64.constprop.0 (libc.so.6 + 0x95813)
#2 0x0000556d869a5b0b qemu_sem_timedwait (qemu-kvm + 0x961b0b)
#3 0x0000556d869cc6b6 worker_thread (qemu-kvm + 0x9886b6)
#4 0x0000556d869a609a qemu_thread_start (qemu-kvm + 0x96209a)
#5 0x00007fef0ee79aaf start_thread (libc.so.6 + 0x8daaf)
#6 0x00007fef0eefe740 __clone3 (libc.so.6 + 0x112740)
Stack trace of thread 236730:
#0 0x00007fef0ee7670a __futex_abstimed_wait_common (libc.so.6 + 0x8a70a)
#1 0x00007fef0ee81813 __new_sem_wait_slow64.constprop.0 (libc.so.6 + 0x95813)
#2 0x0000556d869a5b0b qemu_sem_timedwait (qemu-kvm + 0x961b0b)
#3 0x0000556d869cc6b6 worker_thread (qemu-kvm + 0x9886b6)
#4 0x0000556d869a609a qemu_thread_start (qemu-kvm + 0x96209a)
#5 0x00007fef0ee79aaf start_thread (libc.so.6 + 0x8daaf)
#6 0x00007fef0eefe740 __clone3 (libc.so.6 + 0x112740)
Stack trace of thread 236675:
#0 0x00007fef0ee7670a __futex_abstimed_wait_common (libc.so.6 + 0x8a70a)
#1 0x00007fef0ee78e80 pthread_cond_wait@@GLIBC_2.3.2 (libc.so.6 + 0x8ce80)
#2 0x0000556d869a56ff qemu_cond_wait_impl (qemu-kvm + 0x9616ff)
#3 0x0000556d863dd657 vnc_worker_thread (qemu-kvm + 0x399657)
#4 0x0000556d869a609a qemu_thread_start (qemu-kvm + 0x96209a)
#5 0x00007fef0ee79aaf start_thread (libc.so.6 + 0x8daaf)
#6 0x00007fef0eefe740 __clone3 (libc.so.6 + 0x112740)
Stack trace of thread 236666:
#0 0x00007fef0eef33fb ioctl (libc.so.6 + 0x1073fb)
#1 0x0000556d8674cd77 kvm_vcpu_ioctl (qemu-kvm + 0x708d77)
#2 0x0000556d8674c601 kvm_cpu_exec (qemu-kvm + 0x708601)
#3 0x0000556d8674e5fa kvm_vcpu_thread_fn (qemu-kvm + 0x70a5fa)
#4 0x0000556d869a609a qemu_thread_start (qemu-kvm + 0x96209a)
#5 0x00007fef0ee79aaf start_thread (libc.so.6 + 0x8daaf)
#6 0x00007fef0eefe740 __clone3 (libc.so.6 + 0x112740)
Stack trace of thread 236672:
#0 0x00007fef0eef33fb ioctl (libc.so.6 + 0x1073fb)
#1 0x0000556d8674cd77 kvm_vcpu_ioctl (qemu-kvm + 0x708d77)
#2 0x0000556d8674c601 kvm_cpu_exec (qemu-kvm + 0x708601)
#3 0x0000556d8674e5fa kvm_vcpu_thread_fn (qemu-kvm + 0x70a5fa)
#4 0x0000556d869a609a qemu_thread_start (qemu-kvm + 0x96209a)
#5 0x00007fef0ee79aaf start_thread (libc.so.6 + 0x8daaf)
#6 0x00007fef0eefe740 __clone3 (libc.so.6 + 0x112740)
Stack trace of thread 236669:
#0 0x00007fef0eef33fb ioctl (libc.so.6 + 0x1073fb)
#1 0x0000556d8674cd77 kvm_vcpu_ioctl (qemu-kvm + 0x708d77)
#2 0x0000556d8674c601 kvm_cpu_exec (qemu-kvm + 0x708601)
#3 0x0000556d8674e5fa kvm_vcpu_thread_fn (qemu-kvm + 0x70a5fa)
#4 0x0000556d869a609a qemu_thread_start (qemu-kvm + 0x96209a)
#5 0x00007fef0ee79aaf start_thread (libc.so.6 + 0x8daaf)
#6 0x00007fef0eefe740 __clone3 (libc.so.6 + 0x112740)
Stack trace of thread 236673:
#0 0x00007fef0eef33fb ioctl (libc.so.6 + 0x1073fb)
#1 0x0000556d8674cd77 kvm_vcpu_ioctl (qemu-kvm + 0x708d77)
#2 0x0000556d8674c601 kvm_cpu_exec (qemu-kvm + 0x708601)
#3 0x0000556d8674e5fa kvm_vcpu_thread_fn (qemu-kvm + 0x70a5fa)
#4 0x0000556d869a609a qemu_thread_start (qemu-kvm + 0x96209a)
#5 0x00007fef0ee79aaf start_thread (libc.so.6 + 0x8daaf)
#6 0x00007fef0eefe740 __clone3 (libc.so.6 + 0x112740)
Expected results:
VM can start successfully with exposed nbd data image.
Additional info:
Will attach core dump file later.
Comment 2Klaus Heinrich Kiwi
2021-12-21 19:16:23 UTC
Eric, can you take this? I've classified it as high-priority as it's a crash in apparently a customer-exposed path, but let's discuss if you disagree.
Aihua Liang, can you identify if this is a new testcase, or a regression? In case of the latter, set the regression keyword and identify the last working qemu-kvm version?
Thanks
(In reply to Klaus Heinrich Kiwi from comment #2)
> Eric, can you take this? I've classified it as high-priority as it's a crash
> in apparently a customer-exposed path, but let's discuss if you disagree.
>
> Aihua Liang, can you identify if this is a new testcase, or a regression? In
> case of the latter, set the regression keyword and identify the last working
> qemu-kvm version?
>
> Thanks
Hi, Klaus
It's not a new case and its a regression bug. emu-kvm-6.1.0-8.el9 don't hit it.
BR,
Aliang
I believe this is the bug I described in BZ 1990835 comment 7, and which I’ve sent an RFC for (currently working on writing tests for a proper v1): https://lists.nongnu.org/archive/html/qemu-block/2022-01/msg00765.html
If you prefer, Eric, I could take this bug. (I was about to create a new one for this issue anyway, and the duplicate detector pointed me towards this one, which is how I found it.)
Comment 6Klaus Heinrich Kiwi
2022-02-07 14:32:03 UTC
(In reply to Hanna Reitz from comment #5)
> I believe this is the bug I described in BZ 1990835 comment 7, and which
> I’ve sent an RFC for (currently working on writing tests for a proper v1):
> https://lists.nongnu.org/archive/html/qemu-block/2022-01/msg00765.html
>
> If you prefer, Eric, I could take this bug. (I was about to create a new
> one for this issue anyway, and the duplicate detector pointed me towards
> this one, which is how I found it.)
Hanna, may I take you on your offer? If you have a scratch build that QE can test, please let us know. Thanks!
Test with qemu-kvm-6.2.0-7.el9.hreitz202202071714 by auto, don't hit this issue any more.
(1/3) Host_RHEL.m9.u0.qcow2.virtio_scsi.up.virtio_net.Guest.RHEL.9.0.0.x86_64.io-github-autotest-qemu.blockdev_stream_remote_server_down.q35: PASS (144.42 s)
(2/3) Host_RHEL.m9.u0.qcow2.virtio_scsi.up.virtio_net.Guest.RHEL.9.0.0.x86_64.io-github-autotest-qemu.blockdev_mirror_remote_server_down.q35: PASS (122.59 s)
(3/3) Host_RHEL.m9.u0.qcow2.virtio_scsi.up.virtio_net.Guest.RHEL.9.0.0.x86_64.io-github-autotest-qemu.blockdev_commit_firewall.q35: PASS (321.08 s)
This fix has been merged upstream, and I’ve sent a backport for 8.6 (see BZ 2035185). I’d love to send a backport for this BZ, too, but can’t, because the Brew build server seems to have some subscription problems. If they persist, I suppose I’ll just send a backport without a Brew build.
Hanna
Test with qemu-kvm-6.2.0-9.el9, not hit this issue any more.
(1/3) Host_RHEL.m9.u0.qcow2.virtio_scsi.up.virtio_net.Guest.RHEL.9.0.0.x86_64.io-github-autotest-qemu.blockdev_stream_remote_server_down.q35: PASS (143.82 s)
(2/3) Host_RHEL.m9.u0.qcow2.virtio_scsi.up.virtio_net.Guest.RHEL.9.0.0.x86_64.io-github-autotest-qemu.blockdev_mirror_remote_server_down.q35: PASS (119.67 s)
(3/3) Host_RHEL.m9.u0.qcow2.virtio_scsi.up.virtio_net.Guest.RHEL.9.0.0.x86_64.io-github-autotest-qemu.blockdev_commit_firewall.q35: PASS (321.33 s)
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.
For information on the advisory (new packages: qemu-kvm), and where to find the updated
files, follow the link below.
If the solution does not work for you, open a new bug report.
https://access.redhat.com/errata/RHBA-2022:2307
Description of problem: Qemu core dump when block jobs on nbd node were blocked by firewall or server down Version-Release number of selected component (if applicable): kernel version:5.14.0-30.el9.x86_64 qemu-kvm version:qemu-kvm-6.2.0-1.el9 How reproducible: 100% by auto Steps to Reproduce: 1.Create data disk image. #qemu-img create -f qcow2 /root/avocado/data/avocado-vt/data.qcow2 2G 2.Expose data image #/usr/bin/qemu-nbd -f qcow2 -t -p 10810 --fork --pid-file /tmp/avocado_0e1jqoko/data_nbd_server-20211217-075948-p12l.pid /root/avocado/data/avocado-vt/data.qcow2 3.Start guest with the exposed data disk: /usr/libexec/qemu-kvm \ -S \ -name 'avocado-vt-vm1' \ -sandbox on \ -machine q35,memory-backend=mem-machine_mem \ -device pcie-root-port,id=pcie-root-port-0,multifunction=on,bus=pcie.0,addr=0x1,chassis=1 \ -device pcie-pci-bridge,id=pcie-pci-bridge-0,addr=0x0,bus=pcie-root-port-0 \ -nodefaults \ -device VGA,bus=pcie.0,addr=0x2 \ -m 30720 \ -object memory-backend-ram,size=30720M,id=mem-machine_mem \ -smp 10,maxcpus=10,cores=5,threads=1,dies=1,sockets=2 \ -cpu 'Cascadelake-Server-noTSX',+kvm_pv_unhalt \ -chardev socket,wait=off,path=/tmp/avocado_0e1jqoko/monitor-qmpmonitor1-20211217-075949-ZwO9C7pE,server=on,id=qmp_id_qmpmonitor1 \ -mon chardev=qmp_id_qmpmonitor1,mode=control \ -chardev socket,wait=off,path=/tmp/avocado_0e1jqoko/monitor-catch_monitor-20211217-075949-ZwO9C7pE,server=on,id=qmp_id_catch_monitor \ -mon chardev=qmp_id_catch_monitor,mode=control \ -device pvpanic,ioport=0x505,id=id9Na7Yp \ -chardev socket,wait=off,path=/tmp/avocado_0e1jqoko/serial-serial0-20211217-075949-ZwO9C7pE,server=on,id=chardev_serial0 \ -device isa-serial,id=serial0,chardev=chardev_serial0 \ -chardev socket,id=seabioslog_id_20211217-075949-ZwO9C7pE,path=/tmp/avocado_0e1jqoko/seabios-20211217-075949-ZwO9C7pE,server=on,wait=off \ -device isa-debugcon,chardev=seabioslog_id_20211217-075949-ZwO9C7pE,iobase=0x402 \ -device pcie-root-port,id=pcie-root-port-1,port=0x1,addr=0x1.0x1,bus=pcie.0,chassis=2 \ -device qemu-xhci,id=usb1,bus=pcie-root-port-1,addr=0x0 \ -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 \ -object iothread,id=iothread0 \ -device pcie-root-port,id=pcie-root-port-2,port=0x2,addr=0x1.0x2,bus=pcie.0,chassis=3 \ -device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pcie-root-port-2,addr=0x0,iothread=iothread0 \ -blockdev node-name=file_image1,driver=file,auto-read-only=on,discard=unmap,aio=threads,filename=/home/kvm_autotest_root/images/rhel900-64-virtio-scsi.qcow2,cache.direct=on,cache.no-flush=off \ -blockdev node-name=drive_image1,driver=qcow2,read-only=off,cache.direct=on,cache.no-flush=off,file=file_image1 \ -device scsi-hd,id=image1,drive=drive_image1,write-cache=on \ -blockdev node-name=nbd_nbddata,driver=nbd,auto-read-only=on,discard=unmap,server.type=inet,server.host=dell-per440-09.lab.eng.pek2.redhat.com,server.port=10810,reconnect-delay=30,cache.direct=on,cache.no-flush=off \ -blockdev node-name=drive_nbddata,driver=raw,read-only=off,cache.direct=on,cache.no-flush=off,file=nbd_nbddata \ -device scsi-hd,id=nbddata,drive=drive_nbddata,write-cache=on \ -device pcie-root-port,id=pcie-root-port-3,port=0x3,addr=0x1.0x3,bus=pcie.0,chassis=4 \ -device virtio-net-pci,mac=9a:ed:fc:56:77:68,id=idY9yr3Q,netdev=id1qEUjK,bus=pcie-root-port-3,addr=0x0 \ -netdev tap,id=id1qEUjK,vhost=on,vhostfd=22,fd=19 \ -vnc :0 \ -rtc base=utc,clock=host,driftfix=slew \ -boot menu=off,order=cdn,once=c,strict=off \ -enable-kvm \ -device pcie-root-port,id=pcie_extra_root_port_0,multifunction=on,bus=pcie.0,addr=0x3,chassis=5 4. Continue vm {'execute': 'cont', 'id': 'CE1lLVef'} Actual results: After step4, qemu core dump with info: [qemu output] qemu-kvm: ../io/channel.c:484: void qio_channel_restart_read(void *): Assertion `qemu_get_current_aio_context() == qemu_coroutine_get_aio_context(co)' failed. [qemu output] /tmp/aexpect_SwITdppJ/aexpect-y1zy2kcy.sh: line 1: 236656 Aborted (core dumped) MALLOC_PERTURB_=1 /usr/libexec/qemu-kvm -S -name 'avocado-vt-vm1' -sandbox on -machine q35,memory-backend=mem-machine_mem -device pcie-root-port,id=pcie-root-port-0,multifunction=on,bus=pcie.0,addr=0x1,chassis=1 -device pcie-pci-bridge,id=pcie-pci-bridge-0,addr=0x0,bus=pcie-root-port-0 ... Detailed core dump info: coredumpctl debug 236656 PID: 236656 (qemu-kvm) UID: 0 (root) GID: 0 (root) Signal: 6 (ABRT) Timestamp: Fri 2021-12-17 08:00:02 EST (8min ago) Command Line: /usr/libexec/qemu-kvm -S -name avocado-vt-vm1 -sandbox on -machine q35,memory-backend=mem-machine_mem -device pcie-root-port,id=pcie-root-port-0,multifunction=on,bus=pcie.0,addr=0x1,chassis=1 -device pcie-pci-bridge,id=pcie-pci-bridge-0,addr=0x0,bus=pcie-root-port-0 -nodefaults -device VGA,bus=pcie.0,addr=0x2 -m 30720 -object memory-backend-ram,size=30720M,id=mem-machine_mem -smp 10,maxcpus=10,cores=5,threads=1,dies=1,sockets=2 -cpu Cascadelake-Server-noTSX,+kvm_pv_unhalt -chardev socket,wait=off,path=/tmp/avocado_0e1jqoko/monitor-qmpmonitor1-20211217-075949-ZwO9C7pE,server=on,id=qmp_id_qmpmonitor1 -mon chardev=qmp_id_qmpmonitor1,mode=control -chardev socket,wait=off,path=/tmp/avocado_0e1jqoko/monitor-catch_monitor-20211217-075949-ZwO9C7pE,server=on,id=qmp_id_catch_monitor -mon chardev=qmp_id_catch_monitor,mode=control -device pvpanic,ioport=0x505,id=id9Na7Yp -chardev socket,wait=off,path=/tmp/avocado_0e1jqoko/serial-serial0-20211217-075949-ZwO9C7pE,server=on,id=chardev_serial0 -device isa-serial,id=serial0,chardev=chardev_serial0 -chardev socket,id=seabioslog_id_20211217-075949-ZwO9C7pE,path=/tmp/avocado_0e1jqoko/seabios-20211217-075949-ZwO9C7pE,server=on,wait=off -device isa-debugcon,chardev=seabioslog_id_20211217-075949-ZwO9C7pE,iobase=0x402 -device pcie-root-port,id=pcie-root-port-1,port=0x1,addr=0x1.0x1,bus=pcie.0,chassis=2 -device qemu-xhci,id=usb1,bus=pcie-root-port-1,addr=0x0 -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 -object iothread,id=iothread0 -device pcie-root-port,id=pcie-root-port-2,port=0x2,addr=0x1.0x2,bus=pcie.0,chassis=3 -device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pcie-root-port-2,addr=0x0,iothread=iothread0 -blockdev node-name=file_image1,driver=file,auto-read-only=on,discard=unmap,aio=threads,filename=/home/kvm_autotest_root/images/rhel900-64-virtio-scsi.qcow2,cache.direct=on,cache.no-flush=off -blockdev node-name=drive_image1,driver=qcow2,read-only=off,cache.direct=on,cache.no-flush=off,file=file_image1 -device scsi-hd,id=image1,drive=drive_image1,write-cache=on -blockdev node-name=nbd_nbddata,driver=nbd,auto-read-only=on,discard=unmap,server.type=inet,server.host=dell-per440-09.lab.eng.pek2.redhat.com,server.port=10810,reconnect-delay=30,cache.direct=on,cache.no-flush=off -blockdev node-name=drive_nbddata,driver=raw,read-only=off,cache.direct=on,cache.no-flush=off,file=nbd_nbddata -device scsi-hd,id=nbddata,drive=drive_nbddata,write-cache=on -device pcie-root-port,id=pcie-root-port-3,port=0x3,addr=0x1.0x3,bus=pcie.0,chassis=4 -device virtio-net-pci,mac=9a:ed:fc:56:77:68,id=idY9yr3Q,netdev=id1qEUjK,bus=pcie-root-port-3,addr=0x0 -netdev tap,id=id1qEUjK,vhost=on,vhostfd=22,fd=19 -vnc :0 -rtc base=utc,clock=host,driftfix=slew -boot menu=off,order=cdn,once=c,strict=off -enable-kvm -device pcie-root-port,id=pcie_extra_root_port_0,multifunction=on,bus=pcie.0,addr=0x3,chassis=5 Executable: /usr/libexec/qemu-kvm Control Group: /user.slice/user-0.slice/session-1.scope Unit: session-1.scope Slice: user-0.slice Session: 1 Owner UID: 0 (root) Boot ID: 0107346256b84e8ebd9076ae0ca431c0 Machine ID: 1437cd218cca45cbb2059307f6958ac6 Hostname: dell-per440-09.lab.eng.pek2.redhat.com Storage: /var/lib/systemd/coredump/core.qemu-kvm.0.0107346256b84e8ebd9076ae0ca431c0.236656.1639746002000000.zst (present) Disk Size: 123.9M Message: Process 236656 (qemu-kvm) of user 0 dumped core. Found module linux-vdso.so.1 with build-id: 0417c877078abdfaf5ca0d3d449ce6699e5a520c Found module libpcre2-8.so.0 with build-id: a5dafbac0243bdc2e3b5ab9a07d61d4b32f86927 Found module liblzma.so.5 with build-id: 66b7fca8b63a61d19c5ded440ee6a8bf05dcafa5 Found module libzstd.so.1 with build-id: 415c738420cd82e182b99fc4c6e6821098e80ce4 Found module libselinux.so.1 with build-id: c1d23d40bcf1fae271256fcbe90a46ff6dbcce20 Found module libkmod.so.2 with build-id: 722d3027106c561018756766be8bed5215ed0f31 Found module libuuid.so.1 with build-id: da656ae81596c151559892375d730bd35f76048c Found module libudev.so.1 with build-id: 7c275efc762b4b5a63016164ffd5a53962effbb1 Found module libcrypto.so.3 with build-id: e3a618644eec4fb65c7cd059c42592a686191f72 Found module libkeyutils.so.1 with build-id: 83c6539bd0d3140678ba836b8baa1b215efa2632 Found module libkrb5support.so.0 with build-id: 22e9f06a5b088d401df9153948492077b4e198af Found module libffi.so.8 with build-id: 48e3675db4765a2e42729140922e11a10016f7ab Found module libpcre.so.1 with build-id: cffb947bcc416dca3cd249cdb0a1c6f614549c30 Found module libnl-route-3.so.200 with build-id: 2f297144194e194e379c1c31dcd245220764a488 Found module libnl-3.so.200 with build-id: 5690fe17968270ad4ee00fdafe15cf0a4db9c7ed Found module libdaxctl.so.1 with build-id: c46dc4ef727b75a311e76e41940451a90b9f1080 Found module libndctl.so.6 with build-id: 8c52e205e3982b331bb9683f1a35d53bb1c11c50 Found module libresolv.so.2 with build-id: eaebc6c6a15dcc8af04ea904eb33aeb5557fcafd Found module libcom_err.so.2 with build-id: deb039f83bae82e65be49dd97ac3d900a053e265 Found module libk5crypto.so.3 with build-id: eca094d4bc539db337447b773462de7b592e785b Found module libkrb5.so.3 with build-id: a3fec09569ad99121e79e2d73b0d446169965d26 Found module libgssapi_krb5.so.2 with build-id: a2daa1c10270547f24b82fcb0034e84853d35344 Found module libcrypt.so.2 with build-id: 6ce4e5eb200e61d07398af52f8bcb316cf8466e0 Found module libgmp.so.10 with build-id: 754b87178e725c166a68353f043ba7638e0723e1 Found module libhogweed.so.6 with build-id: 21c32760f1f50b9d69b43e966b93fb3de8baa076 Found module libnettle.so.8 with build-id: 3eda844d5a5df2443a281158b0d7ca6e81483b59 Found module libtasn1.so.6 with build-id: 3d3a2f6f0d4a70919496afe25e329abd189b7882 Found module libunistring.so.2 with build-id: 15e34cdfafa3547f9c700489b842ceb86f6fb73e Found module libidn2.so.0 with build-id: 958c50fc94ecb196b24f3619762e7ec3f28a5b40 Found module libp11-kit.so.0 with build-id: b03163bf5b98313c7347e7270627a6b2dcd35060 Found module libstdc++.so.6 with build-id: 66b742a0a51ccd8a67fdf59a31663b0cd28dfcf2 Found module ld-linux-x86-64.so.2 with build-id: 2b7f1f415256e40ab15e5808c73eb7acf3c73ea3 Found module libc.so.6 with build-id: 723a5d3072f3b79f3716880303439e11f7661fc5 Found module libgcc_s.so.1 with build-id: a1a28a65f942690a11041922ba11c16f716565e2 Found module libm.so.6 with build-id: c58412f6a7fffa4068614708cdb5b15f2ba49c0a Found module libaio.so.1 with build-id: 9b75d21282f8e17ddfa06aff78dae4f8dcce4106 Found module libgmodule-2.0.so.0 with build-id: 952459b04fb217bc3e4a17b902759cf66c89a95d Found module libglib-2.0.so.0 with build-id: 0006a4eed3e835c7d7812e36e5d720c1a6eb8689 Found module libslirp.so.0 with build-id: fa9d492d6d050c6f5cb906a3f087a8a63e4201e2 Found module libibverbs.so.1 with build-id: 3e19d7349fe9ea4c79debeb8716df7000ab13159 Found module librdmacm.so.1 with build-id: 4f6798f3f4164a4aa4d75d8a890c760e2835c495 Found module libnuma.so.1 with build-id: a0c23354fdb39f51ec9e31aeae27a9f8ee034500 Found module libfdt.so.1 with build-id: 31a56e0009a8824c7a09267c8205034c91cb4095 Found module libseccomp.so.2 with build-id: 5a5b4ee3bb99ee0378bd937fdef7a7b7255d0428 Found module libpmem.so.1 with build-id: 084e72da2642981fe75b446293655ae4a68c0663 Found module libsasl2.so.3 with build-id: 2864d8e1600f3b46016fd62c44269e92c5ec0ca1 Found module libgnutls.so.30 with build-id: b08009ada925278615f5592e982e0776c06e065e Found module libpng16.so.16 with build-id: 698f67f36c58fa443c6ba891e4a0f106553a0452 Found module libz.so.1 with build-id: 62196acf9f75625acb09840435bb396c62829e12 Found module liblzo2.so.2 with build-id: c42a1f1fd2f73000efcbe3d2fb6ad196170fd1cf Found module libsnappy.so.1 with build-id: 83553ef5fb4b9e530a3f05b68d92f320a6d53772 Found module libpixman-1.so.0 with build-id: 4420cf633fcf726fc8cef2dd84d817083afe13e5 Found module qemu-kvm with build-id: debf9ea821f74b76b75f78c5558eba8ca491b1b5 Stack trace of thread 236656: #0 0x00007fef0ee7b7fc __pthread_kill_implementation (libc.so.6 + 0x8f7fc) #1 0x00007fef0ee2e676 raise (libc.so.6 + 0x42676) #2 0x00007fef0ee187d3 abort (libc.so.6 + 0x2c7d3) #3 0x00007fef0ee186fb __assert_fail_base.cold (libc.so.6 + 0x2c6fb) #4 0x00007fef0ee27396 __assert_fail (libc.so.6 + 0x3b396) #5 0x0000556d867a4791 qio_channel_restart_read (qemu-kvm + 0x760791) #6 0x0000556d869a147f aio_dispatch_handler (qemu-kvm + 0x95d47f) #7 0x0000556d869a132c aio_dispatch (qemu-kvm + 0x95d32c) #8 0x0000556d869bb522 aio_ctx_dispatch (qemu-kvm + 0x977522) #9 0x00007fef0f14df6f g_main_context_dispatch (libglib-2.0.so.0 + 0x54f6f) #10 0x0000556d869c81e3 main_loop_wait (qemu-kvm + 0x9841e3) #11 0x0000556d86647d37 qemu_main_loop (qemu-kvm + 0x603d37) #12 0x0000556d863a06f2 main (qemu-kvm + 0x35c6f2) #13 0x00007fef0ee19560 __libc_start_call_main (libc.so.6 + 0x2d560) #14 0x00007fef0ee1960c __libc_start_main@@GLIBC_2.34 (libc.so.6 + 0x2d60c) #15 0x0000556d8639fdb5 _start (qemu-kvm + 0x35bdb5) Stack trace of thread 236664: #0 0x00007fef0ee76870 __GI___lll_lock_wait (libc.so.6 + 0x8a870) #1 0x00007fef0ee7cea2 __pthread_mutex_lock.5 (libc.so.6 + 0x90ea2) #2 0x0000556d869a521f qemu_mutex_lock_impl (qemu-kvm + 0x96121f) #3 0x0000556d8663c78a flatview_read_continue (qemu-kvm + 0x5f878a) #4 0x0000556d8663c62e flatview_read (qemu-kvm + 0x5f862e) #5 0x0000556d8663f621 address_space_read_full (qemu-kvm + 0x5fb621) #6 0x0000556d8674c695 kvm_cpu_exec (qemu-kvm + 0x708695) #7 0x0000556d8674e5fa kvm_vcpu_thread_fn (qemu-kvm + 0x70a5fa) #8 0x0000556d869a609a qemu_thread_start (qemu-kvm + 0x96209a) #9 0x00007fef0ee79aaf start_thread (libc.so.6 + 0x8daaf) #10 0x00007fef0eefe740 __clone3 (libc.so.6 + 0x112740) Stack trace of thread 236668: #0 0x00007fef0eef33fb ioctl (libc.so.6 + 0x1073fb) #1 0x0000556d8674cd77 kvm_vcpu_ioctl (qemu-kvm + 0x708d77) #2 0x0000556d8674c601 kvm_cpu_exec (qemu-kvm + 0x708601) #3 0x0000556d8674e5fa kvm_vcpu_thread_fn (qemu-kvm + 0x70a5fa) #4 0x0000556d869a609a qemu_thread_start (qemu-kvm + 0x96209a) #5 0x00007fef0ee79aaf start_thread (libc.so.6 + 0x8daaf) #6 0x00007fef0eefe740 __clone3 (libc.so.6 + 0x112740) Stack trace of thread 236665: #0 0x00007fef0eef33fb ioctl (libc.so.6 + 0x1073fb) #1 0x0000556d8674cd77 kvm_vcpu_ioctl (qemu-kvm + 0x708d77) #2 0x0000556d8674c601 kvm_cpu_exec (qemu-kvm + 0x708601) #3 0x0000556d8674e5fa kvm_vcpu_thread_fn (qemu-kvm + 0x70a5fa) #4 0x0000556d869a609a qemu_thread_start (qemu-kvm + 0x96209a) #5 0x00007fef0ee79aaf start_thread (libc.so.6 + 0x8daaf) #6 0x00007fef0eefe740 __clone3 (libc.so.6 + 0x112740) Stack trace of thread 236670: #0 0x00007fef0eef33fb ioctl (libc.so.6 + 0x1073fb) #1 0x0000556d8674cd77 kvm_vcpu_ioctl (qemu-kvm + 0x708d77) #2 0x0000556d8674c601 kvm_cpu_exec (qemu-kvm + 0x708601) #3 0x0000556d8674e5fa kvm_vcpu_thread_fn (qemu-kvm + 0x70a5fa) #4 0x0000556d869a609a qemu_thread_start (qemu-kvm + 0x96209a) #5 0x00007fef0ee79aaf start_thread (libc.so.6 + 0x8daaf) #6 0x00007fef0eefe740 __clone3 (libc.so.6 + 0x112740) Stack trace of thread 236657: #0 0x00007fef0eef6fbd syscall (libc.so.6 + 0x10afbd) #1 0x0000556d869a5def qemu_event_wait (qemu-kvm + 0x961def) #2 0x0000556d869b1eb1 call_rcu_thread (qemu-kvm + 0x96deb1) #3 0x0000556d869a609a qemu_thread_start (qemu-kvm + 0x96209a) #4 0x00007fef0ee79aaf start_thread (libc.so.6 + 0x8daaf) #5 0x00007fef0eefe740 __clone3 (libc.so.6 + 0x112740) Stack trace of thread 236667: #0 0x00007fef0eef33fb ioctl (libc.so.6 + 0x1073fb) #1 0x0000556d8674cd77 kvm_vcpu_ioctl (qemu-kvm + 0x708d77) #2 0x0000556d8674c601 kvm_cpu_exec (qemu-kvm + 0x708601) #3 0x0000556d8674e5fa kvm_vcpu_thread_fn (qemu-kvm + 0x70a5fa) #4 0x0000556d869a609a qemu_thread_start (qemu-kvm + 0x96209a) #5 0x00007fef0ee79aaf start_thread (libc.so.6 + 0x8daaf) #6 0x00007fef0eefe740 __clone3 (libc.so.6 + 0x112740) Stack trace of thread 236663: #0 0x00007fef0eef187f __poll (libc.so.6 + 0x10587f) #1 0x00007fef0f1a256c g_main_context_iterate.constprop.0 (libglib-2.0.so.0 + 0xa956c) #2 0x00007fef0f14d533 g_main_loop_run (libglib-2.0.so.0 + 0x54533) #3 0x0000556d867cc924 iothread_run (qemu-kvm + 0x788924) #4 0x0000556d869a609a qemu_thread_start (qemu-kvm + 0x96209a) #5 0x00007fef0ee79aaf start_thread (libc.so.6 + 0x8daaf) #6 0x00007fef0eefe740 __clone3 (libc.so.6 + 0x112740) Stack trace of thread 236658: #0 0x00007fef0eef197e ppoll (libc.so.6 + 0x10597e) #1 0x0000556d869a2852 fdmon_poll_wait (qemu-kvm + 0x95e852) #2 0x0000556d869a1fe4 aio_poll (qemu-kvm + 0x95dfe4) #3 0x0000556d867cc906 iothread_run (qemu-kvm + 0x788906) #4 0x0000556d869a609a qemu_thread_start (qemu-kvm + 0x96209a) #5 0x00007fef0ee79aaf start_thread (libc.so.6 + 0x8daaf) #6 0x00007fef0eefe740 __clone3 (libc.so.6 + 0x112740) Stack trace of thread 236671: #0 0x00007fef0eef33fb ioctl (libc.so.6 + 0x1073fb) #1 0x0000556d8674cd77 kvm_vcpu_ioctl (qemu-kvm + 0x708d77) #2 0x0000556d8674c601 kvm_cpu_exec (qemu-kvm + 0x708601) #3 0x0000556d8674e5fa kvm_vcpu_thread_fn (qemu-kvm + 0x70a5fa) #4 0x0000556d869a609a qemu_thread_start (qemu-kvm + 0x96209a) #5 0x00007fef0ee79aaf start_thread (libc.so.6 + 0x8daaf) #6 0x00007fef0eefe740 __clone3 (libc.so.6 + 0x112740) Stack trace of thread 236731: #0 0x00007fef0ee7670a __futex_abstimed_wait_common (libc.so.6 + 0x8a70a) #1 0x00007fef0ee81813 __new_sem_wait_slow64.constprop.0 (libc.so.6 + 0x95813) #2 0x0000556d869a5b0b qemu_sem_timedwait (qemu-kvm + 0x961b0b) #3 0x0000556d869cc6b6 worker_thread (qemu-kvm + 0x9886b6) #4 0x0000556d869a609a qemu_thread_start (qemu-kvm + 0x96209a) #5 0x00007fef0ee79aaf start_thread (libc.so.6 + 0x8daaf) #6 0x00007fef0eefe740 __clone3 (libc.so.6 + 0x112740) Stack trace of thread 236730: #0 0x00007fef0ee7670a __futex_abstimed_wait_common (libc.so.6 + 0x8a70a) #1 0x00007fef0ee81813 __new_sem_wait_slow64.constprop.0 (libc.so.6 + 0x95813) #2 0x0000556d869a5b0b qemu_sem_timedwait (qemu-kvm + 0x961b0b) #3 0x0000556d869cc6b6 worker_thread (qemu-kvm + 0x9886b6) #4 0x0000556d869a609a qemu_thread_start (qemu-kvm + 0x96209a) #5 0x00007fef0ee79aaf start_thread (libc.so.6 + 0x8daaf) #6 0x00007fef0eefe740 __clone3 (libc.so.6 + 0x112740) Stack trace of thread 236675: #0 0x00007fef0ee7670a __futex_abstimed_wait_common (libc.so.6 + 0x8a70a) #1 0x00007fef0ee78e80 pthread_cond_wait@@GLIBC_2.3.2 (libc.so.6 + 0x8ce80) #2 0x0000556d869a56ff qemu_cond_wait_impl (qemu-kvm + 0x9616ff) #3 0x0000556d863dd657 vnc_worker_thread (qemu-kvm + 0x399657) #4 0x0000556d869a609a qemu_thread_start (qemu-kvm + 0x96209a) #5 0x00007fef0ee79aaf start_thread (libc.so.6 + 0x8daaf) #6 0x00007fef0eefe740 __clone3 (libc.so.6 + 0x112740) Stack trace of thread 236666: #0 0x00007fef0eef33fb ioctl (libc.so.6 + 0x1073fb) #1 0x0000556d8674cd77 kvm_vcpu_ioctl (qemu-kvm + 0x708d77) #2 0x0000556d8674c601 kvm_cpu_exec (qemu-kvm + 0x708601) #3 0x0000556d8674e5fa kvm_vcpu_thread_fn (qemu-kvm + 0x70a5fa) #4 0x0000556d869a609a qemu_thread_start (qemu-kvm + 0x96209a) #5 0x00007fef0ee79aaf start_thread (libc.so.6 + 0x8daaf) #6 0x00007fef0eefe740 __clone3 (libc.so.6 + 0x112740) Stack trace of thread 236672: #0 0x00007fef0eef33fb ioctl (libc.so.6 + 0x1073fb) #1 0x0000556d8674cd77 kvm_vcpu_ioctl (qemu-kvm + 0x708d77) #2 0x0000556d8674c601 kvm_cpu_exec (qemu-kvm + 0x708601) #3 0x0000556d8674e5fa kvm_vcpu_thread_fn (qemu-kvm + 0x70a5fa) #4 0x0000556d869a609a qemu_thread_start (qemu-kvm + 0x96209a) #5 0x00007fef0ee79aaf start_thread (libc.so.6 + 0x8daaf) #6 0x00007fef0eefe740 __clone3 (libc.so.6 + 0x112740) Stack trace of thread 236669: #0 0x00007fef0eef33fb ioctl (libc.so.6 + 0x1073fb) #1 0x0000556d8674cd77 kvm_vcpu_ioctl (qemu-kvm + 0x708d77) #2 0x0000556d8674c601 kvm_cpu_exec (qemu-kvm + 0x708601) #3 0x0000556d8674e5fa kvm_vcpu_thread_fn (qemu-kvm + 0x70a5fa) #4 0x0000556d869a609a qemu_thread_start (qemu-kvm + 0x96209a) #5 0x00007fef0ee79aaf start_thread (libc.so.6 + 0x8daaf) #6 0x00007fef0eefe740 __clone3 (libc.so.6 + 0x112740) Stack trace of thread 236673: #0 0x00007fef0eef33fb ioctl (libc.so.6 + 0x1073fb) #1 0x0000556d8674cd77 kvm_vcpu_ioctl (qemu-kvm + 0x708d77) #2 0x0000556d8674c601 kvm_cpu_exec (qemu-kvm + 0x708601) #3 0x0000556d8674e5fa kvm_vcpu_thread_fn (qemu-kvm + 0x70a5fa) #4 0x0000556d869a609a qemu_thread_start (qemu-kvm + 0x96209a) #5 0x00007fef0ee79aaf start_thread (libc.so.6 + 0x8daaf) #6 0x00007fef0eefe740 __clone3 (libc.so.6 + 0x112740) Expected results: VM can start successfully with exposed nbd data image. Additional info: Will attach core dump file later.