Bug 2168209
| Summary: | Qemu coredump after do snapshot of mirrored top image and its converted base image(iothread enabled) | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 9 | Reporter: | aihua liang <aliang> | ||||
| Component: | qemu-kvm | Assignee: | Stefano Garzarella <sgarzare> | ||||
| qemu-kvm sub component: | Block Jobs | QA Contact: | aihua liang <aliang> | ||||
| Status: | CLOSED ERRATA | Docs Contact: | |||||
| Severity: | high | ||||||
| Priority: | medium | CC: | coli, jinzhao, juzhang, kwolf, lijin, mdeng, vgoyal, virt-maint, zhguo | ||||
| Version: | 9.2 | Keywords: | Regression, Triaged | ||||
| Target Milestone: | rc | ||||||
| Target Release: | --- | ||||||
| Hardware: | Unspecified | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | qemu-kvm-7.2.0-10.el9 | Doc Type: | If docs needed, set a value | ||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2023-05-09 07:23:46 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Attachments: |
|
||||||
Hi,Stefano Testing the patch now, will give a feedback after weekend. Thanks, Aliang Test on qemu-kvm-7.2.0-8.el9.stefano202302091814, still hit the core dump issue. Executable: /usr/libexec/qemu-kvm
Control Group: /user.slice/user-0.slice/session-18.scope
Unit: session-18.scope
Slice: user-0.slice
Session: 18
Owner UID: 0 (root)
Boot ID: 52c869c21ef64de49877ac0eed7aeb06
Machine ID: 3919555703fd4043b7f3cc2611ad4d18
Hostname: dell-per740xd-01.lab.eng.pek2.redhat.com
Storage: /var/lib/systemd/coredump/core.qemu-kvm.0.52c869c21ef64de49877ac0eed7aeb06.868109.1676028189000000.zst (present)
Size on Disk: 597.0M
Message: Process 868109 (qemu-kvm) of user 0 dumped core.
Stack trace of thread 868109:
#0 0x00007f5dcf6a154c __pthread_kill_implementation (libc.so.6 + 0xa154c)
#1 0x00007f5dcf654d46 raise (libc.so.6 + 0x54d46)
#2 0x00007f5dcf6287f3 abort (libc.so.6 + 0x287f3)
#3 0x000055a8f91c15a2 qemu_mutex_unlock_impl (qemu-kvm + 0x9c15a2)
#4 0x000055a8f901d2f7 bdrv_do_drained_begin (qemu-kvm + 0x81d2f7)
#5 0x000055a8f8fe630e bdrv_replace_node_noperm (qemu-kvm + 0x7e630e)
#6 0x000055a8f8fe6162 bdrv_append (qemu-kvm + 0x7e6162)
#7 0x000055a8f8fccafc external_snapshot_prepare (qemu-kvm + 0x7ccafc)
#8 0x000055a8f8fcb3ad qmp_transaction (qemu-kvm + 0x7cb3ad)
#9 0x000055a8f90e1de6 qmp_marshal_blockdev_snapshot (qemu-kvm + 0x8e1de6)
#10 0x000055a8f91b19a2 do_qmp_dispatch_bh (qemu-kvm + 0x9b19a2)
#11 0x000055a8f91bd9a1 aio_dispatch (qemu-kvm + 0x9bd9a1)
#12 0x000055a8f91d8652 aio_ctx_dispatch (qemu-kvm + 0x9d8652)
#13 0x00007f5dcfc83e2f g_main_context_dispatch (libglib-2.0.so.0 + 0x54e2f)
#14 0x000055a8f91d9f74 main_loop_wait (qemu-kvm + 0x9d9f74)
#15 0x000055a8f8ce08e7 qemu_main_loop (qemu-kvm + 0x4e08e7)
#16 0x000055a8f8b6692a qemu_default_main (qemu-kvm + 0x36692a)
#17 0x00007f5dcf63feb0 __libc_start_call_main (libc.so.6 + 0x3feb0)
#18 0x00007f5dcf63ff60 __libc_start_main@@GLIBC_2.34 (libc.so.6 + 0x3ff60)
#19 0x000055a8f8b66085 _start (qemu-kvm + 0x366085)
Stack trace of thread 868116:
#0 0x00007f5dcf7429bf __poll (libc.so.6 + 0x1429bf)
#1 0x00007f5dcfcd849c g_main_context_iterate.constprop.0 (libglib-2.0.so.0 + 0xa949c)
#2 0x00007f5dcfc83483 g_main_loop_run (libglib-2.0.so.0 + 0x54483)
#3 0x000055a8f8fd42ff iothread_run (qemu-kvm + 0x7d42ff)
#4 0x000055a8f91c239a qemu_thread_start (qemu-kvm + 0x9c239a)
#5 0x00007f5dcf69f802 start_thread (libc.so.6 + 0x9f802)
#6 0x00007f5dcf63f450 __clone3 (libc.so.6 + 0x3f450)
Stack trace of thread 868123:
#0 0x00007f5dcf63ec6b ioctl (libc.so.6 + 0x3ec6b)
#1 0x000055a8f8f7aa5b kvm_vcpu_ioctl (qemu-kvm + 0x77aa5b)
#2 0x000055a8f8f80261 kvm_cpu_exec (qemu-kvm + 0x780261)
#3 0x000055a8f8f8285a kvm_vcpu_thread_fn (qemu-kvm + 0x78285a)
#4 0x000055a8f91c239a qemu_thread_start (qemu-kvm + 0x9c239a)
#5 0x00007f5dcf69f802 start_thread (libc.so.6 + 0x9f802)
#6 0x00007f5dcf63f450 __clone3 (libc.so.6 + 0x3f450)
Stack trace of thread 868120:
#0 0x00007f5dcf63ec6b ioctl (libc.so.6 + 0x3ec6b)
#1 0x000055a8f8f7aa5b kvm_vcpu_ioctl (qemu-kvm + 0x77aa5b)
#2 0x000055a8f8f80261 kvm_cpu_exec (qemu-kvm + 0x780261)
#3 0x000055a8f8f8285a kvm_vcpu_thread_fn (qemu-kvm + 0x78285a)
#4 0x000055a8f91c239a qemu_thread_start (qemu-kvm + 0x9c239a)
#5 0x00007f5dcf69f802 start_thread (libc.so.6 + 0x9f802)
#6 0x00007f5dcf63f450 __clone3 (libc.so.6 + 0x3f450)
Stack trace of thread 868130:
#0 0x00007f5dcf69c39a __futex_abstimed_wait_common (libc.so.6 + 0x9c39a)
#1 0x00007f5dcf69eba0 pthread_cond_wait@@GLIBC_2.3.2 (libc.so.6 + 0x9eba0)
#2 0x000055a8f91c194f qemu_cond_wait_impl (qemu-kvm + 0x9c194f)
#3 0x000055a8f8b9cc76 vnc_worker_thread (qemu-kvm + 0x39cc76)
#4 0x000055a8f91c239a qemu_thread_start (qemu-kvm + 0x9c239a)
#5 0x00007f5dcf69f802 start_thread (libc.so.6 + 0x9f802)
#6 0x00007f5dcf63f450 __clone3 (libc.so.6 + 0x3f450)
Stack trace of thread 868200:
#0 0x00007f5dcf69c39a __futex_abstimed_wait_common (libc.so.6 + 0x9c39a)
#1 0x00007f5dcf69eea4 pthread_cond_timedwait@@GLIBC_2.3.2 (libc.so.6 + 0x9eea4)
#2 0x000055a8f91c1aec qemu_cond_timedwait_ts (qemu-kvm + 0x9c1aec)
#3 0x000055a8f91c1a90 qemu_cond_timedwait_impl (qemu-kvm + 0x9c1a90)
#4 0x000055a8f91dc857 worker_thread (qemu-kvm + 0x9dc857)
#5 0x000055a8f91c239a qemu_thread_start (qemu-kvm + 0x9c239a)
#6 0x00007f5dcf69f802 start_thread (libc.so.6 + 0x9f802)
#7 0x00007f5dcf63f450 __clone3 (libc.so.6 + 0x3f450)
Stack trace of thread 868111:
#0 0x00007f5dcf742abe ppoll (libc.so.6 + 0x142abe)
#1 0x000055a8f91bee8e fdmon_poll_wait (qemu-kvm + 0x9bee8e)
#2 0x000055a8f91be0ce aio_poll (qemu-kvm + 0x9be0ce)
#3 0x000055a8f8fd42e2 iothread_run (qemu-kvm + 0x7d42e2)
#4 0x000055a8f91c239a qemu_thread_start (qemu-kvm + 0x9c239a)
#5 0x00007f5dcf69f802 start_thread (libc.so.6 + 0x9f802)
#6 0x00007f5dcf63f450 __clone3 (libc.so.6 + 0x3f450)
Stack trace of thread 868201:
#0 0x00007f5dcf69c39a __futex_abstimed_wait_common (libc.so.6 + 0x9c39a)
#1 0x00007f5dcf69eea4 pthread_cond_timedwait@@GLIBC_2.3.2 (libc.so.6 + 0x9eea4)
#2 0x000055a8f91c1aec qemu_cond_timedwait_ts (qemu-kvm + 0x9c1aec)
#3 0x000055a8f91c1a90 qemu_cond_timedwait_impl (qemu-kvm + 0x9c1a90)
#4 0x000055a8f91dc857 worker_thread (qemu-kvm + 0x9dc857)
#5 0x000055a8f91c239a qemu_thread_start (qemu-kvm + 0x9c239a)
#6 0x00007f5dcf69f802 start_thread (libc.so.6 + 0x9f802)
#7 0x00007f5dcf63f450 __clone3 (libc.so.6 + 0x3f450)
Stack trace of thread 868224:
#0 0x00007f5dcf69c39a __futex_abstimed_wait_common (libc.so.6 + 0x9c39a)
#1 0x00007f5dcf69eea4 pthread_cond_timedwait@@GLIBC_2.3.2 (libc.so.6 + 0x9eea4)
#2 0x000055a8f91c1aec qemu_cond_timedwait_ts (qemu-kvm + 0x9c1aec)
#3 0x000055a8f91c1a90 qemu_cond_timedwait_impl (qemu-kvm + 0x9c1a90)
#4 0x000055a8f91dc857 worker_thread (qemu-kvm + 0x9dc857)
#5 0x000055a8f91c239a qemu_thread_start (qemu-kvm + 0x9c239a)
#6 0x00007f5dcf69f802 start_thread (libc.so.6 + 0x9f802)
#7 0x00007f5dcf63f450 __clone3 (libc.so.6 + 0x3f450)
Stack trace of thread 868126:
#0 0x00007f5dcf63ec6b ioctl (libc.so.6 + 0x3ec6b)
#1 0x000055a8f8f7aa5b kvm_vcpu_ioctl (qemu-kvm + 0x77aa5b)
#2 0x000055a8f8f80261 kvm_cpu_exec (qemu-kvm + 0x780261)
#3 0x000055a8f8f8285a kvm_vcpu_thread_fn (qemu-kvm + 0x78285a)
#4 0x000055a8f91c239a qemu_thread_start (qemu-kvm + 0x9c239a)
#5 0x00007f5dcf69f802 start_thread (libc.so.6 + 0x9f802)
#6 0x00007f5dcf63f450 __clone3 (libc.so.6 + 0x3f450)
Stack trace of thread 868124:
#0 0x00007f5dcf63ec6b ioctl (libc.so.6 + 0x3ec6b)
#1 0x000055a8f8f7aa5b kvm_vcpu_ioctl (qemu-kvm + 0x77aa5b)
#2 0x000055a8f8f80261 kvm_cpu_exec (qemu-kvm + 0x780261)
#3 0x000055a8f8f8285a kvm_vcpu_thread_fn (qemu-kvm + 0x78285a)
#4 0x000055a8f91c239a qemu_thread_start (qemu-kvm + 0x9c239a)
#5 0x00007f5dcf69f802 start_thread (libc.so.6 + 0x9f802)
#6 0x00007f5dcf63f450 __clone3 (libc.so.6 + 0x3f450)
Stack trace of thread 868227:
#0 0x00007f5dcf69c39a __futex_abstimed_wait_common (libc.so.6 + 0x9c39a)
#1 0x00007f5dcf69eea4 pthread_cond_timedwait@@GLIBC_2.3.2 (libc.so.6 + 0x9eea4)
#2 0x000055a8f91c1aec qemu_cond_timedwait_ts (qemu-kvm + 0x9c1aec)
#3 0x000055a8f91c1a90 qemu_cond_timedwait_impl (qemu-kvm + 0x9c1a90)
#4 0x000055a8f91dc857 worker_thread (qemu-kvm + 0x9dc857)
#5 0x000055a8f91c239a qemu_thread_start (qemu-kvm + 0x9c239a)
#6 0x00007f5dcf69f802 start_thread (libc.so.6 + 0x9f802)
#7 0x00007f5dcf63f450 __clone3 (libc.so.6 + 0x3f450)
Stack trace of thread 868117:
#0 0x00007f5dcf63ec6b ioctl (libc.so.6 + 0x3ec6b)
#1 0x000055a8f8f7aa5b kvm_vcpu_ioctl (qemu-kvm + 0x77aa5b)
#2 0x000055a8f8f80261 kvm_cpu_exec (qemu-kvm + 0x780261)
#3 0x000055a8f8f8285a kvm_vcpu_thread_fn (qemu-kvm + 0x78285a)
#4 0x000055a8f91c239a qemu_thread_start (qemu-kvm + 0x9c239a)
#5 0x00007f5dcf69f802 start_thread (libc.so.6 + 0x9f802)
#6 0x00007f5dcf63f450 __clone3 (libc.so.6 + 0x3f450)
Stack trace of thread 868119:
#0 0x00007f5dcf63ec6b ioctl (libc.so.6 + 0x3ec6b)
#1 0x000055a8f8f7aa5b kvm_vcpu_ioctl (qemu-kvm + 0x77aa5b)
#2 0x000055a8f8f80261 kvm_cpu_exec (qemu-kvm + 0x780261)
#3 0x000055a8f8f8285a kvm_vcpu_thread_fn (qemu-kvm + 0x78285a)
#4 0x000055a8f91c239a qemu_thread_start (qemu-kvm + 0x9c239a)
#5 0x00007f5dcf69f802 start_thread (libc.so.6 + 0x9f802)
#6 0x00007f5dcf63f450 __clone3 (libc.so.6 + 0x3f450)
Stack trace of thread 868118:
#0 0x00007f5dcf63ec6b ioctl (libc.so.6 + 0x3ec6b)
#1 0x000055a8f8f7aa5b kvm_vcpu_ioctl (qemu-kvm + 0x77aa5b)
#2 0x000055a8f8f80261 kvm_cpu_exec (qemu-kvm + 0x780261)
#3 0x000055a8f8f8285a kvm_vcpu_thread_fn (qemu-kvm + 0x78285a)
#4 0x000055a8f91c239a qemu_thread_start (qemu-kvm + 0x9c239a)
#5 0x00007f5dcf69f802 start_thread (libc.so.6 + 0x9f802)
#6 0x00007f5dcf63f450 __clone3 (libc.so.6 + 0x3f450)
Stack trace of thread 868230:
#0 0x00007f5dcf69c39a __futex_abstimed_wait_common (libc.so.6 + 0x9c39a)
#1 0x00007f5dcf69eea4 pthread_cond_timedwait@@GLIBC_2.3.2 (libc.so.6 + 0x9eea4)
#2 0x000055a8f91c1aec qemu_cond_timedwait_ts (qemu-kvm + 0x9c1aec)
#3 0x000055a8f91c1a90 qemu_cond_timedwait_impl (qemu-kvm + 0x9c1a90)
#4 0x000055a8f91dc857 worker_thread (qemu-kvm + 0x9dc857)
#5 0x000055a8f91c239a qemu_thread_start (qemu-kvm + 0x9c239a)
#6 0x00007f5dcf69f802 start_thread (libc.so.6 + 0x9f802)
#7 0x00007f5dcf63f450 __clone3 (libc.so.6 + 0x3f450)
Stack trace of thread 868231:
#0 0x00007f5dcf69c39a __futex_abstimed_wait_common (libc.so.6 + 0x9c39a)
#1 0x00007f5dcf69eea4 pthread_cond_timedwait@@GLIBC_2.3.2 (libc.so.6 + 0x9eea4)
#2 0x000055a8f91c1aec qemu_cond_timedwait_ts (qemu-kvm + 0x9c1aec)
#3 0x000055a8f91c1a90 qemu_cond_timedwait_impl (qemu-kvm + 0x9c1a90)
#4 0x000055a8f91dc857 worker_thread (qemu-kvm + 0x9dc857)
#5 0x000055a8f91c239a qemu_thread_start (qemu-kvm + 0x9c239a)
#6 0x00007f5dcf69f802 start_thread (libc.so.6 + 0x9f802)
#7 0x00007f5dcf63f450 __clone3 (libc.so.6 + 0x3f450)
Stack trace of thread 868110:
#0 0x00007f5dcf63ee5d syscall (libc.so.6 + 0x3ee5d)
#1 0x000055a8f91c20ef qemu_event_wait (qemu-kvm + 0x9c20ef)
#2 0x000055a8f91ce225 call_rcu_thread (qemu-kvm + 0x9ce225)
#3 0x000055a8f91c239a qemu_thread_start (qemu-kvm + 0x9c239a)
#4 0x00007f5dcf69f802 start_thread (libc.so.6 + 0x9f802)
#5 0x00007f5dcf63f450 __clone3 (libc.so.6 + 0x3f450)
Stack trace of thread 868121:
#0 0x00007f5dcf63ec6b ioctl (libc.so.6 + 0x3ec6b)
#1 0x000055a8f8f7aa5b kvm_vcpu_ioctl (qemu-kvm + 0x77aa5b)
#2 0x000055a8f8f80261 kvm_cpu_exec (qemu-kvm + 0x780261)
#3 0x000055a8f8f8285a kvm_vcpu_thread_fn (qemu-kvm + 0x78285a)
#4 0x000055a8f91c239a qemu_thread_start (qemu-kvm + 0x9c239a)
#5 0x00007f5dcf69f802 start_thread (libc.so.6 + 0x9f802)
#6 0x00007f5dcf63f450 __clone3 (libc.so.6 + 0x3f450)
Stack trace of thread 868243:
#0 0x00007f5dcf69c39a __futex_abstimed_wait_common (libc.so.6 + 0x9c39a)
#1 0x00007f5dcf69eea4 pthread_cond_timedwait@@GLIBC_2.3.2 (libc.so.6 + 0x9eea4)
#2 0x000055a8f91c1aec qemu_cond_timedwait_ts (qemu-kvm + 0x9c1aec)
#3 0x000055a8f91c1a90 qemu_cond_timedwait_impl (qemu-kvm + 0x9c1a90)
#4 0x000055a8f91dc857 worker_thread (qemu-kvm + 0x9dc857)
#5 0x000055a8f91c239a qemu_thread_start (qemu-kvm + 0x9c239a)
#6 0x00007f5dcf69f802 start_thread (libc.so.6 + 0x9f802)
#7 0x00007f5dcf63f450 __clone3 (libc.so.6 + 0x3f450)
Stack trace of thread 868128:
#0 0x00007f5dcf63ec6b ioctl (libc.so.6 + 0x3ec6b)
#1 0x000055a8f8f7aa5b kvm_vcpu_ioctl (qemu-kvm + 0x77aa5b)
#2 0x000055a8f8f80261 kvm_cpu_exec (qemu-kvm + 0x780261)
#3 0x000055a8f8f8285a kvm_vcpu_thread_fn (qemu-kvm + 0x78285a)
#4 0x000055a8f91c239a qemu_thread_start (qemu-kvm + 0x9c239a)
#5 0x00007f5dcf69f802 start_thread (libc.so.6 + 0x9f802)
#6 0x00007f5dcf63f450 __clone3 (libc.so.6 + 0x3f450)
Stack trace of thread 868225:
#0 0x00007f5dcf69c39a __futex_abstimed_wait_common (libc.so.6 + 0x9c39a)
#1 0x00007f5dcf69eea4 pthread_cond_timedwait@@GLIBC_2.3.2 (libc.so.6 + 0x9eea4)
#2 0x000055a8f91c1aec qemu_cond_timedwait_ts (qemu-kvm + 0x9c1aec)
#3 0x000055a8f91c1a90 qemu_cond_timedwait_impl (qemu-kvm + 0x9c1a90)
#4 0x000055a8f91dc857 worker_thread (qemu-kvm + 0x9dc857)
#5 0x000055a8f91c239a qemu_thread_start (qemu-kvm + 0x9c239a)
#6 0x00007f5dcf69f802 start_thread (libc.so.6 + 0x9f802)
#7 0x00007f5dcf63f450 __clone3 (libc.so.6 + 0x3f450)
Stack trace of thread 868271:
#0 0x00007f5dcf69c39a __futex_abstimed_wait_common (libc.so.6 + 0x9c39a)
#1 0x00007f5dcf69eea4 pthread_cond_timedwait@@GLIBC_2.3.2 (libc.so.6 + 0x9eea4)
#2 0x000055a8f91c1aec qemu_cond_timedwait_ts (qemu-kvm + 0x9c1aec)
#3 0x000055a8f91c1a90 qemu_cond_timedwait_impl (qemu-kvm + 0x9c1a90)
#4 0x000055a8f91dc857 worker_thread (qemu-kvm + 0x9dc857)
#5 0x000055a8f91c239a qemu_thread_start (qemu-kvm + 0x9c239a)
#6 0x00007f5dcf69f802 start_thread (libc.so.6 + 0x9f802)
#7 0x00007f5dcf63f450 __clone3 (libc.so.6 + 0x3f450)
Stack trace of thread 868226:
#0 0x00007f5dcf69c39a __futex_abstimed_wait_common (libc.so.6 + 0x9c39a)
#1 0x00007f5dcf69eea4 pthread_cond_timedwait@@GLIBC_2.3.2 (libc.so.6 + 0x9eea4)
#2 0x000055a8f91c1aec qemu_cond_timedwait_ts (qemu-kvm + 0x9c1aec)
#3 0x000055a8f91c1a90 qemu_cond_timedwait_impl (qemu-kvm + 0x9c1a90)
#4 0x000055a8f91dc857 worker_thread (qemu-kvm + 0x9dc857)
#5 0x000055a8f91c239a qemu_thread_start (qemu-kvm + 0x9c239a)
#6 0x00007f5dcf69f802 start_thread (libc.so.6 + 0x9f802)
#7 0x00007f5dcf63f450 __clone3 (libc.so.6 + 0x3f450)
Stack trace of thread 868239:
#0 0x00007f5dcf69c39a __futex_abstimed_wait_common (libc.so.6 + 0x9c39a)
#1 0x00007f5dcf69eea4 pthread_cond_timedwait@@GLIBC_2.3.2 (libc.so.6 + 0x9eea4)
#2 0x000055a8f91c1aec qemu_cond_timedwait_ts (qemu-kvm + 0x9c1aec)
#3 0x000055a8f91c1a90 qemu_cond_timedwait_impl (qemu-kvm + 0x9c1a90)
#4 0x000055a8f91dc857 worker_thread (qemu-kvm + 0x9dc857)
#5 0x000055a8f91c239a qemu_thread_start (qemu-kvm + 0x9c239a)
#6 0x00007f5dcf69f802 start_thread (libc.so.6 + 0x9f802)
#7 0x00007f5dcf63f450 __clone3 (libc.so.6 + 0x3f450)
Stack trace of thread 868242:
#0 0x00007f5dcf69c39a __futex_abstimed_wait_common (libc.so.6 + 0x9c39a)
#1 0x00007f5dcf69eea4 pthread_cond_timedwait@@GLIBC_2.3.2 (libc.so.6 + 0x9eea4)
#2 0x000055a8f91c1aec qemu_cond_timedwait_ts (qemu-kvm + 0x9c1aec)
#3 0x000055a8f91c1a90 qemu_cond_timedwait_impl (qemu-kvm + 0x9c1a90)
#4 0x000055a8f91dc857 worker_thread (qemu-kvm + 0x9dc857)
#5 0x000055a8f91c239a qemu_thread_start (qemu-kvm + 0x9c239a)
#6 0x00007f5dcf69f802 start_thread (libc.so.6 + 0x9f802)
#7 0x00007f5dcf63f450 __clone3 (libc.so.6 + 0x3f450)
Stack trace of thread 868122:
#0 0x00007f5dcf63ec6b ioctl (libc.so.6 + 0x3ec6b)
#1 0x000055a8f8f7aa5b kvm_vcpu_ioctl (qemu-kvm + 0x77aa5b)
#2 0x000055a8f8f80261 kvm_cpu_exec (qemu-kvm + 0x780261)
#3 0x000055a8f8f8285a kvm_vcpu_thread_fn (qemu-kvm + 0x78285a)
#4 0x000055a8f91c239a qemu_thread_start (qemu-kvm + 0x9c239a)
#5 0x00007f5dcf69f802 start_thread (libc.so.6 + 0x9f802)
#6 0x00007f5dcf63f450 __clone3 (libc.so.6 + 0x3f450)
Stack trace of thread 868240:
#0 0x00007f5dcf69c39a __futex_abstimed_wait_common (libc.so.6 + 0x9c39a)
#1 0x00007f5dcf69eea4 pthread_cond_timedwait@@GLIBC_2.3.2 (libc.so.6 + 0x9eea4)
#2 0x000055a8f91c1aec qemu_cond_timedwait_ts (qemu-kvm + 0x9c1aec)
#3 0x000055a8f91c1a90 qemu_cond_timedwait_impl (qemu-kvm + 0x9c1a90)
#4 0x000055a8f91dc857 worker_thread (qemu-kvm + 0x9dc857)
#5 0x000055a8f91c239a qemu_thread_start (qemu-kvm + 0x9c239a)
#6 0x00007f5dcf69f802 start_thread (libc.so.6 + 0x9f802)
#7 0x00007f5dcf63f450 __clone3 (libc.so.6 + 0x3f450)
Stack trace of thread 868323:
#0 0x00007f5dcf69c39a __futex_abstimed_wait_common (libc.so.6 + 0x9c39a)
#1 0x00007f5dcf69eea4 pthread_cond_timedwait@@GLIBC_2.3.2 (libc.so.6 + 0x9eea4)
#2 0x000055a8f91c1aec qemu_cond_timedwait_ts (qemu-kvm + 0x9c1aec)
#3 0x000055a8f91c1a90 qemu_cond_timedwait_impl (qemu-kvm + 0x9c1a90)
#4 0x000055a8f91dc857 worker_thread (qemu-kvm + 0x9dc857)
#5 0x000055a8f91c239a qemu_thread_start (qemu-kvm + 0x9c239a)
#6 0x00007f5dcf69f802 start_thread (libc.so.6 + 0x9f802)
#7 0x00007f5dcf63f450 __clone3 (libc.so.6 + 0x3f450)
Stack trace of thread 868127:
#0 0x00007f5dcf63ec6b ioctl (libc.so.6 + 0x3ec6b)
#1 0x000055a8f8f7aa5b kvm_vcpu_ioctl (qemu-kvm + 0x77aa5b)
#2 0x000055a8f8f80261 kvm_cpu_exec (qemu-kvm + 0x780261)
#3 0x000055a8f8f8285a kvm_vcpu_thread_fn (qemu-kvm + 0x78285a)
#4 0x000055a8f91c239a qemu_thread_start (qemu-kvm + 0x9c239a)
#5 0x00007f5dcf69f802 start_thread (libc.so.6 + 0x9f802)
#6 0x00007f5dcf63f450 __clone3 (libc.so.6 + 0x3f450)
Stack trace of thread 868235:
#0 0x00007f5dcf69c39a __futex_abstimed_wait_common (libc.so.6 + 0x9c39a)
#1 0x00007f5dcf69eea4 pthread_cond_timedwait@@GLIBC_2.3.2 (libc.so.6 + 0x9eea4)
#2 0x000055a8f91c1aec qemu_cond_timedwait_ts (qemu-kvm + 0x9c1aec)
#3 0x000055a8f91c1a90 qemu_cond_timedwait_impl (qemu-kvm + 0x9c1a90)
#4 0x000055a8f91dc857 worker_thread (qemu-kvm + 0x9dc857)
#5 0x000055a8f91c239a qemu_thread_start (qemu-kvm + 0x9c239a)
#6 0x00007f5dcf69f802 start_thread (libc.so.6 + 0x9f802)
#7 0x00007f5dcf63f450 __clone3 (libc.so.6 + 0x3f450)
Stack trace of thread 868320:
#0 0x00007f5dcf69c39a __futex_abstimed_wait_common (libc.so.6 + 0x9c39a)
#1 0x00007f5dcf69eea4 pthread_cond_timedwait@@GLIBC_2.3.2 (libc.so.6 + 0x9eea4)
#2 0x000055a8f91c1aec qemu_cond_timedwait_ts (qemu-kvm + 0x9c1aec)
#3 0x000055a8f91c1a90 qemu_cond_timedwait_impl (qemu-kvm + 0x9c1a90)
#4 0x000055a8f91dc857 worker_thread (qemu-kvm + 0x9dc857)
#5 0x000055a8f91c239a qemu_thread_start (qemu-kvm + 0x9c239a)
#6 0x00007f5dcf69f802 start_thread (libc.so.6 + 0x9f802)
#7 0x00007f5dcf63f450 __clone3 (libc.so.6 + 0x3f450)
Stack trace of thread 868241:
#0 0x00007f5dcf69c39a __futex_abstimed_wait_common (libc.so.6 + 0x9c39a)
#1 0x00007f5dcf69eea4 pthread_cond_timedwait@@GLIBC_2.3.2 (libc.so.6 + 0x9eea4)
#2 0x000055a8f91c1aec qemu_cond_timedwait_ts (qemu-kvm + 0x9c1aec)
#3 0x000055a8f91c1a90 qemu_cond_timedwait_impl (qemu-kvm + 0x9c1a90)
#4 0x000055a8f91dc857 worker_thread (qemu-kvm + 0x9dc857)
#5 0x000055a8f91c239a qemu_thread_start (qemu-kvm + 0x9c239a)
#6 0x00007f5dcf69f802 start_thread (libc.so.6 + 0x9f802)
#7 0x00007f5dcf63f450 __clone3 (libc.so.6 + 0x3f450)
Stack trace of thread 868125:
#0 0x00007f5dcf63ec6b ioctl (libc.so.6 + 0x3ec6b)
#1 0x000055a8f8f7aa5b kvm_vcpu_ioctl (qemu-kvm + 0x77aa5b)
#2 0x000055a8f8f80261 kvm_cpu_exec (qemu-kvm + 0x780261)
#3 0x000055a8f8f8285a kvm_vcpu_thread_fn (qemu-kvm + 0x78285a)
#4 0x000055a8f91c239a qemu_thread_start (qemu-kvm + 0x9c239a)
#5 0x00007f5dcf69f802 start_thread (libc.so.6 + 0x9f802)
#6 0x00007f5dcf63f450 __clone3 (libc.so.6 + 0x3f450)
Stack trace of thread 868322:
#0 0x00007f5dcf69c39a __futex_abstimed_wait_common (libc.so.6 + 0x9c39a)
#1 0x00007f5dcf69eea4 pthread_cond_timedwait@@GLIBC_2.3.2 (libc.so.6 + 0x9eea4)
#2 0x000055a8f91c1aec qemu_cond_timedwait_ts (qemu-kvm + 0x9c1aec)
#3 0x000055a8f91c1a90 qemu_cond_timedwait_impl (qemu-kvm + 0x9c1a90)
#4 0x000055a8f91dc857 worker_thread (qemu-kvm + 0x9dc857)
#5 0x000055a8f91c239a qemu_thread_start (qemu-kvm + 0x9c239a)
#6 0x00007f5dcf69f802 start_thread (libc.so.6 + 0x9f802)
#7 0x00007f5dcf63f450 __clone3 (libc.so.6 + 0x3f450)
ELF object binary architecture: AMD x86-64
@aliang about the trace in comment 6, is there something printed on the stderr? (In reply to Stefano Garzarella from comment #8) > @aliang about the trace in comment 6, is there something printed > on the stderr? Hi, Stefano The printed stderr still be "qemu: qemu_mutex_unlock_impl: Operation not permitted". I cleared the history log today, if you need, I can re-run it and provide a complete one. BR, Aliang Created attachment 1944059 [details]
local reproducer
Possible fix posted upstream: https://lore.kernel.org/qemu-devel/20230214105156.316586-1-sgarzare@redhat.com QE bot(pre verify): Set 'Verified:Tested,SanityOnly' as gating/tier1 test pass. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: qemu-kvm security, bug fix, and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2023:2162 |
Description of problem: Qemu coredump after do snapshot of mirrored top image and its converted base image(iothread enabled) Version-Release number of selected component (if applicable): kernel version:5.14.0-212.el9.x86_64 qemu-kvm version:qemu-kvm-7.2.0-6.el9 How reproducible: less than 20%, random Steps to Reproduce: 1.Start guest with qemu cmd: /usr/libexec/qemu-kvm \ -S \ -name 'avocado-vt-vm1' \ -sandbox on \ -blockdev node-name=file_ovmf_code,driver=file,filename=/usr/share/OVMF/OVMF_CODE.secboot.fd,auto-read-only=on,discard=unmap \ -blockdev node-name=drive_ovmf_code,driver=raw,read-only=on,file=file_ovmf_code \ -blockdev node-name=file_ovmf_vars,driver=file,filename=/root/avocado/data/avocado-vt/avocado-vt-vm1_rhel920-64-virtio-scsi_qcow2_filesystem_VARS.fd,auto-read-only=on,discard=unmap \ -blockdev node-name=drive_ovmf_vars,driver=raw,read-only=off,file=file_ovmf_vars \ -machine q35,memory-backend=mem-machine_mem,pflash0=drive_ovmf_code,pflash1=drive_ovmf_vars \ -device pcie-root-port,id=pcie-root-port-0,multifunction=on,bus=pcie.0,addr=0x1,chassis=1 \ -device pcie-pci-bridge,id=pcie-pci-bridge-0,addr=0x0,bus=pcie-root-port-0 \ -nodefaults \ -m 30720 \ -object '{"qom-type": "memory-backend-ram", "size": 32212254720, "id": "mem-machine_mem"}' \ -smp 10,maxcpus=10,cores=5,threads=1,dies=1,sockets=2 \ -cpu 'Cascadelake-Server-noTSX',+kvm_pv_unhalt \ -chardev socket,path=/var/tmp/avocado_pzd6v87x/monitor-qmpmonitor1-20221219-011444-yoM1Fk2J,wait=off,server=on,id=qmp_id_qmpmonitor1 \ -mon chardev=qmp_id_qmpmonitor1,mode=control \ -chardev socket,path=/var/tmp/avocado_pzd6v87x/monitor-catch_monitor-20221219-011444-yoM1Fk2J,wait=off,server=on,id=qmp_id_catch_monitor \ -mon chardev=qmp_id_catch_monitor,mode=control \ -device pvpanic,ioport=0x505,id=iddutSce \ -chardev socket,path=/var/tmp/avocado_pzd6v87x/serial-serial0-20221219-011444-yoM1Fk2J,wait=off,server=on,id=chardev_serial0 \ -device isa-serial,id=serial0,chardev=chardev_serial0 \ -chardev socket,id=seabioslog_id_20221219-011444-yoM1Fk2J,path=/var/tmp/avocado_pzd6v87x/seabios-20221219-011444-yoM1Fk2J,server=on,wait=off \ -device isa-debugcon,chardev=seabioslog_id_20221219-011444-yoM1Fk2J,iobase=0x402 \ -device pcie-root-port,id=pcie-root-port-1,port=0x1,addr=0x1.0x1,bus=pcie.0,chassis=2 \ -device qemu-xhci,id=usb1,bus=pcie-root-port-1,addr=0x0 \ -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 \ -object '{"qom-type": "iothread", "id": "iothread0"}' \ -device pcie-root-port,id=pcie-root-port-2,port=0x2,addr=0x1.0x2,bus=pcie.0,chassis=3 \ -device '{"id": "virtio_scsi_pci0", "driver": "virtio-scsi-pci", "bus": "pcie-root-port-2", "addr": "0x0", "iothread": "iothread0"}' \ -blockdev '{"node-name": "file_image1", "driver": "file", "auto-read-only": true, "discard": "unmap", "aio": "threads", "filename": "/home/kvm_autotest_root/images/rhel920-64-virtio-scsi.qcow2", "cache": {"direct": true, "no-flush": false}}' \ -blockdev '{"node-name": "drive_image1", "driver": "qcow2", "read-only": false, "cache": {"direct": true, "no-flush": false}, "file": "file_image1"}' \ -device '{"driver": "scsi-hd", "id": "image1", "drive": "drive_image1", -blockdev '{"node-name": "file_data1", "driver": "file", "auto-read-only": true, "discard": "unmap", "aio": "threads", "filename": "/root/avocado/data/avocado-vt/data1.qcow2", "cache": {"direct": true, "no-flush": false}}' \ -blockdev '{"node-name": "drive_data1", "driver": "qcow2", "read-only": false, "cache": {"direct": true, "no-flush": false}, "file": "file_data1"}' \ -device '{"driver": "scsi-hd", "id": "data1", "drive": "drive_data1", "write-cache": "on"}' \ -device pcie-root-port,id=pcie-root-port-3,port=0x3,addr=0x1.0x3,bus=pcie.0,chassis=4 \ -device virtio-net-pci,mac=9a:36:b9:70:1e:73,id=idGMJM6z,netdev=idDwXrrb,bus=pcie-root-port-3,addr=0x0 \ -netdev tap,id=idDwXrrb,vhost=on,vhostfd=16,fd=9 \ -vnc :0 \ -rtc base=utc,clock=host,driftfix=slew \ -boot menu=off,order=cdn,once=c,strict=off \ -chardev socket,id=char_vtpm_avocado-vt-vm1_tpm0,path=/root/avocado/data/avocado-vt/swtpm/avocado-vt-vm1_tpm0_swtpm.sock \ -tpmdev emulator,chardev=char_vtpm_avocado-vt-vm1_tpm0,id=emulator_vtpm_avocado-vt-vm1_tpm0 \ -device tpm-crb,id=tpm-crb_vtpm_avocado-vt-vm1_tpm0,tpmdev=emulator_vtpm_avocado-vt-vm1_tpm0 \ -enable-kvm \ -device pcie-root-port,id=pcie_extra_root_port_0,multifunction=on,bus=pcie.0,addr=0x3,chassis=5 2.Continue guest, after guest is up, write some data in data disk. {'execute': 'cont', 'id': 'eQ1yWMQI'} (guest)#parted -s "/dev/sdb" mkpart primary 0M 2048.0M #yes|mkfs.ext4 -F '/dev/sdb1' #mkdir /mnt/sdb1 #mount -t ext4 /dev/sdb1 /mnt/sdb1 #dd if=/dev/urandom of=/mnt/sdb1/qGmy bs=1M count=10 oflag=direct #md5sum /mnt/sdb1/qGmy > /mnt/sdb1/qGmy.md5 && sync 3.Create snapshot of data disk, and add it. #qemu-img create -f qcow2 -b /root/avocado/data/avocado-vt/data1.qcow2 -F qcow2 /root/avocado/data/avocado-vt/data1sn.qcow2 2G {'execute': 'blockdev-add', 'arguments': {'node-name': 'file_data1sn', 'driver': 'file', 'auto-read-only': True, 'discard': 'unmap', 'aio': 'threads', 'filename': '/root/avocado/data/avocado-vt/data1sn.qcow2', 'cache': {'direct': True, 'no-flush': False}}, 'id': 'NEUuPONk'} {'execute': 'blockdev-add', 'arguments': {'node-name': 'drive_data1sn', 'driver': 'qcow2', 'read-only': False, 'cache': {'direct': True, 'no-flush': False}, 'file': 'file_data1sn', 'backing': None}, 'id': '3b56DU3G'} 4.Do snapshot of data to snapshot file. {'execute': 'blockdev-snapshot', 'arguments': {'node': 'drive_data1', 'overlay': 'drive_data1sn'}, 'id': 'nyuVOuu2'} 5. Write some new data on snapshot file. (guest)#dd if=/dev/urandom of=/mnt/sdb1/msk4 bs=1M count=10 oflag=direct #md5sum /mnt/sdb1/msk4 > /mnt/sdb1/msk4.md5 && sync 6. Create a target convert image. #qemu-img create -f qcow2 /root/avocado/data/avocado-vt/convert1.qcow2 2G 7. Create mirror target image online. {'execute': 'blockdev-create', 'arguments': {'options': {'driver': 'file', 'filename': '/root/avocado/data/avocado-vt/convert1sn.qcow2', 'size': 2147483648}, 'job-id': 'file_convert1sn'}, 'id': 'yROlWk2R'} {'execute': 'job-dismiss', 'arguments': {'id': 'file_convert1sn'}, 'id': 'nvjJ6Ps8'} {'execute': 'blockdev-add', 'arguments': {'node-name': 'file_convert1sn', 'driver': 'file', 'filename': '/root/avocado/data/avocado-vt/convert1sn.qcow2', 'aio': 'threads', 'auto-read-only': True, 'discard': 'unmap'}, 'id': '6LsNR4hA'} {'execute': 'blockdev-create', 'arguments': {'options': {'driver': 'qcow2', 'file': 'file_convert1sn', 'size': 2147483648, 'backing-fmt': 'qcow2', 'backing-file': '/root/avocado/data/avocado-vt/convert1.qcow2'}, 'job-id': 'drive_convert1sn'}, 'id': 'DLWByep4'} {'execute': 'job-dismiss', 'arguments': {'id': 'drive_convert1sn'}, 'id': 'paf9Hnni'} {'execute': 'blockdev-add', 'arguments': {'node-name': 'drive_convert1sn', 'driver': 'qcow2', 'backing': None, 'file': 'file_convert1sn', 'read-only': False}, 'id': 'rNswMbRV'} 7. Mirror from src to target with sync "top". {'execute': 'blockdev-mirror', 'arguments': {'sync': 'top', 'device': 'drive_data1sn', 'target': 'drive_convert1sn', 'job-id': 'drive_data1sn_JJv9'}, 'id': 'dDgd6Dei'} 8. After mirror complted, convert image from src base to target base. #qemu-img convert -f qcow2 -O qcow2 /root/avocado/data/avocado-vt/data1.qcow2 /root/avocado/data/avocado-vt/convert1.qcow2 9. Add the converted target base image {'execute': 'blockdev-add', 'arguments': {'node-name': 'file_convert1', 'driver': 'file', 'auto-read-only': True, 'discard': 'unmap', 'aio': 'threads', 'filename': '/root/avocado/data/avocado-vt/convert1.qcow2', 'cache': {'direct': True, 'no-flush': False}}, 'id': 'Fdy1STFP'} {'execute': 'blockdev-add', 'arguments': {'node-name': 'drive_convert1', 'driver': 'qcow2', 'read-only': False, 'cache': {'direct': True, 'no-flush': False}, 'file': 'file_convert1'}, 'id': 'iDUdUtrr'} 10. Do snapshot from target top to target base. {'execute': 'blockdev-snapshot', 'arguments': {'node': 'drive_convert1', 'overlay': 'drive_convert1sn'}, 'id': 'oOm0J5Wt'} Actual results: After step10, qemu coredump with info: qemu: qemu_mutex_unlock_impl: Operation not permitted /tmp/aexpect_dM1RN4TM/aexpect-4u3euwsm.sh: line 1: 363458 Aborted (core dumped) MALLOC_PERTURB_=1 /usr/libexec/qemu-kvm -S -name 'avocado-vt-vm1' -sandbox on -blockdev node-name=file_ovmf_code,driver=file,filename=/usr/share/OVMF/OVMF_CODE.secboot.fd,auto-read-only=on,discard=unmap -blockdev node-name=drive_ovmf_code,driver=raw,read-only=on,file=file_ovmf_code -blockdev node-name=file_ovmf_vars,driver=file,filename=/root/avocado/data/avocado-vt/avocado-vt-vm1_rhel920-64-virtio-scsi_qcow2_filesystem_VARS.fd,auto-read-only=on,discard=unmap -blockdev node-name=drive_ovmf_vars,driver=raw,read-only=off,file=file_ovmf_vars -machine q35,memory-backend=mem-machine_mem,pflash0=drive_ovmf_code,pflash1=drive_ovmf_vars -device pcie-root-port,id=pcie-root-port-0,multifunction=on,bus=pcie.0,addr=0x1,chassis=1 -device pcie-pci-bridge,id=pcie-pci-bridge-0,addr=0x0,bus=pcie-root-port-0 -nodefaults -device VGA,bus=pcie.0,addr=0x2 -m 30720 -object '{"qom-type": "memory-backend-ram", "size": 32212254720, "id": "mem-machine_mem"}' -smp 10,maxcpus=10,cores=5,threads=1,dies=1,sockets=2 .... Expected results: Do snapshot successfully without coredumps or errors. Additional info: Coredump info: Executable: /usr/libexec/qemu-kvm Control Group: /user.slice/user-0.slice/session-9.scope Unit: session-9.scope Slice: user-0.slice Session: 9 Owner UID: 0 (root) Boot ID: 52c869c21ef64de49877ac0eed7aeb06 Machine ID: 3919555703fd4043b7f3cc2611ad4d18 Hostname: dell-per740xd-01.lab.eng.pek2.redhat.com Storage: /var/lib/systemd/coredump/core.qemu-kvm.0.52c869c21ef64de49877ac0eed7aeb06.363458.1675305507000000.zst (present) Size on Disk: 304.5M Message: Process 363458 (qemu-kvm) of user 0 dumped core. Stack trace of thread 363458: #0 0x00007f03026a154c __pthread_kill_implementation (libc.so.6 + 0xa154c) #1 0x00007f0302654d46 raise (libc.so.6 + 0x54d46) #2 0x00007f03026287f3 abort (libc.so.6 + 0x287f3) #3 0x000055d3bc22dff2 qemu_mutex_unlock_impl (qemu-kvm + 0x9bdff2) #4 0x000055d3bc08cda7 bdrv_do_drained_begin (qemu-kvm + 0x81cda7) #5 0x000055d3bc055e1e bdrv_replace_node_noperm (qemu-kvm + 0x7e5e1e) #6 0x000055d3bc055c92 bdrv_append (qemu-kvm + 0x7e5c92) #7 0x000055d3bc03c62c external_snapshot_prepare (qemu-kvm + 0x7cc62c) #8 0x000055d3bc03aedd qmp_transaction (qemu-kvm + 0x7caedd) #9 0x000055d3bc14e826 qmp_marshal_blockdev_snapshot (qemu-kvm + 0x8de826) #10 0x000055d3bc21e3f2 do_qmp_dispatch_bh (qemu-kvm + 0x9ae3f2) #11 0x000055d3bc22a3f1 aio_dispatch (qemu-kvm + 0x9ba3f1) #12 0x000055d3bc2450a2 aio_ctx_dispatch (qemu-kvm + 0x9d50a2) #13 0x00007f0302d1ae2f g_main_context_dispatch (libglib-2.0.so.0 + 0x54e2f) #14 0x000055d3bc2469c4 main_loop_wait (qemu-kvm + 0x9d69c4) #15 0x000055d3bbd4f8e7 qemu_main_loop (qemu-kvm + 0x4df8e7) #16 0x000055d3bbbd592a qemu_default_main (qemu-kvm + 0x36592a) #17 0x00007f030263feb0 __libc_start_call_main (libc.so.6 + 0x3feb0) #18 0x00007f030263ff60 __libc_start_main@@GLIBC_2.34 (libc.so.6 + 0x3ff60) #19 0x000055d3bbbd5085 _start (qemu-kvm + 0x365085) Stack trace of thread 363465: #0 0x00007f03027429bf __poll (libc.so.6 + 0x1429bf) #1 0x00007f0302d6f49c g_main_context_iterate.constprop.0 (libglib-2.0.so.0 + 0xa949c) #2 0x00007f0302d1a483 g_main_loop_run (libglib-2.0.so.0 + 0x54483) #3 0x000055d3bc043e2f iothread_run (qemu-kvm + 0x7d3e2f) #4 0x000055d3bc22edea qemu_thread_start (qemu-kvm + 0x9bedea) #5 0x00007f030269f802 start_thread (libc.so.6 + 0x9f802) #6 0x00007f030263f450 __clone3 (libc.so.6 + 0x3f450) Stack trace of thread 363470: #0 0x00007f030263ec6b ioctl (libc.so.6 + 0x3ec6b) #1 0x000055d3bbfe998b kvm_vcpu_ioctl (qemu-kvm + 0x77998b) #2 0x000055d3bbfef191 kvm_cpu_exec (qemu-kvm + 0x77f191) #3 0x000055d3bbff178a kvm_vcpu_thread_fn (qemu-kvm + 0x78178a) #4 0x000055d3bc22edea qemu_thread_start (qemu-kvm + 0x9bedea) #5 0x00007f030269f802 start_thread (libc.so.6 + 0x9f802) #6 0x00007f030263f450 __clone3 (libc.so.6 + 0x3f450) Stack trace of thread 363468: #0 0x00007f030263ec6b ioctl (libc.so.6 + 0x3ec6b) #1 0x000055d3bbfe998b kvm_vcpu_ioctl (qemu-kvm + 0x77998b) #2 0x000055d3bbfef191 kvm_cpu_exec (qemu-kvm + 0x77f191) #3 0x000055d3bbff178a kvm_vcpu_thread_fn (qemu-kvm + 0x78178a) #4 0x000055d3bc22edea qemu_thread_start (qemu-kvm + 0x9bedea) #5 0x00007f030269f802 start_thread (libc.so.6 + 0x9f802) #6 0x00007f030263f450 __clone3 (libc.so.6 + 0x3f450) Stack trace of thread 363476: #0 0x00007f030263ec6b ioctl (libc.so.6 + 0x3ec6b) #1 0x000055d3bbfe998b kvm_vcpu_ioctl (qemu-kvm + 0x77998b) #2 0x000055d3bbfef191 kvm_cpu_exec (qemu-kvm + 0x77f191) #3 0x000055d3bbff178a kvm_vcpu_thread_fn (qemu-kvm + 0x78178a) #4 0x000055d3bc22edea qemu_thread_start (qemu-kvm + 0x9bedea) #5 0x00007f030269f802 start_thread (libc.so.6 + 0x9f802) #6 0x00007f030263f450 __clone3 (libc.so.6 + 0x3f450) Stack trace of thread 363570: #0 0x00007f030269c39a __futex_abstimed_wait_common (libc.so.6 + 0x9c39a) #1 0x00007f030269eea4 pthread_cond_timedwait@@GLIBC_2.3.2 (libc.so.6 + 0x9eea4) #2 0x000055d3bc22e53c qemu_cond_timedwait_ts (qemu-kvm + 0x9be53c) #3 0x000055d3bc22e4e0 qemu_cond_timedwait_impl (qemu-kvm + 0x9be4e0) #4 0x000055d3bc2492a7 worker_thread (qemu-kvm + 0x9d92a7) #5 0x000055d3bc22edea qemu_thread_start (qemu-kvm + 0x9bedea) #6 0x00007f030269f802 start_thread (libc.so.6 + 0x9f802) #7 0x00007f030263f450 __clone3 (libc.so.6 + 0x3f450) Stack trace of thread 363460: #0 0x00007f0302742abe ppoll (libc.so.6 + 0x142abe) #1 0x000055d3bc22b8de fdmon_poll_wait (qemu-kvm + 0x9bb8de) #2 0x000055d3bc22ab1e aio_poll (qemu-kvm + 0x9bab1e) #3 0x000055d3bc043e12 iothread_run (qemu-kvm + 0x7d3e12) #4 0x000055d3bc22edea qemu_thread_start (qemu-kvm + 0x9bedea) #5 0x00007f030269f802 start_thread (libc.so.6 + 0x9f802) #6 0x00007f030263f450 __clone3 (libc.so.6 + 0x3f450) Stack trace of thread 363548: #0 0x00007f030269c39a __futex_abstimed_wait_common (libc.so.6 + 0x9c39a) #1 0x00007f030269eea4 pthread_cond_timedwait@@GLIBC_2.3.2 (libc.so.6 + 0x9eea4) #2 0x000055d3bc22e53c qemu_cond_timedwait_ts (qemu-kvm + 0x9be53c) #3 0x000055d3bc22e4e0 qemu_cond_timedwait_impl (qemu-kvm + 0x9be4e0) #4 0x000055d3bc2492a7 worker_thread (qemu-kvm + 0x9d92a7) #5 0x000055d3bc22edea qemu_thread_start (qemu-kvm + 0x9bedea) #6 0x00007f030269f802 start_thread (libc.so.6 + 0x9f802) #7 0x00007f030263f450 __clone3 (libc.so.6 + 0x3f450) Stack trace of thread 363466: #0 0x00007f030263ec6b ioctl (libc.so.6 + 0x3ec6b) #1 0x000055d3bbfe998b kvm_vcpu_ioctl (qemu-kvm + 0x77998b) #2 0x000055d3bbfef191 kvm_cpu_exec (qemu-kvm + 0x77f191) #3 0x000055d3bbff178a kvm_vcpu_thread_fn (qemu-kvm + 0x78178a) #4 0x000055d3bc22edea qemu_thread_start (qemu-kvm + 0x9bedea) #5 0x00007f030269f802 start_thread (libc.so.6 + 0x9f802) #6 0x00007f030263f450 __clone3 (libc.so.6 + 0x3f450) Stack trace of thread 363572: #0 0x00007f030269c39a __futex_abstimed_wait_common (libc.so.6 + 0x9c39a) #1 0x00007f030269eea4 pthread_cond_timedwait@@GLIBC_2.3.2 (libc.so.6 + 0x9eea4) #2 0x000055d3bc22e53c qemu_cond_timedwait_ts (qemu-kvm + 0x9be53c) #3 0x000055d3bc22e4e0 qemu_cond_timedwait_impl (qemu-kvm + 0x9be4e0) #4 0x000055d3bc2492a7 worker_thread (qemu-kvm + 0x9d92a7) #5 0x000055d3bc22edea qemu_thread_start (qemu-kvm + 0x9bedea) #6 0x00007f030269f802 start_thread (libc.so.6 + 0x9f802) #7 0x00007f030263f450 __clone3 (libc.so.6 + 0x3f450) Stack trace of thread 363571: #0 0x00007f030269c39a __futex_abstimed_wait_common (libc.so.6 + 0x9c39a) #1 0x00007f030269eea4 pthread_cond_timedwait@@GLIBC_2.3.2 (libc.so.6 + 0x9eea4) #2 0x000055d3bc22e53c qemu_cond_timedwait_ts (qemu-kvm + 0x9be53c) #3 0x000055d3bc22e4e0 qemu_cond_timedwait_impl (qemu-kvm + 0x9be4e0) #4 0x000055d3bc2492a7 worker_thread (qemu-kvm + 0x9d92a7) #5 0x000055d3bc22edea qemu_thread_start (qemu-kvm + 0x9bedea) #6 0x00007f030269f802 start_thread (libc.so.6 + 0x9f802) #7 0x00007f030263f450 __clone3 (libc.so.6 + 0x3f450) Stack trace of thread 363459: #0 0x00007f030263ee5d syscall (libc.so.6 + 0x3ee5d) #1 0x000055d3bc22eb3f qemu_event_wait (qemu-kvm + 0x9beb3f) #2 0x000055d3bc23ac75 call_rcu_thread (qemu-kvm + 0x9cac75) #3 0x000055d3bc22edea qemu_thread_start (qemu-kvm + 0x9bedea) #4 0x00007f030269f802 start_thread (libc.so.6 + 0x9f802) #5 0x00007f030263f450 __clone3 (libc.so.6 + 0x3f450) Stack trace of thread 363475: #0 0x00007f030263ec6b ioctl (libc.so.6 + 0x3ec6b) #1 0x000055d3bbfe998b kvm_vcpu_ioctl (qemu-kvm + 0x77998b) #2 0x000055d3bbfef191 kvm_cpu_exec (qemu-kvm + 0x77f191) #3 0x000055d3bbff178a kvm_vcpu_thread_fn (qemu-kvm + 0x78178a) #4 0x000055d3bc22edea qemu_thread_start (qemu-kvm + 0x9bedea) #5 0x00007f030269f802 start_thread (libc.so.6 + 0x9f802) #6 0x00007f030263f450 __clone3 (libc.so.6 + 0x3f450) Stack trace of thread 363472: #0 0x00007f030263ec6b ioctl (libc.so.6 + 0x3ec6b) #1 0x000055d3bbfe998b kvm_vcpu_ioctl (qemu-kvm + 0x77998b) #2 0x000055d3bbfef191 kvm_cpu_exec (qemu-kvm + 0x77f191) #3 0x000055d3bbff178a kvm_vcpu_thread_fn (qemu-kvm + 0x78178a) #4 0x000055d3bc22edea qemu_thread_start (qemu-kvm + 0x9bedea) #5 0x00007f030269f802 start_thread (libc.so.6 + 0x9f802) #6 0x00007f030263f450 __clone3 (libc.so.6 + 0x3f450) Stack trace of thread 363580: #0 0x00007f030269c39a __futex_abstimed_wait_common (libc.so.6 + 0x9c39a) #1 0x00007f030269eea4 pthread_cond_timedwait@@GLIBC_2.3.2 (libc.so.6 + 0x9eea4) #2 0x000055d3bc22e53c qemu_cond_timedwait_ts (qemu-kvm + 0x9be53c) #3 0x000055d3bc22e4e0 qemu_cond_timedwait_impl (qemu-kvm + 0x9be4e0) #4 0x000055d3bc2492a7 worker_thread (qemu-kvm + 0x9d92a7) #5 0x000055d3bc22edea qemu_thread_start (qemu-kvm + 0x9bedea) #6 0x00007f030269f802 start_thread (libc.so.6 + 0x9f802) #7 0x00007f030263f450 __clone3 (libc.so.6 + 0x3f450) Stack trace of thread 363477: #0 0x00007f030263ec6b ioctl (libc.so.6 + 0x3ec6b) #1 0x000055d3bbfe998b kvm_vcpu_ioctl (qemu-kvm + 0x77998b) #2 0x000055d3bbfef191 kvm_cpu_exec (qemu-kvm + 0x77f191) #3 0x000055d3bbff178a kvm_vcpu_thread_fn (qemu-kvm + 0x78178a) #4 0x000055d3bc22edea qemu_thread_start (qemu-kvm + 0x9bedea) #5 0x00007f030269f802 start_thread (libc.so.6 + 0x9f802) #6 0x00007f030263f450 __clone3 (libc.so.6 + 0x3f450) Stack trace of thread 363577: #0 0x00007f030269c39a __futex_abstimed_wait_common (libc.so.6 + 0x9c39a) #1 0x00007f030269eea4 pthread_cond_timedwait@@GLIBC_2.3.2 (libc.so.6 + 0x9eea4) #2 0x000055d3bc22e53c qemu_cond_timedwait_ts (qemu-kvm + 0x9be53c) #3 0x000055d3bc22e4e0 qemu_cond_timedwait_impl (qemu-kvm + 0x9be4e0) #4 0x000055d3bc2492a7 worker_thread (qemu-kvm + 0x9d92a7) #5 0x000055d3bc22edea qemu_thread_start (qemu-kvm + 0x9bedea) #6 0x00007f030269f802 start_thread (libc.so.6 + 0x9f802) #7 0x00007f030263f450 __clone3 (libc.so.6 + 0x3f450) Stack trace of thread 363474: #0 0x00007f030263ec6b ioctl (libc.so.6 + 0x3ec6b) #1 0x000055d3bbfe998b kvm_vcpu_ioctl (qemu-kvm + 0x77998b) #2 0x000055d3bbfef191 kvm_cpu_exec (qemu-kvm + 0x77f191) #3 0x000055d3bbff178a kvm_vcpu_thread_fn (qemu-kvm + 0x78178a) #4 0x000055d3bc22edea qemu_thread_start (qemu-kvm + 0x9bedea) #5 0x00007f030269f802 start_thread (libc.so.6 + 0x9f802) #6 0x00007f030263f450 __clone3 (libc.so.6 + 0x3f450) Stack trace of thread 363584: #0 0x00007f030269c39a __futex_abstimed_wait_common (libc.so.6 + 0x9c39a) #1 0x00007f030269eea4 pthread_cond_timedwait@@GLIBC_2.3.2 (libc.so.6 + 0x9eea4) #2 0x000055d3bc22e53c qemu_cond_timedwait_ts (qemu-kvm + 0x9be53c) #3 0x000055d3bc22e4e0 qemu_cond_timedwait_impl (qemu-kvm + 0x9be4e0) #4 0x000055d3bc2492a7 worker_thread (qemu-kvm + 0x9d92a7) #5 0x000055d3bc22edea qemu_thread_start (qemu-kvm + 0x9bedea) #6 0x00007f030269f802 start_thread (libc.so.6 + 0x9f802) #7 0x00007f030263f450 __clone3 (libc.so.6 + 0x3f450) Stack trace of thread 363573: #0 0x00007f030269c39a __futex_abstimed_wait_common (libc.so.6 + 0x9c39a) #1 0x00007f030269eea4 pthread_cond_timedwait@@GLIBC_2.3.2 (libc.so.6 + 0x9eea4) #2 0x000055d3bc22e53c qemu_cond_timedwait_ts (qemu-kvm + 0x9be53c) #3 0x000055d3bc22e4e0 qemu_cond_timedwait_impl (qemu-kvm + 0x9be4e0) #4 0x000055d3bc2492a7 worker_thread (qemu-kvm + 0x9d92a7) #5 0x000055d3bc22edea qemu_thread_start (qemu-kvm + 0x9bedea) #6 0x00007f030269f802 start_thread (libc.so.6 + 0x9f802) #7 0x00007f030263f450 __clone3 (libc.so.6 + 0x3f450) Stack trace of thread 363479: #0 0x00007f030269c39a __futex_abstimed_wait_common (libc.so.6 + 0x9c39a) #1 0x00007f030269eba0 pthread_cond_wait@@GLIBC_2.3.2 (libc.so.6 + 0x9eba0) #2 0x000055d3bc22e39f qemu_cond_wait_impl (qemu-kvm + 0x9be39f) #3 0x000055d3bbc0bc76 vnc_worker_thread (qemu-kvm + 0x39bc76) #4 0x000055d3bc22edea qemu_thread_start (qemu-kvm + 0x9bedea) #5 0x00007f030269f802 start_thread (libc.so.6 + 0x9f802) #6 0x00007f030263f450 __clone3 (libc.so.6 + 0x3f450) Stack trace of thread 363583: #0 0x00007f030269c39a __futex_abstimed_wait_common (libc.so.6 + 0x9c39a) #1 0x00007f030269eea4 pthread_cond_timedwait@@GLIBC_2.3.2 (libc.so.6 + 0x9eea4) #2 0x000055d3bc22e53c qemu_cond_timedwait_ts (qemu-kvm + 0x9be53c) #3 0x000055d3bc22e4e0 qemu_cond_timedwait_impl (qemu-kvm + 0x9be4e0) #4 0x000055d3bc2492a7 worker_thread (qemu-kvm + 0x9d92a7) #5 0x000055d3bc22edea qemu_thread_start (qemu-kvm + 0x9bedea) #6 0x00007f030269f802 start_thread (libc.so.6 + 0x9f802) #7 0x00007f030263f450 __clone3 (libc.so.6 + 0x3f450) Stack trace of thread 363473: #0 0x00007f030263ec6b ioctl (libc.so.6 + 0x3ec6b) #1 0x000055d3bbfe998b kvm_vcpu_ioctl (qemu-kvm + 0x77998b) #2 0x000055d3bbfef191 kvm_cpu_exec (qemu-kvm + 0x77f191) #3 0x000055d3bbff178a kvm_vcpu_thread_fn (qemu-kvm + 0x78178a) #4 0x000055d3bc22edea qemu_thread_start (qemu-kvm + 0x9bedea) #5 0x00007f030269f802 start_thread (libc.so.6 + 0x9f802) #6 0x00007f030263f450 __clone3 (libc.so.6 + 0x3f450) Stack trace of thread 363469: #0 0x00007f030263ec6b ioctl (libc.so.6 + 0x3ec6b) #1 0x000055d3bbfe998b kvm_vcpu_ioctl (qemu-kvm + 0x77998b) #2 0x000055d3bbfef191 kvm_cpu_exec (qemu-kvm + 0x77f191) #3 0x000055d3bbff178a kvm_vcpu_thread_fn (qemu-kvm + 0x78178a) #4 0x000055d3bc22edea qemu_thread_start (qemu-kvm + 0x9bedea) #5 0x00007f030269f802 start_thread (libc.so.6 + 0x9f802) #6 0x00007f030263f450 __clone3 (libc.so.6 + 0x3f450) Stack trace of thread 363581: #0 0x00007f030269c39a __futex_abstimed_wait_common (libc.so.6 + 0x9c39a) #1 0x00007f030269eea4 pthread_cond_timedwait@@GLIBC_2.3.2 (libc.so.6 + 0x9eea4) #2 0x000055d3bc22e53c qemu_cond_timedwait_ts (qemu-kvm + 0x9be53c) #3 0x000055d3bc22e4e0 qemu_cond_timedwait_impl (qemu-kvm + 0x9be4e0) #4 0x000055d3bc2492a7 worker_thread (qemu-kvm + 0x9d92a7) #5 0x000055d3bc22edea qemu_thread_start (qemu-kvm + 0x9bedea) #6 0x00007f030269f802 start_thread (libc.so.6 + 0x9f802) #7 0x00007f030263f450 __clone3 (libc.so.6 + 0x3f450) Stack trace of thread 363619: #0 0x00007f030269c39a __futex_abstimed_wait_common (libc.so.6 + 0x9c39a) #1 0x00007f030269eea4 pthread_cond_timedwait@@GLIBC_2.3.2 (libc.so.6 + 0x9eea4) #2 0x000055d3bc22e53c qemu_cond_timedwait_ts (qemu-kvm + 0x9be53c) #3 0x000055d3bc22e4e0 qemu_cond_timedwait_impl (qemu-kvm + 0x9be4e0) #4 0x000055d3bc2492a7 worker_thread (qemu-kvm + 0x9d92a7) #5 0x000055d3bc22edea qemu_thread_start (qemu-kvm + 0x9bedea) #6 0x00007f030269f802 start_thread (libc.so.6 + 0x9f802) #7 0x00007f030263f450 __clone3 (libc.so.6 + 0x3f450) Stack trace of thread 363587: #0 0x00007f030269c39a __futex_abstimed_wait_common (libc.so.6 + 0x9c39a) #1 0x00007f030269eea4 pthread_cond_timedwait@@GLIBC_2.3.2 (libc.so.6 + 0x9eea4) #2 0x000055d3bc22e53c qemu_cond_timedwait_ts (qemu-kvm + 0x9be53c) #3 0x000055d3bc22e4e0 qemu_cond_timedwait_impl (qemu-kvm + 0x9be4e0) #4 0x000055d3bc2492a7 worker_thread (qemu-kvm + 0x9d92a7) #5 0x000055d3bc22edea qemu_thread_start (qemu-kvm + 0x9bedea) #6 0x00007f030269f802 start_thread (libc.so.6 + 0x9f802) #7 0x00007f030263f450 __clone3 (libc.so.6 + 0x3f450) Stack trace of thread 363586: #0 0x00007f030269c39a __futex_abstimed_wait_common (libc.so.6 + 0x9c39a) #1 0x00007f030269eea4 pthread_cond_timedwait@@GLIBC_2.3.2 (libc.so.6 + 0x9eea4) #2 0x000055d3bc22e53c qemu_cond_timedwait_ts (qemu-kvm + 0x9be53c) #3 0x000055d3bc22e4e0 qemu_cond_timedwait_impl (qemu-kvm + 0x9be4e0) #4 0x000055d3bc2492a7 worker_thread (qemu-kvm + 0x9d92a7) #5 0x000055d3bc22edea qemu_thread_start (qemu-kvm + 0x9bedea) #6 0x00007f030269f802 start_thread (libc.so.6 + 0x9f802) #7 0x00007f030263f450 __clone3 (libc.so.6 + 0x3f450) Stack trace of thread 363578: #0 0x00007f030269c39a __futex_abstimed_wait_common (libc.so.6 + 0x9c39a) #1 0x00007f030269eea4 pthread_cond_timedwait@@GLIBC_2.3.2 (libc.so.6 + 0x9eea4) #2 0x000055d3bc22e53c qemu_cond_timedwait_ts (qemu-kvm + 0x9be53c) #3 0x000055d3bc22e4e0 qemu_cond_timedwait_impl (qemu-kvm + 0x9be4e0) #4 0x000055d3bc2492a7 worker_thread (qemu-kvm + 0x9d92a7) #5 0x000055d3bc22edea qemu_thread_start (qemu-kvm + 0x9bedea) #6 0x00007f030269f802 start_thread (libc.so.6 + 0x9f802) #7 0x00007f030263f450 __clone3 (libc.so.6 + 0x3f450) Stack trace of thread 363471: #0 0x00007f030263ec6b ioctl (libc.so.6 + 0x3ec6b) #1 0x000055d3bbfe998b kvm_vcpu_ioctl (qemu-kvm + 0x77998b) #2 0x000055d3bbfef191 kvm_cpu_exec (qemu-kvm + 0x77f191) #3 0x000055d3bbff178a kvm_vcpu_thread_fn (qemu-kvm + 0x78178a) #4 0x000055d3bc22edea qemu_thread_start (qemu-kvm + 0x9bedea) #5 0x00007f030269f802 start_thread (libc.so.6 + 0x9f802) #6 0x00007f030263f450 __clone3 (libc.so.6 + 0x3f450) Stack trace of thread 363579: #0 0x00007f030269c39a __futex_abstimed_wait_common (libc.so.6 + 0x9c39a) #1 0x00007f030269eea4 pthread_cond_timedwait@@GLIBC_2.3.2 (libc.so.6 + 0x9eea4) #2 0x000055d3bc22e53c qemu_cond_timedwait_ts (qemu-kvm + 0x9be53c) #3 0x000055d3bc22e4e0 qemu_cond_timedwait_impl (qemu-kvm + 0x9be4e0) #4 0x000055d3bc2492a7 worker_thread (qemu-kvm + 0x9d92a7) #5 0x000055d3bc22edea qemu_thread_start (qemu-kvm + 0x9bedea) #6 0x00007f030269f802 start_thread (libc.so.6 + 0x9f802) #7 0x00007f030263f450 __clone3 (libc.so.6 + 0x3f450) Stack trace of thread 363585: #0 0x00007f030269c39a __futex_abstimed_wait_common (libc.so.6 + 0x9c39a) #1 0x00007f030269eea4 pthread_cond_timedwait@@GLIBC_2.3.2 (libc.so.6 + 0x9eea4) #2 0x000055d3bc22e53c qemu_cond_timedwait_ts (qemu-kvm + 0x9be53c) #3 0x000055d3bc22e4e0 qemu_cond_timedwait_impl (qemu-kvm + 0x9be4e0) #4 0x000055d3bc2492a7 worker_thread (qemu-kvm + 0x9d92a7) #5 0x000055d3bc22edea qemu_thread_start (qemu-kvm + 0x9bedea) #6 0x00007f030269f802 start_thread (libc.so.6 + 0x9f802) #7 0x00007f030263f450 __clone3 (libc.so.6 + 0x3f450) Stack trace of thread 363582: #0 0x00007f030269c39a __futex_abstimed_wait_common (libc.so.6 + 0x9c39a) #1 0x00007f030269eea4 pthread_cond_timedwait@@GLIBC_2.3.2 (libc.so.6 + 0x9eea4) #2 0x000055d3bc22e53c qemu_cond_timedwait_ts (qemu-kvm + 0x9be53c) #3 0x000055d3bc22e4e0 qemu_cond_timedwait_impl (qemu-kvm + 0x9be4e0) #4 0x000055d3bc2492a7 worker_thread (qemu-kvm + 0x9d92a7) #5 0x000055d3bc22edea qemu_thread_start (qemu-kvm + 0x9bedea) #6 0x00007f030269f802 start_thread (libc.so.6 + 0x9f802) #7 0x00007f030263f450 __clone3 (libc.so.6 + 0x3f450) Stack trace of thread 363467: #0 0x00007f030263ec6b ioctl (libc.so.6 + 0x3ec6b) #1 0x000055d3bbfe998b kvm_vcpu_ioctl (qemu-kvm + 0x77998b) #2 0x000055d3bbfef191 kvm_cpu_exec (qemu-kvm + 0x77f191) #3 0x000055d3bbff178a kvm_vcpu_thread_fn (qemu-kvm + 0x78178a) #4 0x000055d3bc22edea qemu_thread_start (qemu-kvm + 0x9bedea) #5 0x00007f030269f802 start_thread (libc.so.6 + 0x9f802) #6 0x00007f030263f450 __clone3 (libc.so.6 + 0x3f450) ELF object binary architecture: AMD x86-64 Note: I tried with qemu-kvm-7.2.0-5.el9, and still reproduce it with a ratio: 4/50 Then I run the case on qemu-kvm-7.1.0-7.el9 for 100 times, and all tests pass.