Bug 2040123
| Summary: | Qemu core dumped when do block-stream to a snapshot node on non-enough space storage | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 9 | Reporter: | aihua liang <aliang> |
| Component: | qemu-kvm | Assignee: | Hanna Czenczek <hreitz> |
| qemu-kvm sub component: | Block Jobs | QA Contact: | aihua liang <aliang> |
| Status: | CLOSED ERRATA | Docs Contact: | |
| Severity: | medium | ||
| Priority: | medium | CC: | aliang, coli, hreitz, jinzhao, kkiwi, kwolf, mrezanin, ngu, smitterl, stefanha, thuth, virt-maint, virt-qe-z |
| Version: | 9.0 | Keywords: | Triaged |
| Target Milestone: | rc | Flags: | pm-rhel:
mirror+
|
| Target Release: | 9.0 | ||
| Hardware: | All | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | qemu-kvm-6.2.0-8.el9 | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | 2036178 | Environment: | |
| Last Closed: | 2022-05-17 12:25:11 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | 2036178 | ||
| Bug Blocks: | |||
Setting Assignee + QA Contact to the same as in BZ 2036178 (I hope that's ok) If I've got the comments in the RHEL 8.6 counterpart right, the fix should be available in time for 9.0, so I'm setting ITR accordingly now. Have set ITM, cancel the needinfo. Thanks, Aliang *** Bug 2021820 has been marked as a duplicate of this bug. *** QE bot(pre verify): Set 'Verified:Tested,SanityOnly' as gating/tier1 test pass. Test with qemu-kvm-6.2.0-8.el9, the problem has been resolve.
Test Steps:
1.Create base.img and top.img
#qemu-img create -f qcow2 base.img 2G
#qemu-img create -f qcow2 top.img 2G
2.Write data to base.img
#qemu-io -c 'write 0 2147483136' base.img
3.Start vm with accel=tcg
/usr/libexec/qemu-kvm \
-name 'avocado-vt-vm1' \
-sandbox on \
-machine q35,memory-backend=mem-machine_mem,accel=tcg \
-device pcie-root-port,id=pcie-root-port-0,multifunction=on,bus=pcie.0,addr=0x1,chassis=1 \
-device pcie-pci-bridge,id=pcie-pci-bridge-0,addr=0x0,bus=pcie-root-port-0 \
-nodefaults \
-device VGA,bus=pcie.0,addr=0x2 \
-m 30720 \
-object memory-backend-ram,size=30720M,id=mem-machine_mem \
-smp 10,maxcpus=10,cores=5,threads=1,dies=1,sockets=2 \
-cpu 'Cascadelake-Server',ss=on,vmx=on,pdcm=on,hypervisor=on,tsc-adjust=on,umip=on,pku=on,md-clear=on,stibp=on,arch-capabilities=on,xsaves=on,ibpb=on,ibrs=on,amd-stibp=on,amd-ssbd=on,rdctl-no=on,ibrs-all=on,skip-l1dfl-vmentry=on,mds-no=on,pschange-mc-no=on,tsx-ctrl=on,hle=off,rtm=off,kvm_pv_unhalt=on \
-chardev socket,wait=off,id=qmp_id_qmpmonitor1,path=/tmp/monitor-qmpmonitor1-20220105-012140-9igI6qZZ,server=on \
-mon chardev=qmp_id_qmpmonitor1,mode=control \
-chardev socket,wait=off,id=qmp_id_catch_monitor,path=/tmp/monitor-catch_monitor-20220105-012140-9igI6qZZ,server=on \
-mon chardev=qmp_id_catch_monitor,mode=control \
-device pvpanic,ioport=0x505,id=idsULHUn \
-chardev socket,wait=off,id=chardev_serial0,path=/tmp/serial-serial0-20220105-012140-9igI6qZZ,server=on \
-device isa-serial,id=serial0,chardev=chardev_serial0 \
-chardev socket,id=seabioslog_id_20220105-012140-9igI6qZZ,path=/tmp/seabios-20220105-012140-9igI6qZZ,server=on,wait=off \
-device isa-debugcon,chardev=seabioslog_id_20220105-012140-9igI6qZZ,iobase=0x402 \
-device pcie-root-port,id=pcie-root-port-1,port=0x1,addr=0x1.0x1,bus=pcie.0,chassis=2 \
-device qemu-xhci,id=usb1,bus=pcie-root-port-1,addr=0x0 \
-device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 \
-device pcie-root-port,id=pcie-root-port-2,port=0x2,addr=0x1.0x2,bus=pcie.0,chassis=3 \
-device pcie-root-port,id=pcie-root-port-3,port=0x3,addr=0x1.0x3,bus=pcie.0,chassis=4 \
-device virtio-net-pci,mac=9a:d5:a4:20:d5:09,id=idVy3YIg,netdev=idXjRF2L,bus=pcie-root-port-3,addr=0x0 \
-netdev tap,id=idXjRF2L,vhost=on \
-vnc :0 \
-rtc base=utc,clock=host,driftfix=slew \
-boot menu=off,order=cdn,once=c,strict=off \
-enable-kvm \
-device pcie-root-port,id=pcie_extra_root_port_0,multifunction=on,bus=pcie.0,addr=0x3,chassis=5 \
-monitor stdio \
4.Add throttle group
{ "execute" : "object-add", "arguments": { "qom-type" : "throttle-group","id": "thrgr","x-bps-total":16384}}
5.Add base node and top node
{'execute':'blockdev-add','arguments':{'driver':'qcow2','node-name':'base','file':{'driver':'throttle','throttle-group':'thrgr','file':{'driver':'file','filename':'/home/base.img'}}}}
{"return": {}}
{'execute':'blockdev-add','arguments':{'driver':'qcow2','node-name':'top','file':{'driver':'blkdebug','inject-error':[{'event':'pwritev','immediately':true,'once':true}],'image':{'driver':'file','filename':'/home/top.img'}},'backing':'base'}}
{"return": {}}
6.Add scsi device
{'execute':'device_add','arguments':{'driver': 'virtio-scsi-pci','id':'vscsi','bus':'pcie-root-port-2'}}
{"return": {}}
{'execute':'device_add','arguments':{'driver': 'scsi-hd','bus':'vscsi.0','drive':'top'}}
{"return": {}}
7.Do stream
{'execute':'block-stream','arguments':{'device':'top','job-id':'j1'}}
{"timestamp": {"seconds": 1644911009, "microseconds": 689213}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "j1"}}
{"timestamp": {"seconds": 1644911009, "microseconds": 689299}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "j1"}}
{"return": {}}
8. During step7, reset vm
{'execute':'system_reset'}
After step8, block job completed with input/output error.
{"timestamp": {"seconds": 1644911017, "microseconds": 593201}, "event": "BLOCK_JOB_ERROR", "data": {"device": "j1", "operation": "read", "action": "report"}}
{"timestamp": {"seconds": 1644911017, "microseconds": 593256}, "event": "JOB_STATUS_CHANGE", "data": {"status": "aborting", "id": "j1"}}
{"timestamp": {"seconds": 1644911017, "microseconds": 593301}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "j1", "len": 2147483648, "offset": 0, "speed": 0, "type": "stream", "error": "Input/output error"}}
{"timestamp": {"seconds": 1644911017, "microseconds": 593327}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "j1"}}
{"timestamp": {"seconds": 1644911017, "microseconds": 593346}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "j1"}}
{"timestamp": {"seconds": 1644911017, "microseconds": 596673}, "event": "RESET", "data": {"guest": false, "reason": "host-qmp-system-reset"}}
{"timestamp": {"seconds": 1644911017, "microseconds": 599857}, "event": "RESET", "data": {"guest": true, "reason": "guest-reset"}}
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (new packages: qemu-kvm), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2022:2307 |
Test on qemu-kvm-6.2.0-3.el9，also hit this issue, with error msg: (qemu) qemu-kvm: ../block/io.c:516: void bdrv_do_drained_end(BlockDriverState *, _Bool, BdrvChild *, _Bool, int *): Assertion `bs->quiesce_counter > 0' failed. bug.txt: line 36: 1405298 Aborted (core dumped) /usr/libexec/qemu-kvm -S -name 'avocado-vt-vm1' -sandbox on -machine q35,memory-backend=mem-machine_mem,accel=tcg -device pcie-root-port,id=pcie-root-port-0,multifunction=on,bus=pcie.0,addr=0x1,chassis=1 -device pcie-pci-bridge,id=pcie-pci-bridge-0,addr=0x0,bus=pcie-root-port-0 ... gbd info: Stack trace of thread 1405298: #0 0x00007ff982ab37fc __pthread_kill_implementation (libc.so.6 + 0x8f7fc) #1 0x00007ff982a66676 raise (libc.so.6 + 0x42676) #2 0x00007ff982a507d3 abort (libc.so.6 + 0x2c7d3) #3 0x00007ff982a506fb __assert_fail_base.cold (libc.so.6 + 0x2c6fb) #4 0x00007ff982a5f396 __assert_fail (libc.so.6 + 0x3b396) #5 0x000055f25d547b7a bdrv_do_drained_end (qemu-kvm + 0x7d2b7a) #6 0x000055f25d547f9e bdrv_drained_end (qemu-kvm + 0x7d2f9e) #7 0x000055f25d269731 scsi_device_purge_requests (qemu-kvm + 0x4f4731) #8 0x000055f25d274358 scsi_disk_reset (qemu-kvm + 0x4ff358) #9 0x000055f25d4c5898 resettable_phase_hold (qemu-kvm + 0x750898) #10 0x000055f25d4bf74c bus_reset_child_foreach (qemu-kvm + 0x74a74c) #11 0x000055f25d4c583e resettable_phase_hold (qemu-kvm + 0x75083e) #12 0x000055f25d4c31bb device_reset_child_foreach (qemu-kvm + 0x74e1bb) #13 0x000055f25d4c583e resettable_phase_hold (qemu-kvm + 0x75083e) #14 0x000055f25d4bf74c bus_reset_child_foreach (qemu-kvm + 0x74a74c) #15 0x000055f25d4c583e resettable_phase_hold (qemu-kvm + 0x75083e) #16 0x000055f25d4c31bb device_reset_child_foreach (qemu-kvm + 0x74e1bb) #17 0x000055f25d4c583e resettable_phase_hold (qemu-kvm + 0x75083e) #18 0x000055f25d4bf74c bus_reset_child_foreach (qemu-kvm + 0x74a74c) #19 0x000055f25d4c583e resettable_phase_hold (qemu-kvm + 0x75083e) #20 0x000055f25d4c31bb device_reset_child_foreach (qemu-kvm + 0x74e1bb) #21 0x000055f25d4c583e resettable_phase_hold (qemu-kvm + 0x75083e) #22 0x000055f25d4bf74c bus_reset_child_foreach (qemu-kvm + 0x74a74c) #23 0x000055f25d4c583e resettable_phase_hold (qemu-kvm + 0x75083e) #24 0x000055f25d4c31bb device_reset_child_foreach (qemu-kvm + 0x74e1bb) #25 0x000055f25d4c583e resettable_phase_hold (qemu-kvm + 0x75083e) #26 0x000055f25d4bf74c bus_reset_child_foreach (qemu-kvm + 0x74a74c) #27 0x000055f25d4c583e resettable_phase_hold (qemu-kvm + 0x75083e) #28 0x000055f25d4c5b74 resettable_cold_reset_fn (qemu-kvm + 0x750b74) #29 0x000055f25d2e265a pc_machine_reset (qemu-kvm + 0x56d65a) #30 0x000055f25d377eb0 qemu_system_reset (qemu-kvm + 0x602eb0) #31 0x000055f25d378f1f qemu_main_loop (qemu-kvm + 0x603f1f) #32 0x000055f25d0d16f2 main (qemu-kvm + 0x35c6f2) #33 0x00007ff982a51560 __libc_start_call_main (libc.so.6 + 0x2d560) #34 0x00007ff982a5160c __libc_start_main@@GLIBC_2.34 (libc.so.6 + 0x2d60c) #35 0x000055f25d0d0db5 _start (qemu-kvm + 0x35bdb5) Stack trace of thread 1405299: #0 0x00007ff982b2efbd syscall (libc.so.6 + 0x10afbd) #1 0x000055f25d6dc83f qemu_event_wait (qemu-kvm + 0x96783f) #2 0x000055f25d6e8901 call_rcu_thread (qemu-kvm + 0x973901) #3 0x000055f25d6dcaea qemu_thread_start (qemu-kvm + 0x967aea) #4 0x00007ff982ab1aaf start_thread (libc.so.6 + 0x8daaf) #5 0x00007ff982b36740 __clone3 (libc.so.6 + 0x112740) Stack trace of thread 1405306: #0 0x00007ff982b2987f __poll (libc.so.6 + 0x10587f) #1 0x00007ff982dda56c g_main_context_iterate.constprop.0 (libglib-2.0.so.0 + 0xa956c) #2 0x00007ff982d85533 g_main_loop_run (libglib-2.0.so.0 + 0x54533) #3 0x000055f25d503374 iothread_run (qemu-kvm + 0x78e374) #4 0x000055f25d6dcaea qemu_thread_start (qemu-kvm + 0x967aea) #5 0x00007ff982ab1aaf start_thread (libc.so.6 + 0x8daaf) #6 0x00007ff982b36740 __clone3 (libc.so.6 + 0x112740) Stack trace of thread 1405307: #0 0x00007ff982aae70a __futex_abstimed_wait_common (libc.so.6 + 0x8a70a) #1 0x00007ff982ab0e80 pthread_cond_wait@@GLIBC_2.3.2 (libc.so.6 + 0x8ce80) #2 0x000055f25d6dc14f qemu_cond_wait_impl (qemu-kvm + 0x96714f) #3 0x000055f25d368157 qemu_wait_io_event (qemu-kvm + 0x5f3157) #4 0x000055f25d485001 kvm_vcpu_thread_fn (qemu-kvm + 0x710001) #5 0x000055f25d6dcaea qemu_thread_start (qemu-kvm + 0x967aea) #6 0x00007ff982ab1aaf start_thread (libc.so.6 + 0x8daaf) #7 0x00007ff982b36740 __clone3 (libc.so.6 + 0x112740) Stack trace of thread 1405308: #0 0x00007ff982aae70a __futex_abstimed_wait_common (libc.so.6 + 0x8a70a) #1 0x00007ff982ab0e80 pthread_cond_wait@@GLIBC_2.3.2 (libc.so.6 + 0x8ce80) #2 0x000055f25d6dc14f qemu_cond_wait_impl (qemu-kvm + 0x96714f) #3 0x000055f25d368157 qemu_wait_io_event (qemu-kvm + 0x5f3157) #4 0x000055f25d485001 kvm_vcpu_thread_fn (qemu-kvm + 0x710001) #5 0x000055f25d6dcaea qemu_thread_start (qemu-kvm + 0x967aea) #6 0x00007ff982ab1aaf start_thread (libc.so.6 + 0x8daaf) #7 0x00007ff982b36740 __clone3 (libc.so.6 + 0x112740) Stack trace of thread 1405310: #0 0x00007ff982aae70a __futex_abstimed_wait_common (libc.so.6 + 0x8a70a) #1 0x00007ff982ab0e80 pthread_cond_wait@@GLIBC_2.3.2 (libc.so.6 + 0x8ce80) #2 0x000055f25d6dc14f qemu_cond_wait_impl (qemu-kvm + 0x96714f) #3 0x000055f25d368157 qemu_wait_io_event (qemu-kvm + 0x5f3157) #4 0x000055f25d485001 kvm_vcpu_thread_fn (qemu-kvm + 0x710001) #5 0x000055f25d6dcaea qemu_thread_start (qemu-kvm + 0x967aea) #6 0x00007ff982ab1aaf start_thread (libc.so.6 + 0x8daaf) #7 0x00007ff982b36740 __clone3 (libc.so.6 + 0x112740) Stack trace of thread 1405309: #0 0x00007ff982aae70a __futex_abstimed_wait_common (libc.so.6 + 0x8a70a) #1 0x00007ff982ab0e80 pthread_cond_wait@@GLIBC_2.3.2 (libc.so.6 + 0x8ce80) #2 0x000055f25d6dc14f qemu_cond_wait_impl (qemu-kvm + 0x96714f) #3 0x000055f25d368157 qemu_wait_io_event (qemu-kvm + 0x5f3157) #4 0x000055f25d485001 kvm_vcpu_thread_fn (qemu-kvm + 0x710001) #5 0x000055f25d6dcaea qemu_thread_start (qemu-kvm + 0x967aea) #6 0x00007ff982ab1aaf start_thread (libc.so.6 + 0x8daaf) #7 0x00007ff982b36740 __clone3 (libc.so.6 + 0x112740) Stack trace of thread 1405311: #0 0x00007ff982aae70a __futex_abstimed_wait_common (libc.so.6 + 0x8a70a) #1 0x00007ff982ab0e80 pthread_cond_wait@@GLIBC_2.3.2 (libc.so.6 + 0x8ce80) #2 0x000055f25d6dc14f qemu_cond_wait_impl (qemu-kvm + 0x96714f) #3 0x000055f25d368157 qemu_wait_io_event (qemu-kvm + 0x5f3157) #4 0x000055f25d485001 kvm_vcpu_thread_fn (qemu-kvm + 0x710001) #5 0x000055f25d6dcaea qemu_thread_start (qemu-kvm + 0x967aea) #6 0x00007ff982ab1aaf start_thread (libc.so.6 + 0x8daaf) #7 0x00007ff982b36740 __clone3 (libc.so.6 + 0x112740) Stack trace of thread 1405313: #0 0x00007ff982aae70a __futex_abstimed_wait_common (libc.so.6 + 0x8a70a) #1 0x00007ff982ab0e80 pthread_cond_wait@@GLIBC_2.3.2 (libc.so.6 + 0x8ce80) #2 0x000055f25d6dc14f qemu_cond_wait_impl (qemu-kvm + 0x96714f) #3 0x000055f25d368157 qemu_wait_io_event (qemu-kvm + 0x5f3157) #4 0x000055f25d485001 kvm_vcpu_thread_fn (qemu-kvm + 0x710001) #5 0x000055f25d6dcaea qemu_thread_start (qemu-kvm + 0x967aea) #6 0x00007ff982ab1aaf start_thread (libc.so.6 + 0x8daaf) #7 0x00007ff982b36740 __clone3 (libc.so.6 + 0x112740) Stack trace of thread 1405314: #0 0x00007ff982aae70a __futex_abstimed_wait_common (libc.so.6 + 0x8a70a) #1 0x00007ff982ab0e80 pthread_cond_wait@@GLIBC_2.3.2 (libc.so.6 + 0x8ce80) #2 0x000055f25d6dc14f qemu_cond_wait_impl (qemu-kvm + 0x96714f) #3 0x000055f25d368157 qemu_wait_io_event (qemu-kvm + 0x5f3157) #4 0x000055f25d485001 kvm_vcpu_thread_fn (qemu-kvm + 0x710001) #5 0x000055f25d6dcaea qemu_thread_start (qemu-kvm + 0x967aea) #6 0x00007ff982ab1aaf start_thread (libc.so.6 + 0x8daaf) #7 0x00007ff982b36740 __clone3 (libc.so.6 + 0x112740) Stack trace of thread 1405316: #0 0x00007ff982aae70a __futex_abstimed_wait_common (libc.so.6 + 0x8a70a) #1 0x00007ff982ab0e80 pthread_cond_wait@@GLIBC_2.3.2 (libc.so.6 + 0x8ce80) #2 0x000055f25d6dc14f qemu_cond_wait_impl (qemu-kvm + 0x96714f) #3 0x000055f25d368157 qemu_wait_io_event (qemu-kvm + 0x5f3157) #4 0x000055f25d485001 kvm_vcpu_thread_fn (qemu-kvm + 0x710001) #5 0x000055f25d6dcaea qemu_thread_start (qemu-kvm + 0x967aea) #6 0x00007ff982ab1aaf start_thread (libc.so.6 + 0x8daaf) #7 0x00007ff982b36740 __clone3 (libc.so.6 + 0x112740) Stack trace of thread 1405318: #0 0x00007ff982aae70a __futex_abstimed_wait_common (libc.so.6 + 0x8a70a) #1 0x00007ff982ab0e80 pthread_cond_wait@@GLIBC_2.3.2 (libc.so.6 + 0x8ce80) #2 0x000055f25d6dc14f qemu_cond_wait_impl (qemu-kvm + 0x96714f) #3 0x000055f25d368157 qemu_wait_io_event (qemu-kvm + 0x5f3157) #4 0x000055f25d485001 kvm_vcpu_thread_fn (qemu-kvm + 0x710001) #5 0x000055f25d6dcaea qemu_thread_start (qemu-kvm + 0x967aea) #6 0x00007ff982ab1aaf start_thread (libc.so.6 + 0x8daaf) #7 0x00007ff982b36740 __clone3 (libc.so.6 + 0x112740) Stack trace of thread 1405353: #0 0x00007ff982aae70a __futex_abstimed_wait_common (libc.so.6 + 0x8a70a) #1 0x00007ff982ab9813 __new_sem_wait_slow64.constprop.0 (libc.so.6 + 0x95813) #2 0x000055f25d6dc55b qemu_sem_timedwait (qemu-kvm + 0x96755b) #3 0x000055f25d703106 worker_thread (qemu-kvm + 0x98e106) #4 0x000055f25d6dcaea qemu_thread_start (qemu-kvm + 0x967aea) #5 0x00007ff982ab1aaf start_thread (libc.so.6 + 0x8daaf) #6 0x00007ff982b36740 __clone3 (libc.so.6 + 0x112740) Stack trace of thread 1405315: #0 0x00007ff982aae70a __futex_abstimed_wait_common (libc.so.6 + 0x8a70a) #1 0x00007ff982ab0e80 pthread_cond_wait@@GLIBC_2.3.2 (libc.so.6 + 0x8ce80) #2 0x000055f25d6dc14f qemu_cond_wait_impl (qemu-kvm + 0x96714f) #3 0x000055f25d368157 qemu_wait_io_event (qemu-kvm + 0x5f3157) #4 0x000055f25d485001 kvm_vcpu_thread_fn (qemu-kvm + 0x710001) #5 0x000055f25d6dcaea qemu_thread_start (qemu-kvm + 0x967aea) #6 0x00007ff982ab1aaf start_thread (libc.so.6 + 0x8daaf) #7 0x00007ff982b36740 __clone3 (libc.so.6 + 0x112740) Stack trace of thread 1405326: #0 0x00007ff982aae70a __futex_abstimed_wait_common (libc.so.6 + 0x8a70a) #1 0x00007ff982ab0e80 pthread_cond_wait@@GLIBC_2.3.2 (libc.so.6 + 0x8ce80) #2 0x000055f25d6dc14f qemu_cond_wait_impl (qemu-kvm + 0x96714f) #3 0x000055f25d10e657 vnc_worker_thread (qemu-kvm + 0x399657) #4 0x000055f25d6dcaea qemu_thread_start (qemu-kvm + 0x967aea) #5 0x00007ff982ab1aaf start_thread (libc.so.6 + 0x8daaf) #6 0x00007ff982b36740 __clone3 (libc.so.6 + 0x112740)