Note: This bug is displayed in read-only format because
the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Created attachment 1457425[details]
full_cmd_line
Description of problem:
qemu core dumped when virtio-scsi-pci with 'iommu=on'
Version-Release number of selected component (if applicable):
3.10.0-919.el7.x86_64
qemu-kvm-rhev-2.12.0-7.el7.x86_64
How reproducible:
100%
Steps to Reproduce:
1.Boot VM with 'iommu=on':
/usr/libexec/qemu-kvm \
-machine q35,kernel-irqchip=split \
......
-device pcie-root-port,id=pcie.0-root-port-6,slot=6,chassis=6,addr=0x6,bus=pcie.0 \
-device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pcie.0-root-port-6,addr=0x0,iommu_platform=on,ats=on \
-drive id=drive_image1,if=none,snapshot=off,aio=threads,cache=none,format=qcow2,file=/home/kvm_autotest_root/images/rhel76-64-virtio-scsi.qcow2 \
-device scsi-hd,id=image1,drive=drive_image1 \
-device intel-iommu,device-iotlb=on,intremap=on \
......
Actual results:
(qemu) Bad ram offset 562c064d80f0
q35_iommu_vhost.sh: line 39: 2706 Aborted (core dumped) /usr/libexec/qemu-kvm
Expected results:
No core dump
Additional info:
1.Full command line is attached.
2.(gdb) bt
#0 0x00007f6324322207 in raise () at /lib64/libc.so.6
#1 0x00007f63243238f8 in abort () at /lib64/libc.so.6
#2 0x0000562c032063da in qemu_get_ram_block (addr=<optimized out>) at /usr/src/debug/qemu-2.12.0/exec.c:1115
#3 0x0000562c0320a185 in qemu_map_ram_ptr (ram_block=<optimized out>, addr=94747084292336)
at /usr/src/debug/qemu-2.12.0/exec.c:2288
#4 0x0000562c03210635 in lduw_le_phys_cached (cache=<optimized out>, cache=<optimized out>, endian=
DEVICE_LITTLE_ENDIAN, result=0x0, attrs=..., addr=0) at /usr/src/debug/qemu-2.12.0/memory_ldst.inc.c:281
#5 0x0000562c03210635 in lduw_le_phys_cached (result=0x0, attrs=..., addr=0, cache=0x562c03da58a0 <io_mem_unassigned>)
at /usr/src/debug/qemu-2.12.0/memory_ldst.inc.c:315
#6 0x0000562c03210635 in lduw_le_phys_cached (cache=cache@entry=0x562c06666760, addr=addr@entry=0)
at /usr/src/debug/qemu-2.12.0/memory_ldst.inc.c:334
#7 0x0000562c032a3994 in virtio_queue_set_notification (vdev=<optimized out>, pa=0, cache=0x562c06666760)
at /usr/src/debug/qemu-2.12.0/include/hw/virtio/virtio-access.h:166
#8 0x0000562c032a3994 in virtio_queue_set_notification (mask=1, vq=0x562c0836e100)
at /usr/src/debug/qemu-2.12.0/hw/virtio/virtio.c:282
#9 0x0000562c032a3994 in virtio_queue_set_notification (vq=vq@entry=0x562c0836e100, enable=enable@entry=0)
at /usr/src/debug/qemu-2.12.0/hw/virtio/virtio.c:329
#10 0x0000562c032a3b32 in virtio_queue_set_notification (vq=vq@entry=0x562c0836e100, enable=enable@entry=0)
at /usr/src/debug/qemu-2.12.0/hw/virtio/virtio.c:319
#11 0x0000562c03294e62 in virtio_scsi_handle_cmd_vq (s=s@entry=0x562c08366170, vq=vq@entry=0x562c0836e100)
at /usr/src/debug/qemu-2.12.0/hw/scsi/virtio-scsi.c:595
#12 0x0000562c03295d7a in virtio_scsi_data_plane_handle_cmd (vdev=<optimized out>, vq=0x562c0836e100)
at /usr/src/debug/qemu-2.12.0/hw/scsi/virtio-scsi-dataplane.c:60
#13 0x0000562c032a3d76 in virtio_queue_host_notifier_aio_poll (vq=0x562c0836e100)
at /usr/src/debug/qemu-2.12.0/hw/virtio/virtio.c:1512
#14 0x0000562c032a3d76 in virtio_queue_host_notifier_aio_poll (opaque=0x562c0836e168)
at /usr/src/debug/qemu-2.12.0/hw/virtio/virtio.c:2427
#15 0x0000562c03558a2e in run_poll_handlers_once (ctx=ctx@entry=0x562c064117c0) at util/aio-posix.c:497
#16 0x0000562c03559475 in aio_poll (blocking=true, ctx=0x562c064117c0) at util/aio-posix.c:573
#17 0x0000562c03559475 in aio_poll (ctx=ctx@entry=0x562c064117c0, blocking=<optimized out>) at util/aio-posix.c:602
---Type <return> to continue, or q <return> to quit---
#18 0x0000562c03556b3f in aio_wait_bh_oneshot (ctx=0x562c064117c0, cb=cb@entry=0x562c03295b90 <virtio_scsi_dataplane_stop_bh>, opaque=opaque@entry=0x562c08366170) at util/aio-wait.c:70
#19 0x0000562c0329644a in virtio_scsi_dataplane_stop (vdev=0x562c08366170)
at /usr/src/debug/qemu-2.12.0/hw/scsi/virtio-scsi-dataplane.c:211
#20 0x0000562c0341ea65 in virtio_bus_stop_ioeventfd (bus=bus@entry=0x562c083660f8) at hw/virtio/virtio-bus.c:246
#21 0x0000562c0341bdd1 in virtio_pci_common_write (proxy=0x562c0835e000) at hw/virtio/virtio-pci.c:294
#22 0x0000562c0341bdd1 in virtio_pci_common_write (opaque=0x562c0835e000, addr=<optimized out>, val=0, size=<optimized out>)
at hw/virtio/virtio-pci.c:1283
#23 0x0000562c03255ae3 in memory_region_write_accessor (mr=<optimized out>, addr=<optimized out>, value=<optimized out>, size=<optimized out>, shift=<optimized out>, mask=<optimized out>, attrs=...) at /usr/src/debug/qemu-2.12.0/memory.c:530
#24 0x0000562c032537f9 in access_with_adjusted_size (addr=addr@entry=20, value=value@entry=0x7f631a84e708, size=size@entry=1, access_size_min=<optimized out>, access_size_max=<optimized out>, access_fn=access_fn@entry=
0x562c03255aa0 <memory_region_write_accessor>, mr=mr@entry=0x562c0835e9d0, attrs=attrs@entry=...)
at /usr/src/debug/qemu-2.12.0/memory.c:597
#25 0x0000562c03257895 in memory_region_dispatch_write (mr=mr@entry=0x562c0835e9d0, addr=addr@entry=20, data=0, size=size@entry=1, attrs=attrs@entry=...) at /usr/src/debug/qemu-2.12.0/memory.c:1474
#26 0x0000562c032077cb in flatview_write (mr=0x562c0835e9d0, l=1, addr1=20, len=1, buf=0x7f633d3b3028 <Address 0x7f633d3b3028 out of bounds>, attrs=..., addr=4229955604, fv=0x562c091e53c0) at /usr/src/debug/qemu-2.12.0/exec.c:3100
#27 0x0000562c032077cb in flatview_write (fv=0x562c091e53c0, addr=<optimized out>, attrs=..., buf=<optimized out>, len=<optimized out>) at /usr/src/debug/qemu-2.12.0/exec.c:3144
#28 0x0000562c0320b20f in address_space_write (as=<optimized out>, addr=<optimized out>, attrs=..., buf=<optimized out>, len=<optimized out>) at /usr/src/debug/qemu-2.12.0/exec.c:3260
#29 0x0000562c0320b2b5 in address_space_rw (as=<optimized out>, addr=<optimized out>, attrs=...,
attrs@entry=..., buf=buf@entry=0x7f633d3b3028 <Address 0x7f633d3b3028 out of bounds>, len=<optimized out>, is_write=<optimized out>) at /usr/src/debug/qemu-2.12.0/exec.c:3271
#30 0x0000562c03265ea8 in kvm_cpu_exec (cpu=cpu@entry=0x562c066f6000) at /usr/src/debug/qemu-2.12.0/accel/kvm/kvm-all.c:1992
#31 0x0000562c03243996 in qemu_kvm_cpu_thread_fn (arg=0x562c066f6000) at /usr/src/debug/qemu-2.12.0/cpus.c:1215
#32 0x00007f63246c0dd5 in start_thread () at /lib64/libpthread.so.0
#33 0x00007f63243e9ead in clone () at /lib64/libc.so.6
3.It works well with qemu-kvm-rhev-2.10.0-21.el7_5.1.x86_64.
4.From qemu-kvm-rhev-2.10.0-21.el7_5.2.x86_64 to qemu-kvm-rhev-2.12.0-4.el7.x86_64, guest hangs and cannot boot up successfully.
(qemu) qemu-kvm: Guest says index 65363 is available
# ps aux | grep qemu
root 6111 143 7.0 5002584 575128 pts/5 Sl+ 16:43 1:30 /usr/libexec/qemu-kvm
# gdb attach 6111
(gdb) bt
#0 0x00007fb3cccd44dd in __lll_lock_wait () at /lib64/libpthread.so.0
#1 0x00007fb3ccccfdcb in _L_lock_883 () at /lib64/libpthread.so.0
#2 0x00007fb3ccccfc98 in pthread_mutex_lock () at /lib64/libpthread.so.0
#3 0x0000558a7749b9ff in qemu_mutex_lock (mutex=mutex@entry=0x558a77a61ba0 <qemu_global_mutex>)
at util/qemu-thread-posix.c:65
#4 0x0000558a771b293c in qemu_mutex_lock_iothread () at /usr/src/debug/qemu-2.10.0/cpus.c:1591
#5 0x0000558a77498fbf in main_loop_wait (timeout=29261337) at util/main-loop.c:258
#6 0x0000558a77498fbf in main_loop_wait (nonblocking=nonblocking@entry=0) at util/main-loop.c:515
#7 0x0000558a7717899a in main () at vl.c:1937
#8 0x0000558a7717899a in main (argc=<optimized out>, argv=<optimized out>, envp=<optimized out>) at vl.c:4828
5.It can be reproduced with qemu-kvm-rhev-2.12.0-5.el7.x86_64.
Created attachment 1457425 [details] full_cmd_line Description of problem: qemu core dumped when virtio-scsi-pci with 'iommu=on' Version-Release number of selected component (if applicable): 3.10.0-919.el7.x86_64 qemu-kvm-rhev-2.12.0-7.el7.x86_64 How reproducible: 100% Steps to Reproduce: 1.Boot VM with 'iommu=on': /usr/libexec/qemu-kvm \ -machine q35,kernel-irqchip=split \ ...... -device pcie-root-port,id=pcie.0-root-port-6,slot=6,chassis=6,addr=0x6,bus=pcie.0 \ -device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pcie.0-root-port-6,addr=0x0,iommu_platform=on,ats=on \ -drive id=drive_image1,if=none,snapshot=off,aio=threads,cache=none,format=qcow2,file=/home/kvm_autotest_root/images/rhel76-64-virtio-scsi.qcow2 \ -device scsi-hd,id=image1,drive=drive_image1 \ -device intel-iommu,device-iotlb=on,intremap=on \ ...... Actual results: (qemu) Bad ram offset 562c064d80f0 q35_iommu_vhost.sh: line 39: 2706 Aborted (core dumped) /usr/libexec/qemu-kvm Expected results: No core dump Additional info: 1.Full command line is attached. 2.(gdb) bt #0 0x00007f6324322207 in raise () at /lib64/libc.so.6 #1 0x00007f63243238f8 in abort () at /lib64/libc.so.6 #2 0x0000562c032063da in qemu_get_ram_block (addr=<optimized out>) at /usr/src/debug/qemu-2.12.0/exec.c:1115 #3 0x0000562c0320a185 in qemu_map_ram_ptr (ram_block=<optimized out>, addr=94747084292336) at /usr/src/debug/qemu-2.12.0/exec.c:2288 #4 0x0000562c03210635 in lduw_le_phys_cached (cache=<optimized out>, cache=<optimized out>, endian= DEVICE_LITTLE_ENDIAN, result=0x0, attrs=..., addr=0) at /usr/src/debug/qemu-2.12.0/memory_ldst.inc.c:281 #5 0x0000562c03210635 in lduw_le_phys_cached (result=0x0, attrs=..., addr=0, cache=0x562c03da58a0 <io_mem_unassigned>) at /usr/src/debug/qemu-2.12.0/memory_ldst.inc.c:315 #6 0x0000562c03210635 in lduw_le_phys_cached (cache=cache@entry=0x562c06666760, addr=addr@entry=0) at /usr/src/debug/qemu-2.12.0/memory_ldst.inc.c:334 #7 0x0000562c032a3994 in virtio_queue_set_notification (vdev=<optimized out>, pa=0, cache=0x562c06666760) at /usr/src/debug/qemu-2.12.0/include/hw/virtio/virtio-access.h:166 #8 0x0000562c032a3994 in virtio_queue_set_notification (mask=1, vq=0x562c0836e100) at /usr/src/debug/qemu-2.12.0/hw/virtio/virtio.c:282 #9 0x0000562c032a3994 in virtio_queue_set_notification (vq=vq@entry=0x562c0836e100, enable=enable@entry=0) at /usr/src/debug/qemu-2.12.0/hw/virtio/virtio.c:329 #10 0x0000562c032a3b32 in virtio_queue_set_notification (vq=vq@entry=0x562c0836e100, enable=enable@entry=0) at /usr/src/debug/qemu-2.12.0/hw/virtio/virtio.c:319 #11 0x0000562c03294e62 in virtio_scsi_handle_cmd_vq (s=s@entry=0x562c08366170, vq=vq@entry=0x562c0836e100) at /usr/src/debug/qemu-2.12.0/hw/scsi/virtio-scsi.c:595 #12 0x0000562c03295d7a in virtio_scsi_data_plane_handle_cmd (vdev=<optimized out>, vq=0x562c0836e100) at /usr/src/debug/qemu-2.12.0/hw/scsi/virtio-scsi-dataplane.c:60 #13 0x0000562c032a3d76 in virtio_queue_host_notifier_aio_poll (vq=0x562c0836e100) at /usr/src/debug/qemu-2.12.0/hw/virtio/virtio.c:1512 #14 0x0000562c032a3d76 in virtio_queue_host_notifier_aio_poll (opaque=0x562c0836e168) at /usr/src/debug/qemu-2.12.0/hw/virtio/virtio.c:2427 #15 0x0000562c03558a2e in run_poll_handlers_once (ctx=ctx@entry=0x562c064117c0) at util/aio-posix.c:497 #16 0x0000562c03559475 in aio_poll (blocking=true, ctx=0x562c064117c0) at util/aio-posix.c:573 #17 0x0000562c03559475 in aio_poll (ctx=ctx@entry=0x562c064117c0, blocking=<optimized out>) at util/aio-posix.c:602 ---Type <return> to continue, or q <return> to quit--- #18 0x0000562c03556b3f in aio_wait_bh_oneshot (ctx=0x562c064117c0, cb=cb@entry=0x562c03295b90 <virtio_scsi_dataplane_stop_bh>, opaque=opaque@entry=0x562c08366170) at util/aio-wait.c:70 #19 0x0000562c0329644a in virtio_scsi_dataplane_stop (vdev=0x562c08366170) at /usr/src/debug/qemu-2.12.0/hw/scsi/virtio-scsi-dataplane.c:211 #20 0x0000562c0341ea65 in virtio_bus_stop_ioeventfd (bus=bus@entry=0x562c083660f8) at hw/virtio/virtio-bus.c:246 #21 0x0000562c0341bdd1 in virtio_pci_common_write (proxy=0x562c0835e000) at hw/virtio/virtio-pci.c:294 #22 0x0000562c0341bdd1 in virtio_pci_common_write (opaque=0x562c0835e000, addr=<optimized out>, val=0, size=<optimized out>) at hw/virtio/virtio-pci.c:1283 #23 0x0000562c03255ae3 in memory_region_write_accessor (mr=<optimized out>, addr=<optimized out>, value=<optimized out>, size=<optimized out>, shift=<optimized out>, mask=<optimized out>, attrs=...) at /usr/src/debug/qemu-2.12.0/memory.c:530 #24 0x0000562c032537f9 in access_with_adjusted_size (addr=addr@entry=20, value=value@entry=0x7f631a84e708, size=size@entry=1, access_size_min=<optimized out>, access_size_max=<optimized out>, access_fn=access_fn@entry= 0x562c03255aa0 <memory_region_write_accessor>, mr=mr@entry=0x562c0835e9d0, attrs=attrs@entry=...) at /usr/src/debug/qemu-2.12.0/memory.c:597 #25 0x0000562c03257895 in memory_region_dispatch_write (mr=mr@entry=0x562c0835e9d0, addr=addr@entry=20, data=0, size=size@entry=1, attrs=attrs@entry=...) at /usr/src/debug/qemu-2.12.0/memory.c:1474 #26 0x0000562c032077cb in flatview_write (mr=0x562c0835e9d0, l=1, addr1=20, len=1, buf=0x7f633d3b3028 <Address 0x7f633d3b3028 out of bounds>, attrs=..., addr=4229955604, fv=0x562c091e53c0) at /usr/src/debug/qemu-2.12.0/exec.c:3100 #27 0x0000562c032077cb in flatview_write (fv=0x562c091e53c0, addr=<optimized out>, attrs=..., buf=<optimized out>, len=<optimized out>) at /usr/src/debug/qemu-2.12.0/exec.c:3144 #28 0x0000562c0320b20f in address_space_write (as=<optimized out>, addr=<optimized out>, attrs=..., buf=<optimized out>, len=<optimized out>) at /usr/src/debug/qemu-2.12.0/exec.c:3260 #29 0x0000562c0320b2b5 in address_space_rw (as=<optimized out>, addr=<optimized out>, attrs=..., attrs@entry=..., buf=buf@entry=0x7f633d3b3028 <Address 0x7f633d3b3028 out of bounds>, len=<optimized out>, is_write=<optimized out>) at /usr/src/debug/qemu-2.12.0/exec.c:3271 #30 0x0000562c03265ea8 in kvm_cpu_exec (cpu=cpu@entry=0x562c066f6000) at /usr/src/debug/qemu-2.12.0/accel/kvm/kvm-all.c:1992 #31 0x0000562c03243996 in qemu_kvm_cpu_thread_fn (arg=0x562c066f6000) at /usr/src/debug/qemu-2.12.0/cpus.c:1215 #32 0x00007f63246c0dd5 in start_thread () at /lib64/libpthread.so.0 #33 0x00007f63243e9ead in clone () at /lib64/libc.so.6 3.It works well with qemu-kvm-rhev-2.10.0-21.el7_5.1.x86_64. 4.From qemu-kvm-rhev-2.10.0-21.el7_5.2.x86_64 to qemu-kvm-rhev-2.12.0-4.el7.x86_64, guest hangs and cannot boot up successfully. (qemu) qemu-kvm: Guest says index 65363 is available # ps aux | grep qemu root 6111 143 7.0 5002584 575128 pts/5 Sl+ 16:43 1:30 /usr/libexec/qemu-kvm # gdb attach 6111 (gdb) bt #0 0x00007fb3cccd44dd in __lll_lock_wait () at /lib64/libpthread.so.0 #1 0x00007fb3ccccfdcb in _L_lock_883 () at /lib64/libpthread.so.0 #2 0x00007fb3ccccfc98 in pthread_mutex_lock () at /lib64/libpthread.so.0 #3 0x0000558a7749b9ff in qemu_mutex_lock (mutex=mutex@entry=0x558a77a61ba0 <qemu_global_mutex>) at util/qemu-thread-posix.c:65 #4 0x0000558a771b293c in qemu_mutex_lock_iothread () at /usr/src/debug/qemu-2.10.0/cpus.c:1591 #5 0x0000558a77498fbf in main_loop_wait (timeout=29261337) at util/main-loop.c:258 #6 0x0000558a77498fbf in main_loop_wait (nonblocking=nonblocking@entry=0) at util/main-loop.c:515 #7 0x0000558a7717899a in main () at vl.c:1937 #8 0x0000558a7717899a in main (argc=<optimized out>, argv=<optimized out>, envp=<optimized out>) at vl.c:4828 5.It can be reproduced with qemu-kvm-rhev-2.12.0-5.el7.x86_64.