Bug 1717329
| Summary: | Target qemu coredump after failed to mirror from src to nbd target(data plane enable) | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux Advanced Virtualization | Reporter: | aihua liang <aliang> |
| Component: | qemu-kvm | Assignee: | Kevin Wolf <kwolf> |
| Status: | CLOSED DUPLICATE | QA Contact: | aihua liang <aliang> |
| Severity: | unspecified | Docs Contact: | |
| Priority: | high | ||
| Version: | 8.1 | CC: | coli, jinzhao, juzhang, ngu, qzhang, virt-maint |
| Target Milestone: | rc | ||
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2019-09-04 03:36:14 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
In latest version: qemu-kvm-4.1.0-5.module+el8.1.0+4076+b5e41ebc.x86_64, both src and dst qemu coredump. As the coredump of dst in bz1748253 and bz1717329 is the same, track this issue with bug 1748253, and close this bug as a dup. Thanks, aliang *** This bug has been marked as a duplicate of bug 1748253 *** |
Description of problem: Target qemu coredump after failed to mirror from src to nbd target Version-Release number of selected component (if applicable): kernel version:4.18.0-94.el8.x86_64 qemu-kvm version:qemu-kvm-4.0.0-3.module+el8.1.0+3265+26c4ed71.x86_64 How reproducible: 100% Steps to Reproduce: 1.Start src guest with qemu cmds: /usr/libexec/qemu-kvm \ -name 'avocado-vt-vm1' \ -machine q35 \ -nodefaults \ -device VGA,bus=pcie.0,addr=0x1 \ -chardev socket,id=qmp_id_qmpmonitor1,path=/var/tmp/monitor-qmpmonitor1-20190602-221944-MrlxVzia,server,nowait \ -mon chardev=qmp_id_qmpmonitor1,mode=control \ -chardev socket,id=qmp_id_catch_monitor,path=/var/tmp/monitor-catch_monitor-20190602-221944-MrlxVzia,server,nowait \ -mon chardev=qmp_id_catch_monitor,mode=control \ -device pvpanic,ioport=0x505,id=idn20piu \ -chardev socket,id=serial_id_serial0,path=/var/tmp/serial-serial0-20190602-221944-MrlxVzia,server,nowait \ -device isa-serial,chardev=serial_id_serial0 \ -chardev socket,id=seabioslog_id_20190602-221944-MrlxVzia,path=/var/tmp/seabios-20190602-221944-MrlxVzia,server,nowait \ -device isa-debugcon,chardev=seabioslog_id_20190602-221944-MrlxVzia,iobase=0x402 \ -device pcie-root-port,id=pcie.0-root-port-2,slot=2,chassis=2,addr=0x2,bus=pcie.0 \ -device qemu-xhci,id=usb1,bus=pcie.0-root-port-2,addr=0x0 \ -device pcie-root-port,id=pcie.0-root-port-3,slot=3,chassis=3,addr=0x3,bus=pcie.0 \ -object iothread,id=iothread0 \ -device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pcie.0-root-port-3,addr=0x0,iothread=iothread0 \ -drive id=drive_image1,if=none,snapshot=off,aio=threads,cache=none,format=qcow2,file=/home/kvm_autotest_root/images/rhel77-64-virtio.qcow2 \ -device scsi-hd,id=image1,drive=drive_image1 \ -device pcie-root-port,id=pcie.0-root-port-4,slot=4,chassis=4,addr=0x4,bus=pcie.0 \ -device virtio-net-pci,mac=9a:33:34:35:36:37,id=idj01pFr,vectors=4,netdev=idMgbx8B,bus=pcie.0-root-port-4,addr=0x0 \ -netdev tap,id=idMgbx8B,vhost=on \ -m 7168 \ -smp 4,maxcpus=4,cores=2,threads=1,sockets=2 \ -cpu 'Skylake-Client',+kvm_pv_unhalt \ -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 \ -vnc :0 \ -rtc base=utc,clock=host,driftfix=slew \ -boot order=cdn,once=c,menu=off,strict=off \ -enable-kvm \ -monitor stdio \ -qmp tcp:0:3000,server,nowait \ 2.In dst, create an image by: #qemu-img create -f qcow2 /home/kvm_autotest_root/images/rhel77-64-virtio.qcow2 20G 3.Start dst guest with qemu cmds: /usr/libexec/qemu-kvm \ -name 'avocado-vt-vm1' \ -machine q35 \ -nodefaults \ -device VGA,bus=pcie.0,addr=0x1 \ -chardev socket,id=qmp_id_qmpmonitor1,path=/var/tmp/monitor-qmpmonitor1-20190602-221944-MrlxVzia,server,nowait \ -mon chardev=qmp_id_qmpmonitor1,mode=control \ -chardev socket,id=qmp_id_catch_monitor,path=/var/tmp/monitor-catch_monitor-20190602-221944-MrlxVzia,server,nowait \ -mon chardev=qmp_id_catch_monitor,mode=control \ -device pvpanic,ioport=0x505,id=idn20piu \ -chardev socket,id=serial_id_serial0,path=/var/tmp/serial-serial0-20190602-221944-MrlxVzia,server,nowait \ -device isa-serial,chardev=serial_id_serial0 \ -chardev socket,id=seabioslog_id_20190602-221944-MrlxVzia,path=/var/tmp/seabios-20190602-221944-MrlxVzia,server,nowait \ -device isa-debugcon,chardev=seabioslog_id_20190602-221944-MrlxVzia,iobase=0x402 \ -device pcie-root-port,id=pcie.0-root-port-2,slot=2,chassis=2,addr=0x2,bus=pcie.0 \ -device qemu-xhci,id=usb1,bus=pcie.0-root-port-2,addr=0x0 \ -device pcie-root-port,id=pcie.0-root-port-3,slot=3,chassis=3,addr=0x3,bus=pcie.0 \ -object iothread,id=iothread0 \ -device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pcie.0-root-port-3,addr=0x0,iothread=iothread0 \ -drive id=drive_image1,if=none,snapshot=off,aio=threads,cache=none,format=qcow2,file=/home/kvm_autotest_root/images/rhel77-64-virtio.qcow2 \ -device scsi-hd,id=image1,drive=drive_image1 \ -device pcie-root-port,id=pcie.0-root-port-4,slot=4,chassis=4,addr=0x4,bus=pcie.0 \ -device virtio-net-pci,mac=9a:33:34:35:36:37,id=idj01pFr,vectors=4,netdev=idMgbx8B,bus=pcie.0-root-port-4,addr=0x0 \ -netdev tap,id=idMgbx8B,vhost=on \ -m 7168 \ -smp 4,maxcpus=4,cores=2,threads=1,sockets=2 \ -cpu 'Skylake-Client',+kvm_pv_unhalt \ -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 \ -vnc :0 \ -rtc base=utc,clock=host,driftfix=slew \ -boot order=cdn,once=c,menu=off,strict=off \ -enable-kvm \ -monitor stdio \ -qmp tcp:0:3000,server,nowait \ -incoming tcp:0:5000 \ 4.Expose dst image via nbd. { "execute": "nbd-server-start", "arguments": { "addr": { "type": "inet","data": { "host":"10.73.224.36","port":"3333"}}}} { "execute": "nbd-server-add", "arguments": { "device": "drive_image1","writable": true } } 5.Start mirror from src to dst { "execute": "drive-mirror", "arguments": { "device": "drive_image1","target":"nbd://10.73.224.36:3333/drive_image1", "sync": "full","format": "raw", "mode": "existing" } } {"timestamp": {"seconds": 1559721809, "microseconds": 276972}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "drive_image1"}} {"timestamp": {"seconds": 1559721809, "microseconds": 277051}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "drive_image1"}} {"return": {}} {"timestamp": {"seconds": 1559721809, "microseconds": 564001}, "event": "BLOCK_JOB_ERROR", "data": {"device": "drive_image1", "operation": "write", "action": "report"}} {"timestamp": {"seconds": 1559721809, "microseconds": 564048}, "event": "BLOCK_JOB_ERROR", "data": {"device": "drive_image1", "operation": "write", "action": "report"}} {"timestamp": {"seconds": 1559721809, "microseconds": 564071}, "event": "BLOCK_JOB_ERROR", "data": {"device": "drive_image1", "operation": "write", "action": "report"}} {"timestamp": {"seconds": 1559721809, "microseconds": 564093}, "event": "BLOCK_JOB_ERROR", "data": {"device": "drive_image1", "operation": "write", "action": "report"}} {"timestamp": {"seconds": 1559721809, "microseconds": 564132}, "event": "BLOCK_JOB_ERROR", "data": {"device": "drive_image1", "operation": "write", "action": "report"}} {"timestamp": {"seconds": 1559721809, "microseconds": 564183}, "event": "BLOCK_JOB_ERROR", "data": {"device": "drive_image1", "operation": "write", "action": "report"}} {"timestamp": {"seconds": 1559721809, "microseconds": 564211}, "event": "BLOCK_JOB_ERROR", "data": {"device": "drive_image1", "operation": "write", "action": "report"}} {"timestamp": {"seconds": 1559721809, "microseconds": 564263}, "event": "BLOCK_JOB_ERROR", "data": {"device": "drive_image1", "operation": "write", "action": "report"}} {"timestamp": {"seconds": 1559721809, "microseconds": 564329}, "event": "BLOCK_JOB_ERROR", "data": {"device": "drive_image1", "operation": "write", "action": "report"}} {"timestamp": {"seconds": 1559721809, "microseconds": 564368}, "event": "BLOCK_JOB_ERROR", "data": {"device": "drive_image1", "operation": "write", "action": "report"}} {"timestamp": {"seconds": 1559721809, "microseconds": 564404}, "event": "BLOCK_JOB_ERROR", "data": {"device": "drive_image1", "operation": "write", "action": "report"}} {"timestamp": {"seconds": 1559721809, "microseconds": 564425}, "event": "BLOCK_JOB_ERROR", "data": {"device": "drive_image1", "operation": "write", "action": "report"}} {"timestamp": {"seconds": 1559721809, "microseconds": 564479}, "event": "BLOCK_JOB_ERROR", "data": {"device": "drive_image1", "operation": "write", "action": "report"}} {"timestamp": {"seconds": 1559721809, "microseconds": 564502}, "event": "BLOCK_JOB_ERROR", "data": {"device": "drive_image1", "operation": "write", "action": "report"}} {"timestamp": {"seconds": 1559721809, "microseconds": 564525}, "event": "BLOCK_JOB_ERROR", "data": {"device": "drive_image1", "operation": "write", "action": "report"}} {"timestamp": {"seconds": 1559721809, "microseconds": 564565}, "event": "BLOCK_JOB_ERROR", "data": {"device": "drive_image1", "operation": "write", "action": "report"}} {"timestamp": {"seconds": 1559721809, "microseconds": 564833}, "event": "JOB_STATUS_CHANGE", "data": {"status": "aborting", "id": "drive_image1"}} {"timestamp": {"seconds": 1559721809, "microseconds": 565166}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "drive_image1", "len": 21474836480, "offset": 0, "speed": 0, "type": "mirror", "error": "Input/output error"}} {"timestamp": {"seconds": 1559721809, "microseconds": 565232}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "drive_image1"}} {"timestamp": {"seconds": 1559721809, "microseconds": 565273}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "drive_image1"}} Actual results: After step5, block mirror failed with error "Input/output error". And after that, the target qemu coredump with info: (qemu) qemu-kvm: io/channel.c:410: qio_channel_restart_read: Assertion `qemu_get_current_aio_context() == qemu_coroutine_get_aio_context(co)' failed. live_snapshot.txt: line 36: 4800 Aborted (core dumped) /usr/libexec/qemu-kvm -name 'avocado-vt-vm1' -machine q35 -nodefaults -device VGA,bus=pcie.0,addr=0x1 coredump info as bellow: (gdb) bt #0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50 #1 0x00007fa6de789c95 in __GI_abort () at abort.c:79 #2 0x00007fa6de789b69 in __assert_fail_base (fmt=0x7fa6de8f0940 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=0x564e91249360 "qemu_get_current_aio_context() == qemu_coroutine_get_aio_context(co)", file=0x564e91249326 "io/channel.c", line=410, function=<optimized out>) at assert.c:92 #3 0x00007fa6de797df6 in __GI___assert_fail ( assertion=assertion@entry=0x564e91249360 "qemu_get_current_aio_context() == qemu_coroutine_get_aio_context(co)", file=file@entry=0x564e91249326 "io/channel.c", line=line@entry=410, function=function@entry=0x564e91249470 <__PRETTY_FUNCTION__.18120> "qio_channel_restart_read") at assert.c:101 #4 0x0000564e91059dd0 in qio_channel_restart_read (opaque=<optimized out>) at io/channel.c:409 #5 qio_channel_restart_read (opaque=<optimized out>) at io/channel.c:403 #6 0x0000564e910ad662 in aio_dispatch_handlers (ctx=ctx@entry=0x564e92e88bc0) at util/aio-posix.c:430 #7 0x0000564e910ae00c in aio_dispatch (ctx=0x564e92e88bc0) at util/aio-posix.c:461 #8 0x0000564e910aaaa2 in aio_ctx_dispatch (source=<optimized out>, callback=<optimized out>, user_data=<optimized out>) at util/async.c:261 #9 0x00007fa6e36125ed in g_main_dispatch (context=0x564e92f01a00) at gmain.c:3176 #10 g_main_context_dispatch (context=context@entry=0x564e92f01a00) at gmain.c:3829 #11 0x0000564e910ad1e8 in glib_pollfds_poll () at util/main-loop.c:213 #12 os_host_main_loop_wait (timeout=<optimized out>) at util/main-loop.c:236 #13 main_loop_wait (nonblocking=<optimized out>) at util/main-loop.c:512 #14 0x0000564e90ea2579 in main_loop () at vl.c:1988 #15 0x0000564e90d57078 in main (argc=<optimized out>, argv=<optimized out>, envp=<optimized out>) at vl.c:4637 Expected results: Mirror to nbd target successfully. Additional info: When disable dataplane, don't hit this issue.