Note: This bug is displayed in read-only format because
the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Created attachment 934364[details]
under gdb, bt ful prints.
Description of problem:
RHEL guest with vhost=off and sudbuf=100, if during pktgen stress from guest to host, then pause guest (stop it), qemu crashed
enable vhost won't hit such issue.
Version-Release number of selected component (if applicable):
# uname -r
2.6.32-500.el6.x86_64
# rpm -q qemu-kvm
qemu-kvm-0.12.1.2-2.443.el6.x86_64
How reproducible:
100%
Steps to Reproduce:
1.Boot guest with sndbuf=100 and vhost=off
# /usr/libexec/qemu-kvm -cpu Penryn -enable-kvm -m 4G -smp 4,sockets=1,cores=4,threads=1 -name test -rtc base=localtime,clock=host,driftfix=slew -k en-us -boot menu=on -vnc :3 -vga cirrus -usb -device usb-tablet -monitor stdio -drive file=/home/rhel7-64.qcow2,if=none,id=drive-system-disk,media=disk,format=qcow2,aio=native,werror=stop,rerror=stop -device virtio-blk-pci,drive=drive-system-disk,id=system-disk,addr=0x3 -qmp unix:/tmp/q1,server,nowait -netdev tap,id=hostnet0,vhost=off,sndbuf=100,script=/etc/qemu-ifup -device virtio-net-pci,netdev=hostnet0,id=vnet0,mac=52:54:00:1a:2b:01
2.In guest, start pktgen test to host
3.stop guest under hmp
(qemu) stop
Actual results:
qemu crashed:
(qemu) qemu-kvm: /builddir/build/BUILD/qemu-kvm-0.12.1.2/hw/virtio-net.c:814: virtio_net_flush_tx: Assertion `n->vdev.vm_running' failed.
(gdb) bt
#0 0x00007ffff483c915 in raise () from /lib64/libc.so.6
#1 0x00007ffff483e0f5 in abort () from /lib64/libc.so.6
#2 0x00007ffff4835a3e in __assert_fail_base () from /lib64/libc.so.6
#3 0x00007ffff4835b00 in __assert_fail () from /lib64/libc.so.6
#4 0x00007ffff7dcab8e in virtio_net_flush_tx (n=0x7ffff9cdb920,
vq=0x7ffff9ce7ab0) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/virtio-net.c:814
#5 0x00007ffff7e2836e in qemu_net_queue_flush (queue=0x7ffff86f9910)
at /usr/src/debug/qemu-kvm-0.12.1.2/net/queue.c:255
#6 0x00007ffff7e2780d in qemu_flush_queued_packets (vc=<value optimized out>)
at /usr/src/debug/qemu-kvm-0.12.1.2/net.c:529
#7 0x00007ffff7db39f6 in main_loop_wait (timeout=1000)
at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:4060
#8 0x00007ffff7dd740a in kvm_main_loop ()
at /usr/src/debug/qemu-kvm-0.12.1.2/qemu-kvm.c:2258
#9 0x00007ffff7db8687 in main_loop (argc=<value optimized out>,
argv=<value optimized out>, envp=<value optimized out>)
at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:4268
#10 main (argc=<value optimized out>, argv=<value optimized out>,
envp=<value optimized out>) at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:6725
Expected results:
No crash, guest and host can work well
Additional info:
Test with vhost=on, not hit such issue
Test in RHEL7 environment, no such issue(both test with qemu-kvm-1.5.3-69.el7.x86_64 and qemu-kvm-rhev-2.1.0-3.el7.x86_64)
Hi Jason
I found it interesting that when I file this bug, I am using the script as attachment, in fact it is a wrong test that I fill the wrong mac address to the dst mac address, then hit such issue, and if I fill the right one, will not hit such issue.
So I retest it in rhel7(when I file this bug, I use right dst mac address in testing rhel7 and use wrong mac in testing rhel6 ) with wrong dst mac address, I can hit such bug, I will file it against the rhel7.
So the steps that can hit this issue will change in comment 0
For step2, when test pktgen from guest to host, I used wrong "dst mac address" in the script.
e.g.
the mac address (of switch in host) that map the ip address "10.66.11.12" in host is
80:C1:6E:F5:3B:29
And when I fill it in the script, I used a wrong mac address 80:C1:6E:F5:3B:21
#cat pktgen.sh
...
pgset "dst 10.66.11.12"
pgset "dst_mac 80:C1:6E:F5:3B:21"
...
Hi Jason
I am not sure if this is the bug you told me via irc.
(In reply to Qian Guo from comment #4)
> Hi Jason
>
> I found it interesting that when I file this bug, I am using the script as
> attachment, in fact it is a wrong test that I fill the wrong mac address to
> the dst mac address, then hit such issue, and if I fill the right one, will
> not hit such issue.
>
> So I retest it in rhel7(when I file this bug, I use right dst mac address in
> testing rhel7 and use wrong mac in testing rhel6 ) with wrong dst mac
> address, I can hit such bug, I will file it against the rhel7.
>
> So the steps that can hit this issue will change in comment 0
>
> For step2, when test pktgen from guest to host, I used wrong "dst mac
> address" in the script.
> e.g.
>
> the mac address (of switch in host) that map the ip address "10.66.11.12" in
> host is
> 80:C1:6E:F5:3B:29
> And when I fill it in the script, I used a wrong mac address
> 80:C1:6E:F5:3B:21
> #cat pktgen.sh
> ...
> pgset "dst 10.66.11.12"
> pgset "dst_mac 80:C1:6E:F5:3B:21"
> ...
>
> Hi Jason
>
> I am not sure if this is the bug you told me via irc.
I think it's the same issue.
Thanks
Reproduce this bug with qemu-kvm-0.12.1.2-2.443.el6.x86_64
And verified that it is fixed by qemu-kvm-0.12.1.2-2.452.el6
Steps:
1.Boot guest with vhost=off and sndbuf=100
# /usr/libexec/qemu-kvm -cpu Penryn -enable-kvm -m 4G -smp 4,sockets=1,cores=4,threads=1 -name test -rtc base=localtime,clock=host,driftfix=slew -k en-us -boot menu=on -vnc :3 -vga cirrus -usb -device usb-tablet -monitor stdio -drive file=/home/RHEL-Server-6.6-64.qcow2,if=none,id=drive-system-disk,media=disk,format=qcow2,aio=native,werror=stop,rerror=stop -device virtio-blk-pci,drive=drive-system-disk,id=system-disk,addr=0x3 -qmp unix:/tmp/q1,server,nowait -netdev tap,id=hostnet0,vhost=off,sndbuf=100,script=/etc/qemu-ifup -device virtio-net-pci,netdev=hostnet0,id=vnet0,mac=52:54:00:1a:2b:01
2.Set the dst_mac as a wrong one in the pktgen.sh, and dst_ip as the host.
3.Do pktgen test in guest
4. Stop the qemu
result:
with qemu-kvm-0.12.1.2-2.443.el6.x86_64, qemu coredumpd:
(qemu) qemu-kvm: /builddir/build/BUILD/qemu-kvm-0.12.1.2/hw/virtio-net.c:814: virtio_net_flush_tx: Assertion `n->vdev.vm_running' failed.
Aborted (core dumped)
And with qemu-kvm-0.12.1.2-2.452.el6, guest works well.
Created attachment 934364 [details] under gdb, bt ful prints. Description of problem: RHEL guest with vhost=off and sudbuf=100, if during pktgen stress from guest to host, then pause guest (stop it), qemu crashed enable vhost won't hit such issue. Version-Release number of selected component (if applicable): # uname -r 2.6.32-500.el6.x86_64 # rpm -q qemu-kvm qemu-kvm-0.12.1.2-2.443.el6.x86_64 How reproducible: 100% Steps to Reproduce: 1.Boot guest with sndbuf=100 and vhost=off # /usr/libexec/qemu-kvm -cpu Penryn -enable-kvm -m 4G -smp 4,sockets=1,cores=4,threads=1 -name test -rtc base=localtime,clock=host,driftfix=slew -k en-us -boot menu=on -vnc :3 -vga cirrus -usb -device usb-tablet -monitor stdio -drive file=/home/rhel7-64.qcow2,if=none,id=drive-system-disk,media=disk,format=qcow2,aio=native,werror=stop,rerror=stop -device virtio-blk-pci,drive=drive-system-disk,id=system-disk,addr=0x3 -qmp unix:/tmp/q1,server,nowait -netdev tap,id=hostnet0,vhost=off,sndbuf=100,script=/etc/qemu-ifup -device virtio-net-pci,netdev=hostnet0,id=vnet0,mac=52:54:00:1a:2b:01 2.In guest, start pktgen test to host 3.stop guest under hmp (qemu) stop Actual results: qemu crashed: (qemu) qemu-kvm: /builddir/build/BUILD/qemu-kvm-0.12.1.2/hw/virtio-net.c:814: virtio_net_flush_tx: Assertion `n->vdev.vm_running' failed. (gdb) bt #0 0x00007ffff483c915 in raise () from /lib64/libc.so.6 #1 0x00007ffff483e0f5 in abort () from /lib64/libc.so.6 #2 0x00007ffff4835a3e in __assert_fail_base () from /lib64/libc.so.6 #3 0x00007ffff4835b00 in __assert_fail () from /lib64/libc.so.6 #4 0x00007ffff7dcab8e in virtio_net_flush_tx (n=0x7ffff9cdb920, vq=0x7ffff9ce7ab0) at /usr/src/debug/qemu-kvm-0.12.1.2/hw/virtio-net.c:814 #5 0x00007ffff7e2836e in qemu_net_queue_flush (queue=0x7ffff86f9910) at /usr/src/debug/qemu-kvm-0.12.1.2/net/queue.c:255 #6 0x00007ffff7e2780d in qemu_flush_queued_packets (vc=<value optimized out>) at /usr/src/debug/qemu-kvm-0.12.1.2/net.c:529 #7 0x00007ffff7db39f6 in main_loop_wait (timeout=1000) at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:4060 #8 0x00007ffff7dd740a in kvm_main_loop () at /usr/src/debug/qemu-kvm-0.12.1.2/qemu-kvm.c:2258 #9 0x00007ffff7db8687 in main_loop (argc=<value optimized out>, argv=<value optimized out>, envp=<value optimized out>) at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:4268 #10 main (argc=<value optimized out>, argv=<value optimized out>, envp=<value optimized out>) at /usr/src/debug/qemu-kvm-0.12.1.2/vl.c:6725 Expected results: No crash, guest and host can work well Additional info: Test with vhost=on, not hit such issue Test in RHEL7 environment, no such issue(both test with qemu-kvm-1.5.3-69.el7.x86_64 and qemu-kvm-rhev-2.1.0-3.el7.x86_64)