Bug 676579
| Summary: | virtio_net: missing schedule on oom | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 6 | Reporter: | Michael S. Tsirkin <mst> |
| Component: | kernel | Assignee: | Michael S. Tsirkin <mst> |
| Status: | CLOSED ERRATA | QA Contact: | Virtualization Bugs <virt-bugs> |
| Severity: | unspecified | Docs Contact: | |
| Priority: | urgent | ||
| Version: | 6.0 | CC: | arozansk, dhoward, jasowang, khong, ndai, plyons, tburke |
| Target Milestone: | rc | Keywords: | Triaged, ZStream |
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | kernel-2.6.32-117.el6 | Doc Type: | Bug Fix |
| Doc Text: |
Intensive usage of resources on a guest lead to a failure of networking on that guest: packets could no longer be received. The failure occurred when a DMA (Direct Memory Access) ring was consumed before NAPI (New API; an interface for networking devices which makes use of interrupt mitigation techniques) was enabled which resulted in a failure to receive the next interrupt request. The regular interrupt handler was not affected in this situation (because it can process packets in-place), however, the OOM (Out Of Memory) handler did not detect the aforementioned situation and caused networking to fail. With this update, NAPI is subsequently scheduled for each napi_enable operation; thus, networking no longer fails under the aforementioned circumstances.
|
Story Points: | --- |
| Clone Of: | Environment: | ||
| Last Closed: | 2011-05-19 12:49:11 UTC | Type: | --- |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | |||
| Bug Blocks: | 684268 | ||
|
Description
Michael S. Tsirkin
2011-02-10 09:58:50 UTC
This request was evaluated by Red Hat Product Management for inclusion in a Red Hat Enterprise Linux maintenance release. Product Management has requested further review of this request by Red Hat Engineering, for potential inclusion in a Red Hat Enterprise Linux Update release for currently deployed products. This request is not yet committed for inclusion in an Update release. This bug can be reproduced through following scenario: +---------------+ +------------------------------+ | Netserver | LAN | | |---------------|---------|Netperf(*2000+ tasks) | | VM (512M) | | | +---------------+ +------------------------------+ Run thousands of netperf clients in background to stress the netserver. kernel-2.6.32-113.el6.x86_64 Patch(es) available on kernel-2.6.32-117.el6 Reproduced on kernel-2.6.32-116.el6.x86_64, and verified on kernel-2.6.32-117.el6.x86_64. PASS. Steps: 1) boot guest with 512M mem and virtio net. 2) run netserver inside guest. 3) on host, launch 2000 netperf clients in background to stress netserver. 4) ping guest (network lost, need to restart guest network to restore) Update guest kernel to 2.6.32-117 and test again, no network lost. CLI: /usr/libexec/qemu-kvm -S -M rhel6.1.0 -enable-kvm -m 512 -smp 2,sockets=2,cores=1,threads=1 -name RHEL6.1-virtio_net_test -uuid 362f0255-b6e4-2a75-9506-af9c2e5ceb5d -nodefconfig -nodefaults -chardev socket,id=monitor,path=/var/lib/libvirt/qemu/RHEL6.1-virtio_net_test.monitor,server,nowait -mon chardev=monitor,mode=control -rtc base=utc -boot c -drive file=/home/khong/RHEL6.1-virtio_net_test.img,if=none,id=drive-virtio-disk0,format=raw,cache=none -device virtio-blk-pci,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0 -netdev tap,fd=20,id=hostnet0,vhost=on,vhostfd=22 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:1f:f2:62,bus=pci.0,addr=0x3 -chardev pty,id=serial0 -device isa-serial,chardev=serial0 -usb -device usb-tablet,id=input0 -vnc 127.0.0.1:0 -vga cirrus -device AC97,id=sound0,bus=pci.0,addr=0x4 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6 script to run netperf clients: #! /bin/sh ip=$guest_ip i=0 while [ $i -lt 2000 ] do netperf -H $ip -l 300 & i=`expr $i + 1` echo launch Client-No.$i done
Technical note added. If any revisions are required, please edit the "Technical Notes" field
accordingly. All revisions will be proofread by the Engineering Content Services team.
New Contents:
Intensive usage of resources on a guest lead to a failure of networking on that guest: packets could no longer be received. The failure occurred when a DMA (Direct Memory Access) ring was consumed before NAPI (New API; an interface for networking devices which makes use of interrupt mitigation techniques) was enabled which resulted in a failure to receive the next interrupt request. The regular interrupt handler was not affected in this situation (because it can process packets in-place), however, the OOM (Out Of Memory) handler did not detect the aforementioned situation and caused networking to fail. With this update, NAPI is subsequently scheduled for each napi_enable operation; thus, networking no longer fails under the aforementioned circumstances.
An advisory has been issued which should help the problem described in this bug report. This report is therefore being closed with a resolution of ERRATA. For more information on therefore solution and/or where to find the updated files, please follow the link below. You may reopen this bug report if the solution does not work for you. http://rhn.redhat.com/errata/RHSA-2011-0542.html |