Bug 748328

Summary: Guest boot up with vf cause out of memory when guest memory is close to free memory in host
Product: Red Hat Enterprise Linux 6 Reporter: Joy Pu <ypu>
Component: qemu-kvmAssignee: Virtualization Maintenance <virt-maint>
Status: CLOSED NOTABUG QA Contact: Virtualization Bugs <virt-bugs>
Severity: medium Docs Contact:
Priority: high    
Version: 6.2CC: acathrow, juzhang, mkenneth, tburke, virt-maint
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2011-12-08 08:31:40 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On:    
Bug Blocks: 580954    

Description Joy Pu 2011-10-24 06:51:20 UTC
Description:
Boot up a RHEL 6.1 geust with a vf and with a memory close to or more than free memory in host, it will caused an oom and qemu process will be killed with oom. This will happen even the memory of guest is close to the free memory in host. Boot up the same guest with the same cmd line just remove the "-device pci-assign,host=03:10.0,id=test,iommu=1" option will not cause oom.

Version-Release number of selected component (if applicable):
kernel: 
2.6.32-211.el6.x86_64
kvm&qemu: 
qemu-img-0.12.1.2-2.200.el6.x86_64
gpxe-roms-qemu-0.9.7-6.9.el6.noarch
qemu-kvm-debuginfo-0.12.1.2-2.200.el6.x86_64
qemu-kvm-0.12.1.2-2.200.el6.x86_64
qemu-kvm-tools-0.12.1.2-2.200.el6.x86_64


How reproducible:
Always

Steps to Reproduce:
1. Reload the kvm and igb module with options support for device assign and SRIOV
$ modprobe -r kvm_intel;modprobe -r kvm; modprrobe -r igb
$ modprobe kvm allow_unsafe_assigned_interrupts=1
$ modprobe kvm_intel
$ modprobe igb max_vfs=7
2. Unbind device from host kernel driver. 
$ lspci |grep 82576
$ lspci -n -s 03:10.0
$ echo "8086 10ca" > /sys/bus/pci/drivers/pci-stub/new_id
$ echo 0000:03:10.0 > /sys/bus/pci/devices/0000\:03\:10.0/driver/unbind 
$ echo 0000:03:10.0 > /sys/bus/pci/drivers/pci-stub/bind 
3. Boot up guest with vf and memory close to free memory in host
$qemu -device pci-assign,host=03:10.0,id=test,iommu=1 -mem $free_memory or
$qemu -pcidevice host=03:10.0 -mem $free_memory

Actual results:
Host get oom and it kill qemu process sometimes make host kernel panic
Expected results:
Can boot up a guest normally

Additional info:
1. cmdline:
qemu-kvm -name 'vm1' -chardev socket,id=qmp_monitor_id_qmpmonitor1,path=/tmp/monitor-qmpmonitor1-20111024-135253-EMnq,server,nowait -mon chardev=qmp_monitor_id_qmpmonitor1,mode=control -chardev socket,id=serial_id_20111024-135253-EMnq,path=/tmp/serial-20111024-135253-EMnq,server,nowait -device isa-serial,chardev=serial_id_20111024-135253-EMnq -drive file='/root/autotest-devel/client/tests/kvm/images/RHEL-Server-6.1-32-virtio.qcow2',index=0,if=none,id=drive-virtio-disk1,media=disk,cache=none,format=qcow2,aio=native -device virtio-blk-pci,bus=pci.0,addr=0x4,drive=drive-virtio-disk1,id=virtio-disk1 -net none -m 11957 -smp 2,cores=1,threads=1,sockets=2 -cpu cpu64-rhel6,+sse2,+x2apic -spice port=8000,disable-ticketing -vga qxl   -rtc base=utc,clock=host,driftfix=slew -boot order=cdn,once=c,menu=off    -no-kvm-pit-reinjection  -M rhel6.2.0 -usb -device usb-tablet -enable-kvm -device pci-assign,host=03:10.0,id=test,iommu=1

Comment 2 Joy Pu 2011-10-24 06:54:52 UTC
Memory info in host:
free -m
             total       used       free     shared    buffers     cached
Mem:         11957        865      11091          0          3         18
-/+ buffers/cache:        843      11113
Swap:         8191        620       7571

Comment 3 Dor Laor 2011-12-08 08:31:40 UTC
When using pci pass through we automatically pin all of the guest pages, thus we require more free memory on the host than the guest size.