Hide Forgot
Description of problem: For a device using MSI-X interrupts assigned to a guest, if we poke the /proc/irq/$IRQ/smp_affinity the guest doesn't see any change in interrupt delivery. The reason for this is that the MSI-X table is only snapshot'd when MSI-X is first enabled. Subsequent changes to the destination (embedded in the vector address) don't update the programming in KVM where the interrupt gets delivered to the guest. Fix this and check for similar issues with MSI/INTx. Version-Release number of selected component (if applicable): rhel6.2 How reproducible: always Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Additional info: A crude workaround for this is to use setpci to toggle the MSI-X enable bit in PCI config space of the guest. This will re-snapshot the MSI-X vector table.
This request was not resolved in time for the current release. Red Hat invites you to ask your support representative to propose this request, if still desired, for consideration in the next release of Red Hat Enterprise Linux.
This request was erroneously removed from consideration in Red Hat Enterprise Linux 6.4, which is currently under development. This request will be evaluated for inclusion in Red Hat Enterprise Linux 6.4.
*** Bug 833264 has been marked as a duplicate of this bug. ***
Created attachment 600862 [details] proc-interrupt and sar data
Created attachment 600863 [details] sosreport from the host
Created attachment 600864 [details] sosreport from the guest
Created attachment 600865 [details] /proc/interrupts, smp_affinity, and sar data
Reproduced this issue on qemu-kvm-0.12.1.2-2.295.el6: Steps: 1. Find a host with a MSI-X nic card could be assign to guest, I select a host with 82576 host. 2. Generate VFs and unbind. #modprobe -r igb; modprobe igb max_vfs=7 #echo "8086 10ca" > /sys/bus/pci/drivers/pci-stub/new_id #echo 0000:01:10.0 > /sys/bus/pci/devices/0000\:01\:10.0/driver/unbind #echo 0000:01:10.0 > /sys/bus/pci/drivers/pci-stub/bind 3. Boot the guest with the VF attached. /usr/libexec/qemu-kvm ...... -device pci-assign,host=01:10.0,id=hostnet1 4. Check the nic device interrupt and smp_affinity inside guest. # cat /proc/interrupts | grep eth2562 CPU0 CPU1 28: 97 0 PCI-MSI-edge eth2562-tx-0 29: 345 0 PCI-MSI-edge eth2562-rx-0 30: 12 0 PCI-MSI-edge eth2562 # cat /proc/irq/29/smp_affinity 1 5. Now we could see the smp_affinity displays 1 and transfer data from guest to external all the interrupts from eth2562 will increase in CPU0. 6. Set smp_affinity to CPU1 # service irqbalance stop # echo 2 > /proc/irq/29/smp_affinity # cat /proc/irq/29/smp_affinity 2 7. Continue transfer data from guest to external host. # cat /proc/interrupts | grep eth2562 28: 109 0 PCI-MSI-edge eth2562-tx-0 29: 365 0 PCI-MSI-edge eth2562-rx-0 30: 12 0 PCI-MSI-edge eth2562 Result: Interrupts still increases in cpu0, instead of cpu1. ======================== Verified pass on qemu-kvm-0.12.1.2-2.334.el6.x86_64. Steps: Same as above. But after step 7: [root@localhost ~]# cat /proc/interrupts | grep eth2562 CPU0 CPU1 28: 344 41 PCI-MSI-edge eth2562-tx-0 29: 617 306 PCI-MSI-edge eth2562-rx-0 30: 12 0 PCI-MSI-edge eth2562 Result: Interrupts will increase in cpu1. Interrupts will stop increasing in cpu0. So, this bug is fixed.
*** Bug 879453 has been marked as a duplicate of this bug. ***
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. http://rhn.redhat.com/errata/RHBA-2013-0527.html