Bug 1698396

Created attachment 1554156 [details]
xml for vm

Description of problem:
Boot vm as vhost user client mode, with iommu='on' ats='on', dpdk's testpmd can not receive packets.

Version-Release number of selected component (if applicable):
3.10.0-1037.el7.x86_64
openvswitch2.11-2.11.0-5.el7fdp.x86_64
dpdk-18.11-4.el7_6.x86_64
qemu-kvm-rhev-2.12.0-25.el7.x86_64
libvirt-4.5.0-10.el7.x86_64

How reproducible:
100%

Steps to Reproduce:
1. Boot ovs, refer to[1]

2. Boot VM with vhostuser client mode, set "iommu='on' ats='on'", full xml will be attached.

   <interface type='vhostuser'>
     <mac address='18:66:da:5f:dd:02'/>
     <source type='unix' path='/var/run/openvswitch/vhost-user0' mode='client'/>
     <model type='virtio'/>
     <driver name='vhost' rx_queue_size='1024' iommu='on' ats='on'/>
     <address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
   </interface>
  
3. In guest, load vfio and start testpmd
# modprobe vfio
# modprobe vfio-pci 

# /usr/bin/testpmd \
-l 1,2,3 -n 4 -d \
/usr/lib64/librte_pmd_virtio.so \
-w 0000:00:06.0 -w 0000:00:07.0 \
-- --nb-cores=2 -i \
--disable-rss \
--rxd=512 --txd=512 \
--rxq=1 --txq=1


4. Start MoonGen in another host, testpmd can not receive packets.

Actual results:
dpdk's testpmd can not receive packets.

Expected results:
dpdk's testpmd can receive packets.

Additional info:
1. Tested without iommu='on' ats='on', testpmd works well.
2. Tested with qemu as vhost user server mode, testpmd works well.
3. Tested with openvswitch2.11-2.11.0-1, testpmd can't receive packets neither.
4. There is a similar bug in https://bugzilla.redhat.com/show_bug.cgi?id=1532956.

References:
[1]
# ovs-vsctl show
7259d18d-97a8-4d67-b572-3b399f9982f5
    Bridge "ovsbr1"
        Port "ovsbr1"
            Interface "ovsbr1"
                type: internal
        Port "vhost-user2"
            Interface "vhost-user2"
                type: dpdkvhostuser
        Port "dpdk2"
            Interface "dpdk2"
                type: dpdk
                options: {dpdk-devargs="0000:5f:00.0", n_rxq="1", n_txq="1"}
    Bridge "ovsbr0"
        Port "dpdk1"
            Interface "dpdk1"
                type: dpdk
                options: {dpdk-devargs="0000:5e:00.1", n_rxq="1", n_txq="1"}
        Port "dpdk0"
            Interface "dpdk0"
                type: dpdk
                options: {dpdk-devargs="0000:5e:00.0", n_rxq="1", n_txq="1"}
        Port "vhost-user1"
            Interface "vhost-user1"
                type: dpdkvhostuser
        Port "ovsbr0"
            Interface "ovsbr0"
                type: internal
        Port "vhost-user0"
            Interface "vhost-user0"
                type: dpdkvhostuser