Bug 1377210

Summary: dpdk's testpmd can not receive data with assigned VFs
Product: Red Hat Enterprise Linux 7 Reporter: Pei Zhang <pezhang>
Component: dpdkAssignee: Kevin Traynor <ktraynor>
Status: CLOSED NOTABUG QA Contact: Jean-Tsung Hsiao <jhsiao>
Severity: medium Docs Contact:
Priority: medium    
Version: 7.3CC: ailan, alex.williamson, aloughla, atragler, chayang, fleitner, jean-mickael.guerin, juzhang, knoel, maxime.coquelin, michen, mleitner, sukulkar, thalabathy.venkatesan, wchoi, xfu, xiywang
Target Milestone: rcKeywords: Extras
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-06-12 10:14:37 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Pei Zhang 2016-09-19 08:12:23 UTC
Description of problem:
Assign VFs to guest, dpdk's testpmd can not receive data from packets generator.

As VFs can not be bind to uio_pci_generic, so test with upstream dpdk which support vfio.


Version-Release number of selected component (if applicable):
Host:
3.10.0-507.rt56.412.el7.x86_64
qemu-kvm-rhev-2.6.0-22.el7.x86_64


Guest:
3.10.0-507.rt56.412.el7.x86_64

How reproducible:
100%

Steps to Reproduce:
1. Create 2 VFs each PF
# echo 2 > /sys/bus/pci/devices/0000\:04\:00.0/sriov_numvfs
# echo 2 > /sys/bus/pci/devices/0000\:04\:00.1/sriov_numvfs

2. Bind 2 VFs to vfio
# ls /sys/bus/pci/drivers/vfio-pci/
0000:04:10.2  0000:04:10.3  bind  module  new_id  remove_id  uevent  unbind


3. Boot guest with 2 assigned VFs from these 2 PFs
 <hostdev mode='subsystem' type='pci' managed='yes'>
       <driver name='vfio'/>
       <source>
          <address domain='0x0000' bus='0x04' slot='0x10' function='0x2'/>
       </source>
          <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
     </hostdev> 

     <hostdev mode='subsystem' type='pci' managed='yes'>
       <driver name='vfio'/>
       <source>
         <address domain='0x0000' bus='0x04' slot='0x10' function='0x3'/>
       </source>
       <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
     </hostdev>

4. In guest, bind them to vfio
(1)load vfio
# modprobe -r vfio
# modprobe -r vfio_iommu_type1
# modprobe vfio enable_unsafe_noiommu_mode=Y
# modprobe vfio-pci
# cat /sys/module/vfio/parameters/enable_unsafe_noiommu_mode 
Y

(2) bind to vfio
# lspci -n -s 0000:00:02.0
00:02.0 0200: 8086:1515 (rev 01)

# echo 0000:00:02.0 > /sys/bus/pci/devices/0000\:00\:02.0/driver/unbind 
# echo 0000:00:06.0 > /sys/bus/pci/devices/0000\:00\:06.0/driver/unbind 
# echo "8086 1583" > /sys/bus/pci/drivers/vfio-pci/new_id 
# echo "8086 1583" > /sys/bus/pci/drivers/vfio-pci/remove_id 
# ls /sys/bus/pci/drivers/vfio-pci/
0000:00:02.0  0000:00:06.0  bind  module  new_id  remove_id  uevent  unbind

5. In guest, start testpmd
# cat testpmd-ixgbe.sh 
queues=1
cores=2
/root/dpdk-16.07/x86_64-native-linuxapp-gcc/build/app/test-pmd/testpmd -l 0,1,2 -n 1 -d  /root/dpdk-16.07/x86_64-native-linuxapp-gcc/lib/librte_pmd_ixgbe.so \
-w 00:02.0 -w 00:06.0 \
-- \
--disable-hw-vlan -i \
--nb-cores=${cores} \
--disable-rss \
--rxq=${queues} --txq=${queues} \
--auto-start \
--rxd=256 --txd=256 \

6. In another host, start MoonGen as packets generator
# ./build/MoonGen examples/l2-load-latency.lua 0 1 150

7. Check testpmd, no packets flow.
testpmd> show port stats all 

  ######################## NIC statistics for port 0  ########################
  RX-packets: 0          RX-missed: 0          RX-bytes:  0
  RX-errors: 0
  RX-nombuf:  0         
  TX-packets: 0          TX-errors: 0          TX-bytes:  0

  Throughput (since last show)
  Rx-pps:            0
  Tx-pps:            0
  ############################################################################

  ######################## NIC statistics for port 1  ########################
  RX-packets: 0          RX-missed: 0          RX-bytes:  0
  RX-errors: 0
  RX-nombuf:  0         
  TX-packets: 0          TX-errors: 0          TX-bytes:  0

  Throughput (since last show)
  Rx-pps:            0
  Tx-pps:            0
  ############################################################################


Actual results:
testpmd can not receive data.

Expected results:
testpmd should receive data.

Additional info:
1. With PFs, testpmd can receive data.

2. In guest, kernel driver ixgbevf and i40evf can work, #ping works.  

3. X540-AT2/ixgbe and XL710/i40e hit same issue.

4. port info from testpmd
(1) ixgbe
testpmd> show  port info 0

********************* Infos for port 0  *********************
MAC address: 86:B3:04:E7:AB:17
Connect to socket: 0
memory allocation on the socket: 0
Link status: up
Link speed: 10000 Mbps
Link duplex: full-duplex
Promiscuous mode: disabled
Allmulticast mode: disabled
Maximum number of MAC addresses: 128
Maximum number of MAC addresses of hash filtering: 4096
VLAN offload: 
  strip off 
  filter off 
  qinq(extend) off 
No flow type is supported.
Max possible RX queues: 4
Max possible number of RXDs per queue: 4096
Min possible number of RXDs per queue: 32
RXDs number alignment: 8
Max possible TX queues: 4
Max possible number of TXDs per queue: 4096
Min possible number of TXDs per queue: 32
TXDs number alignment: 8

(2) i40e
testpmd> show port info 0

********************* Infos for port 0  *********************
MAC address: 9A:D8:E0:41:30:72
Connect to socket: 0
memory allocation on the socket: 0
Link status: up
Link speed: 40000 Mbps
Link duplex: full-duplex
Promiscuous mode: enabled
Allmulticast mode: disabled
Maximum number of MAC addresses: 64
Maximum number of MAC addresses of hash filtering: 0
VLAN offload: 
  strip off 
  filter off 
  qinq(extend) off 
Hash key size in bytes: 52
Redirection table size: 64
Supported flow types:
  ipv4-frag
  ipv4-tcp
  ipv4-udp
  ipv4-sctp
  ipv4-other
  ipv6-frag
  ipv6-tcp
  ipv6-udp
  ipv6-sctp
  ipv6-other
  l2_payload
Max possible RX queues: 4
Max possible number of RXDs per queue: 4096
Min possible number of RXDs per queue: 64
RXDs number alignment: 32
Max possible TX queues: 4
Max possible number of TXDs per queue: 4096
Min possible number of TXDs per queue: 64
TXDs number alignment: 32

5. VFs can not bind to uio_pci_generic(test with dpdk-2.2.0-3.el7.x86_64)
# modprobe uio
# modprobe uio_pci_generic
# echo "8086 1515" > /sys/bus/pci/drivers/uio_pci_generic/new_id 
# echo "8086 1515" > /sys/bus/pci/drivers/uio_pci_generic/remove_id 

no devices show.
# ls /sys/bus/pci/drivers/uio_pci_generic/
bind  module  new_id  remove_id  uevent  unbind

Comment 4 Alex Williamson 2016-09-21 14:58:49 UTC
(In reply to Pei Zhang from comment #0)
> 
> (2) bind to vfio
> # lspci -n -s 0000:00:02.0
> 00:02.0 0200: 8086:1515 (rev 01)
                     ^^^^
This is an X540 VF
 
> # echo 0000:00:02.0 > /sys/bus/pci/devices/0000\:00\:02.0/driver/unbind 
> # echo 0000:00:06.0 > /sys/bus/pci/devices/0000\:00\:06.0/driver/unbind 
> # echo "8086 1583" > /sys/bus/pci/drivers/vfio-pci/new_id 
> # echo "8086 1583" > /sys/bus/pci/drivers/vfio-pci/remove_id
               ^^^^
This is an XL710 PF ???
 
> # ls /sys/bus/pci/drivers/vfio-pci/
> 0000:00:02.0  0000:00:06.0  bind  module  new_id  remove_id  uevent  unbind
> 
> 5. In guest, start testpmd
> # cat testpmd-ixgbe.sh 
> queues=1
> cores=2
> /root/dpdk-16.07/x86_64-native-linuxapp-gcc/build/app/test-pmd/testpmd -l
> 0,1,2 -n 1 -d 
> /root/dpdk-16.07/x86_64-native-linuxapp-gcc/lib/librte_pmd_ixgbe.so \


Does librte_pmd_ixgbe.so support ixgbevf?  I seem to find that this -d option is unnecessary.

> -w 00:02.0 -w 00:06.0 \
> -- \
> --disable-hw-vlan -i \
> --nb-cores=${cores} \
> --disable-rss \
> --rxq=${queues} --txq=${queues} \
> --auto-start \
> --rxd=256 --txd=256 \
> 
> 6. In another host, start MoonGen as packets generator
> # ./build/MoonGen examples/l2-load-latency.lua 0 1 150
> 
> 7. Check testpmd, no packets flow.
> testpmd> show port stats all 
> 
>   ######################## NIC statistics for port 0 
> ########################
>   RX-packets: 0          RX-missed: 0          RX-bytes:  0
>   RX-errors: 0
>   RX-nombuf:  0         
>   TX-packets: 0          TX-errors: 0          TX-bytes:  0
> 
>   Throughput (since last show)
>   Rx-pps:            0
>   Tx-pps:            0
>  
> ############################################################################
> 
>   ######################## NIC statistics for port 1 
> ########################
>   RX-packets: 0          RX-missed: 0          RX-bytes:  0
>   RX-errors: 0
>   RX-nombuf:  0         
>   TX-packets: 0          TX-errors: 0          TX-bytes:  0
> 
>   Throughput (since last show)
>   Rx-pps:            0
>   Tx-pps:            0
>  
> ############################################################################
> 
> 
> Actual results:
> testpmd can not receive data.
> 
> Expected results:
> testpmd should receive data.
> 
> Additional info:
> 1. With PFs, testpmd can receive data.
> 
> 2. In guest, kernel driver ixgbevf and i40evf can work, #ping works.  

Perhaps dpdk drivers are simply broken.  Who supports them?

Comment 5 Maxime Coquelin 2016-09-22 09:06:01 UTC
(In reply to Alex Williamson from comment #4)
> (In reply to Pei Zhang from comment #0)
> > 
> > (2) bind to vfio
> > # lspci -n -s 0000:00:02.0
> > 00:02.0 0200: 8086:1515 (rev 01)
>                      ^^^^
> This is an X540 VF
>  
> > # echo 0000:00:02.0 > /sys/bus/pci/devices/0000\:00\:02.0/driver/unbind 
> > # echo 0000:00:06.0 > /sys/bus/pci/devices/0000\:00\:06.0/driver/unbind 
> > # echo "8086 1583" > /sys/bus/pci/drivers/vfio-pci/new_id 
> > # echo "8086 1583" > /sys/bus/pci/drivers/vfio-pci/remove_id
>                ^^^^
> This is an XL710 PF ???
Why don't you just unbind it from ixgbevf, and re-bind it in vfio-pci?
The same should apply to the guest.

> > # ls /sys/bus/pci/drivers/vfio-pci/
> > 0000:00:02.0  0000:00:06.0  bind  module  new_id  remove_id  uevent  unbind
> > 
> > 5. In guest, start testpmd
> > # cat testpmd-ixgbe.sh 
> > queues=1
> > cores=2
> > /root/dpdk-16.07/x86_64-native-linuxapp-gcc/build/app/test-pmd/testpmd -l
> > 0,1,2 -n 1 -d 
> > /root/dpdk-16.07/x86_64-native-linuxapp-gcc/lib/librte_pmd_ixgbe.so \
> 
> 
> Does librte_pmd_ixgbe.so support ixgbevf?  I seem to find that this -d
> option is unnecessary.

Yes, the librte_pmd_ixgbe.so also support ixgbevf.

I'll have a try with my HW, even if not exactly the same 10G reference.

Comment 6 Pei Zhang 2016-09-22 10:48:58 UTC
(In reply to Alex Williamson from comment #4)
> (In reply to Pei Zhang from comment #0)
> > 
> > (2) bind to vfio
> > # lspci -n -s 0000:00:02.0
> > 00:02.0 0200: 8086:1515 (rev 01)
>                      ^^^^
> This is an X540 VF
>  
> > # echo 0000:00:02.0 > /sys/bus/pci/devices/0000\:00\:02.0/driver/unbind 
> > # echo 0000:00:06.0 > /sys/bus/pci/devices/0000\:00\:06.0/driver/unbind 
> > # echo "8086 1583" > /sys/bus/pci/drivers/vfio-pci/new_id 
> > # echo "8086 1583" > /sys/bus/pci/drivers/vfio-pci/remove_id
>                ^^^^
> This is an XL710 PF ???
>  
Sorry for confusing, it's typo. should be:
# echo "8086 1515" > /sys/bus/pci/drivers/vfio-pci/new_id 
# echo "8086 1515" > /sys/bus/pci/drivers/vfio-pci/remove_id

Best Regards,
Pei

Comment 7 thalabathy.venkatesan 2017-04-27 04:56:08 UTC
Its working for me, and able to see the packets are landing on my guest through VF.
Here VF will receive tagged packets and your test-pmd guest not enabled vlan strip.
Hence packets would have dropped at VF level

Could you please try below following command after binding NIC to PMD driver / dpdk

set fwd mac_retry
vlan set strip on 0
vlan set strip on 1
start

<presuming you have only two ports connected to test-pmd guest. 


Regards/Thalabathy

Comment 8 Pei Zhang 2017-06-12 10:14:37 UTC
Thanks Thalabathy, it works! 


Summary key points:
1. When sending packets from MoonGen, the des mac of the packet should be set as MAC of VF. In other words, VF matters MAC address. Otherwise L2 network packets can not received by any VF.


2. Like Thalabathy mentioned in Comment 7
testpmd> set fwd mac retry
Set mac packet forwarding mode with retry
testpmd> vlan set strip on 0
testpmd> vlan set strip on 1


Results: Both VF assigned in guest can receive packets.
testpmd> start 
mac packet forwarding with retry - ports=2 - cores=2 - streams=2 - NUMA support enabled, MP over anonymous pages disabled
TX retry num: 64, delay between TX retries: 1us
Logical Core 2 (socket 0) forwards packets on 1 streams:
  RX P=0/Q=0 (socket 0) -> TX P=1/Q=0 (socket 0) peer=02:00:00:00:00:01
Logical Core 3 (socket 0) forwards packets on 1 streams:
  RX P=1/Q=0 (socket 0) -> TX P=0/Q=0 (socket 0) peer=02:00:00:00:00:00

  mac packet forwarding with retry - CRC stripping enabled - packets/burst=32
  nb forwarding cores=2 - nb forwarding ports=2
  RX queues=1 - RX desc=128 - RX free threshold=32
  RX threshold registers: pthresh=8 hthresh=8 wthresh=0
  TX queues=1 - TX desc=512 - TX free threshold=32
  TX threshold registers: pthresh=32 hthresh=0 wthresh=0
  TX RS bit threshold=32 - TXQ flags=0xf01
testpmd> show port stats all 

  ######################## NIC statistics for port 0  ########################
  RX-packets: 5379523    RX-missed: 0          RX-bytes:  322771380
  RX-errors: 0
  RX-nombuf:  0         
  TX-packets: 5379271    TX-errors: 0          TX-bytes:  322756260

  Throughput (since last show)
  Rx-pps:       995257
  Tx-pps:       995257
  ############################################################################

  ######################## NIC statistics for port 1  ########################
  RX-packets: 5379518    RX-missed: 0          RX-bytes:  322771140
  RX-errors: 0
  RX-nombuf:  0         
  TX-packets: 5379321    TX-errors: 0          TX-bytes:  322759320

  Throughput (since last show)
  Rx-pps:       995256
  Tx-pps:       995257
  ############################################################################


So close this bug as 'NOTABUG'. 



Thanks,
Pei