***NOTE***

Opening this bug because it was not able to be verified in 19.D due to the firmware not being available.  Opening a new bug to target 19.E for verification and support of this fix.

*******************************************

This issue was discovered while testing MLNX_OFED version for Verizon. opening the BZ since same fix should be taken to RHEL-7.

Description of problem:
PF to VF VGT Plus TX not offloaded

Flow:
- Set PRIO_TAG_REQUIRED_EN=1 in the FW ( note that a special FW version is required, as of today it's not available on mellanox website yet).
- Bring up ASAP tunnel based system (VXLAN).
- Create VGT interface over one VF on machine A.
- Set IP address to VF over machine B with the same network

- Add the following rules to machine A:
ovs-ofctl add-flow ovs-sriov1 'in_port=vxlan0, actions=mod_vlan_vid:7, output:enp129s0f0_1'
ovs-ofctl add-flow ovs-sriov1 'in_port=enp129s0f0_1, actions=output:vxlan0'

- Add the following rules to machine B:
ovs-ofctl add-flow ovs-sriov1 'in_port=vxlan0, actions=strip_vlan, output:enp129s0f0_1'
ovs-ofctl add-flow ovs-sriov1 'in_port=enp129s0f0_1, actions=output:vxlan0'

- Strat bidir traffic.

Result:
RX rules are offloaded as expected.
TX rules is NOT offloaded.

ovs-dpctl dump-flows -m

2019-05-15T07:25:27Z|00001|dpif_netlink|INFO|The kernel module does not support meters.
ufid:639c0879-4047-4557-8703-0765039e4457, skb_priority(0/0),tunnel(tun_id=0x5,src=101.196.16.1,dst=101.196.15.1,ttl=0/0,tp_dst=4789,flags(+key)),skb_mark(0/0),in_port(vxlan_sys_4789),packet_type(ns=0/0,id=0/0),eth(src=00:00:00:00:00:00/00:00:00:00:00:00,dst=00:00:00:00:00:00/00:00:00:00:00:00),eth_type(0x8100),vlan(vid=7,pcp=0),encap(eth_type(0x0800),ipv4(src=0.0.0.0/0.0.0.0,dst=0.0.0.0/0.0.0.0,proto=0/0,tos=0/0,ttl=0/0,frag=no)), packets:13, bytes:1326, used:0.240s, offloaded:yes, dp:tc, actions:pop_vlan,enp129s0f0_1
ufid:3e3798b7-0b5e-4698-aa20-ac2a829031bc, skb_priority(0/0),skb_mark(0/0),in_port(enp129s0f0_1),packet_type(ns=0/0,id=0/0),eth(src=00:00:00:00:00:00/00:00:00:00:00:00,dst=00:00:00:00:00:00/00:00:00:00:00:00),eth_type(0x0800),ipv4(src=0.0.0.0/0.0.0.0,dst=0.0.0.0/0.0.0.0,proto=0/0,tos=0/0x3,ttl=0/0,frag=no), packets:0, bytes:0, used:13.300s, offloaded:yes, dp:tc, actions:set(tunnel(tun_id=0x5,src=101.196.15.1,dst=101.196.16.1,tp_dst=4789,flags(key))),vxlan_sys_4789
tcpdump -envi enp129s0f0_1

10:25:38.358111 e4:11:0f:81:00:03 > e4:11:10:81:00:03, ethertype 802.1Q (0x8100), length 102: vlan 0, p 0, ethertype IPv4, (tos 0x0, ttl 64, id 58675, offset 0, flags [DF], proto ICMP (1), length 84)
    20.196.15.1 > 20.196.16.1: ICMP echo request, id 17779, seq 25, length 64
10:25:39.358101 e4:11:0f:81:00:03 > e4:11:10:81:00:03, ethertype 802.1Q (0x8100), length 102: vlan 0, p 0, ethertype IPv4, (tos 0x0, ttl 64, id 59106, offset 0, flags [DF], proto ICMP (1), length 84)
    20.196.15.1 > 20.196.16.1: ICMP echo request, id 17779, seq 26, length 64
10:25:40.358123 e4:11:0f:81:00:03 > e4:11:10:81:00:03, ethertype 802.1Q (0x8100), length 102: vlan 0, p 0, ethertype IPv4, (tos 0x0, ttl 64, id 59500, offset 0, flags [DF], proto ICMP (1), length 84)
    20.196.15.1 > 20.196.16.1: ICMP echo request, id 17779, seq 27, length 64
10:25:41.358100 e4:11:0f:81:00:03 > e4:11:10:81:00:03, ethertype 802.1Q (0x8100), length 102: vlan 0, p 0, ethertype IPv4, (tos 0x0, ttl 64, id 60090, offset 0, flags [DF], proto ICMP (1), length 84)
    20.196.15.1 > 20.196.16.1: ICMP echo request, id 17779, seq 28, length 64