Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1499645

Summary: [ovs-dpdk conntrack]tftp can't establish connection in the DNAT topo
Product: Red Hat Enterprise Linux 7 Reporter: Jiying Qiu <jiqiu>
Component: openvswitchAssignee: David Marchand <dmarchan>
Status: CLOSED DUPLICATE QA Contact: Jiying Qiu <jiqiu>
Severity: medium Docs Contact:
Priority: medium    
Version: 7.4CC: atragler, dmarchan, fleitner, pvauter, qding
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-12-18 12:51:48 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Jiying Qiu 2017-10-09 07:46:28 UTC
Description of problem:
tftp can't establish connection in the DNAT topo.

Version-Release number of selected component (if applicable):
ovs_version: "2.8.0"
[root@dell-per730-51 conntrack_dpdk]# uname -a
Linux dell-per730-51.rhts.eng.pek2.redhat.com 3.10.0-693.el7.x86_64 #1 SMP Thu Jul 6 19:56:57 EDT 2017 x86_64 x86_64 x86_64 GNU/Linux


How reproducible:
everytime

topo:

the ip of vm is 10.167.100.1, ip of remote host's nic is 172.16.100.2. Remote try to use tftp to connect 172.16.100.254, which should be change to 10.167.100.1 by ovs.

10.167.100.1           172.16.100.2 
+--------+            +------+
| vm--ovs+------------+remote|       
+--------+            +------+ 

Steps to Reproduce:
1.create ovs in host,add dpdk0 and vhost to ovs. create guest with vhost0
2.configure flows on ovs, using dnat
3.tftp from remote to 172.167.100.254


Actual results:
the data connection cant't establish

Expected results:
the data connection can establish

Additional info:
log:
[root@dell-per730-51 conntrack_dpdk]# dpdk-devbind -s

Network devices using DPDK-compatible driver
============================================
0000:82:00.0 'Ethernet Controller X710 for 10GbE SFP+' drv=vfio-pci unused=i40e

Network devices using kernel driver
===================================
0000:01:00.0 'NetXtreme BCM5720 Gigabit Ethernet PCIe' if=em1 drv=tg3 unused=vfio-pci 
0000:01:00.1 'NetXtreme BCM5720 Gigabit Ethernet PCIe' if=em2 drv=tg3 unused=vfio-pci 
0000:02:00.0 'NetXtreme BCM5720 Gigabit Ethernet PCIe' if=em3 drv=tg3 unused=vfio-pci 
0000:02:00.1 'NetXtreme BCM5720 Gigabit Ethernet PCIe' if=em4 drv=tg3 unused=vfio-pci 
0000:82:00.1 'Ethernet Controller X710 for 10GbE SFP+' if=p4p2 drv=i40e unused=vfio-pci 

Other network devices
=====================
<none>

Crypto devices using DPDK-compatible driver
===========================================
<none>

Crypto devices using kernel driver
==================================
<none>

Other crypto devices
====================
<none>
[root@dell-per730-51 conntrack_dpdk]# ovs-vsctl show 
678ee373-d09a-4d1a-ab73-11b403ca1886
    Bridge "ovsbr0"
        Port "vhost1"
            Interface "vhost1"
                type: dpdkvhostuser
                options: {n_rxq="1"}
        Port "vhost0"
            Interface "vhost0"
                type: dpdkvhostuser
                options: {n_rxq="1"}
        Port "vhost2"
            Interface "vhost2"
                type: dpdkvhostuser
                options: {n_rxq="1"}
        Port "ovsbr0"
            Interface "ovsbr0"
                type: internal
        Port "dpdk0"
            Interface "dpdk0"
                type: dpdk
                options: {dpdk-devargs="0000:82:00.0", n_rxq="1"}
    ovs_version: "2.8.0"
[root@dell-per730-51 conntrack_dpdk]#virsh console g1
localhost login: root
Password: 
Last login: Sun Oct  8 23:10:44 on ttyS0
[root@localhost ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:00:00:01:03:01 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::200:ff:fe01:301/64 scope link 
       valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:00:00:01:04:01 brd ff:ff:ff:ff:ff:ff
    inet 10.167.100.1/24 scope global eth1
       valid_lft forever preferred_lft forever
    inet6 2001:db8:ffff:100::1/64 scope global 
       valid_lft forever preferred_lft forever

[root@dell-per730-51 conntrack_dpdk]# ovs-ofctl dump-flows ovsbr0
 cookie=0x0, duration=2563.727s, table=0, n_packets=192, n_bytes=48244, priority=100,ct_state=-trk,ip actions=ct(table=1)
 cookie=0x0, duration=2563.698s, table=0, n_packets=44, n_bytes=2298, priority=100,arp,arp_op=1 actions=move:NXM_OF_ARP_TPA[]->NXM_NX_REG2[],resubmit(,8),resubmit(,10)
 cookie=0x0, duration=2563.694s, table=0, n_packets=19, n_bytes=1140, priority=10,arp actions=NORMAL
 cookie=0x0, duration=2562.574s, table=0, n_packets=2387, n_bytes=343174, priority=1 actions=drop
 cookie=0x0, duration=2563.722s, table=1, n_packets=8, n_bytes=496, ct_state=+new+trk,udp,nw_dst=172.16.100.254,tp_dst=69 actions=ct(commit,nat(dst=10.167.100.1),alg=tftp),output:vhost0
 cookie=0x0, duration=2563.717s, table=1, n_packets=0, n_bytes=0, ct_state=+new+rel+trk,ip,nw_dst=172.16.100.254 actions=ct(commit,nat),output:vhost0
 cookie=0x0, duration=2563.712s, table=1, n_packets=2, n_bytes=124, ct_state=+est+trk,ip,nw_dst=172.16.100.254 actions=ct(nat),output:vhost0
 cookie=0x0, duration=2563.707s, table=1, n_packets=10, n_bytes=5580, ct_state=+new+rel+trk,ip,nw_src=10.167.100.1 actions=ct(commit,nat),output:dpdk0
 cookie=0x0, duration=2563.703s, table=1, n_packets=50, n_bytes=27900, ct_state=+est+trk,ip,nw_src=10.167.100.1 actions=ct(nat),output:dpdk0
 cookie=0x0, duration=2257.596s, table=8, n_packets=10, n_bytes=600, reg2=0xac1064fe actions=load:0x10401->OXM_OF_PKT_REG0[]
 cookie=0x0, duration=2256.393s, table=8, n_packets=30, n_bytes=1458, priority=0 actions=load:0->OXM_OF_PKT_REG0[]
 cookie=0x0, duration=2438.001s, table=10, n_packets=34, n_bytes=1698, priority=100,arp,reg0=0,reg1=0 actions=NORMAL
 cookie=0x0, duration=2437.990s, table=10, n_packets=10, n_bytes=600, priority=10,arp,arp_op=1 actions=load:0x2->NXM_OF_ARP_OP[],move:NXM_NX_ARP_SHA[]->NXM_NX_ARP_THA[],move:OXM_OF_PKT_REG0[0..47]->NXM_NX_ARP_SHA[],move:NXM_OF_ARP_SPA[]->NXM_OF_ARP_TPA[],move:NXM_NX_REG2[]->NXM_OF_ARP_SPA[],move:NXM_OF_ETH_SRC[]->NXM_OF_ETH_DST[],move:OXM_OF_PKT_REG0[0..47]->NXM_OF_ETH_SRC[],move:NXM_OF_IN_PORT[]->NXM_NX_REG3[0..15],load:0->NXM_OF_IN_PORT[],output:NXM_NX_REG3[0..15]
 cookie=0x0, duration=2436.882s, table=10, n_packets=0, n_bytes=0, priority=0 actions=drop

[root@dell-per730-51 conntrack_dpdk]# ovs-appctl dpctl/dump-conntrack
udp,orig=(src=172.16.100.2,dst=172.16.100.254,sport=41042,dport=69),reply=(src=10.167.100.1,dst=172.16.100.2,sport=69,dport=41042),helper=tftp
udp,orig=(src=10.167.100.1,dst=172.16.100.2,sport=48350,dport=41042),reply=(src=172.16.100.2,dst=10.167.100.1,sport=41042,dport=48350)

[root@hp-dl385pg8-04 conntrack_dpdk]# tftp 172.16.100.254 
tftp> get 1024.pkt
Transfer timed out.

tftp> get 1024.pkt
Transfer timed out.


Pakets Caputured :
[root@dell-per730-51 conntrack_dpdk]# ovs-tcpdump -i vhost0 -nne 
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on mivhost0, link-type EN10MB (Ethernet), capture size 262144 bytes
02:51:19.243920 8c:7c:ff:20:f1:16 > 00:00:00:01:04:01, ethertype IPv4 (0x0800), length 62: 172.16.100.2.41042 > 10.167.100.1.69:  20 RRQ "1024.pkt" netascii
02:51:19.244898 00:00:00:01:04:01 > 8c:7c:ff:20:f1:16, ethertype IPv4 (0x0800), length 558: 10.167.100.1.47094 > 172.16.100.2.41042: UDP, length 516
02:51:20.245920 00:00:00:01:04:01 > 8c:7c:ff:20:f1:16, ethertype IPv4 (0x0800), length 558: 10.167.100.1.47094 > 172.16.100.2.41042: UDP, length 516 
02:51:22.247944 00:00:00:01:04:01 > 8c:7c:ff:20:f1:16, ethertype IPv4 (0x0800), length 558: 10.167.100.1.47094 > 172.16.100.2.41042: UDP, length 516
02:51:24.258629 00:00:00:01:04:01 > 8c:7c:ff:20:f1:16, ethertype ARP (0x0806), length 42: Request who-has 172.16.100.2 tell 10.167.100.1, length 28
02:51:24.258943 8c:7c:ff:20:f1:16 > 00:00:00:01:04:01, ethertype ARP (0x0806), length 60: Reply 172.16.100.2 is-at 8c:7c:ff:20:f1:16, length 46 
02:51:26.252022 00:00:00:01:04:01 > 8c:7c:ff:20:f1:16, ethertype IPv4 (0x0800), length 558: 10.167.100.1.47094 > 172.16.100.2.41042: UDP, length 516
02:51:34.260078 00:00:00:01:04:01 > 8c:7c:ff:20:f1:16, ethertype IPv4 (0x0800), length 558: 10.167.100.1.47094 > 172.16.100.2.41042: UDP, length 516


[root@dell-per730-51 conntrack_dpdk]# ovs-tcpdump -i dpdk0 -nne 
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on midpdk0, link-type EN10MB (Ethernet), capture size 262144 bytes
02:50:23.062391 8c:7c:ff:20:f1:16 > 00:00:00:01:04:01, ethertype IPv4 (0x0800), length 62: 172.16.100.2.41042 > 172.16.100.254.69:  20 RRQ "1024.pkt" netascii
02:50:23.063532 00:00:00:01:04:01 > 8c:7c:ff:20:f1:16, ethertype IPv4 (0x0800), length 558: 10.167.100.1.41733 > 172.16.100.2.41042: UDP, length 516
02:50:23.064029 8c:7c:ff:20:f1:16 > 00:00:00:01:04:01, ethertype IPv4 (0x0800), length 60: 172.16.100.2.41042 > 172.16.100.254.41733: UDP, length 4
02:50:24.064535 00:00:00:01:04:01 > 8c:7c:ff:20:f1:16, ethertype IPv4 (0x0800), length 558: 10.167.100.1.41733 > 172.16.100.2.41042: UDP, length 516
02:50:24.064970 8c:7c:ff:20:f1:16 > 00:00:00:01:04:01, ethertype IPv4 (0x0800), length 60: 172.16.100.2.41042 > 172.16.100.254.41733: UDP, length 4
02:50:26.066559 00:00:00:01:04:01 > 8c:7c:ff:20:f1:16, ethertype IPv4 (0x0800), length 558: 10.167.100.1.41733 > 172.16.100.2.41042: UDP, length 516
02:50:26.066999 8c:7c:ff:20:f1:16 > 00:00:00:01:04:01, ethertype IPv4 (0x0800), length 60: 172.16.100.2.41042 > 172.16.100.254.41733: UDP, length 4
02:50:28.066314 00:00:00:01:04:01 > 8c:7c:ff:20:f1:16, ethertype ARP (0x0806), length 42: Request who-has 172.16.100.2 tell 10.167.100.1, length 28
02:50:28.066592 8c:7c:ff:20:f1:16 > 00:00:00:01:04:01, ethertype ARP (0x0806), length 60: Reply 172.16.100.2 is-at 8c:7c:ff:20:f1:16, length 46

Comment 2 Flavio Leitner 2018-02-16 12:56:39 UTC
This could be a dup of #1485800

Comment 6 David Marchand 2018-12-18 12:51:48 UTC
The underlying problem is that the nat operation applied on the ct created from the tftp alg is wrong because of a problem in the ct engine.
Closing as duplicate of 1485800 which has the same issue.

*** This bug has been marked as a duplicate of bug 1485800 ***