Bug 2061593
| Summary: | East-West traffic is broken with gateway router if DNAT configured | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux Fast Datapath | Reporter: | Numan Siddique <nusiddiq> |
| Component: | ovn-2021 | Assignee: | xsimonar |
| Status: | CLOSED ERRATA | QA Contact: | Jianlin Shi <jishi> |
| Severity: | medium | Docs Contact: | |
| Priority: | high | ||
| Version: | FDP 21.K | CC: | ctrautma, jiji, lorenzo.bianconi, mmichels, xsimonar |
| Target Milestone: | --- | Flags: | xsimonar:
needinfo+
|
| Target Release: | FDP 22.D | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | ovn-2021-21.12.0-80.el8fdp | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2022-08-01 14:11:03 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
reproduced with following script:
systemctl start openvswitch
systemctl start ovn-northd
ovn-nbctl set-connection ptcp:6641
ovn-sbctl set-connection ptcp:6642
ovs-vsctl set open . external_ids:system-id=hv1 external_ids:ovn-remote=tcp:1.1.178.25:6642 external_ids:ovn-encap-type=geneve external_ids:ovn-encap-ip=1.1.178.25
systemctl restart ovn-controller
ovn-nbctl lr-add R1
ovn-nbctl ls-add sw0
ovn-nbctl ls-add sw1
ovn-nbctl ls-add public
ovn-nbctl lrp-add R1 rp-sw0 00:00:01:01:02:03 192.168.1.1/24
ovn-nbctl lrp-add R1 rp-sw1 00:00:03:01:02:03 192.168.2.1/24
ovn-nbctl lrp-add R1 rp-public 00:00:02:01:02:03 172.16.1.1/24 1000::a/64
# -- lrp-set-gateway-chassis rp-public hv1
ovn-nbctl set logical_router R1 options:chassis=hv1
ovs-vsctl add-br br-ext
ovn-nbctl lsp-add sw0 sw0-rp -- set Logical_Switch_Port sw0-rp \
type=router options:router-port=rp-sw0 \
-- lsp-set-addresses sw0-rp router
ovn-nbctl lsp-add sw1 sw1-rp -- set Logical_Switch_Port sw1-rp \
type=router options:router-port=rp-sw1 \
-- lsp-set-addresses sw1-rp router
ovn-nbctl lsp-add public public-rp -- set Logical_Switch_Port public-rp \
type=router options:router-port=rp-public \
-- lsp-set-addresses public-rp router
ovs-vsctl add-port br-int sw01 -- set interface sw01 type=internal external_ids:iface-id=sw01
ip netns add sw01
ip link set sw01 netns sw01
ip netns exec sw01 ip link set sw01 address f0:00:00:01:02:03
ip netns exec sw01 ip link set sw01 up
ip netns exec sw01 ip addr add 192.168.1.2/24 dev sw01
ip netns exec sw01 ip route add default via 192.168.1.1 dev sw01
ovn-nbctl lsp-add sw0 sw01 \
-- lsp-set-addresses sw01 "f0:00:00:01:02:03 192.168.1.2"
ovs-vsctl add-port br-int sw11 -- set interface sw11 type=internal external_ids:iface-id=sw11
ip netns add sw11
ip link set sw11 netns sw11
ip netns exec sw11 ip link set sw11 address f0:00:00:02:02:03
ip netns exec sw11 ip link set sw11 up
ip netns exec sw11 ip addr add 192.168.2.2/24 dev sw11
ip netns exec sw11 ip route add default via 192.168.2.1 dev sw11
ovn-nbctl lsp-add sw1 sw11 \
-- lsp-set-addresses sw11 "f0:00:00:02:02:03 192.168.2.2"
ovs-vsctl add-port br-ext server -- set interface server type=internal
ip netns add server
ip netns exec server ip link set lo up
ip link set server netns server
ip netns exec server ip link set server up
ip netns exec server ip addr add 172.16.1.50/24 dev server
ip netns exec server ip route add default via 172.16.1.1 dev server
ovs-vsctl set Open_vSwitch . external-ids:ovn-bridge-mappings=phynet:br-ext
ovn-nbctl lsp-add public public1 \
-- lsp-set-addresses public1 unknown \
-- lsp-set-type public1 localnet \
-- lsp-set-options public1 network_name=phynet
ip netns exec sw01 nc -k -l 8000 &
sleep 2
ovn-nbctl lr-nat-add R1 snat 172.16.1.21 192.168.2.0/24
ip netns exec sw11 nc 192.168.1.2 8000 <<< h
ip netns exec sw11 ping 192.168.1.2 -c 1
ovn-nbctl lr-nat-add R1 dnat_and_snat 172.16.1.2 192.168.1.2
ip netns exec sw11 nc 192.168.1.2 8000 <<< h
ip netns exec sw11 nc 192.168.1.2 8000 <<< h
ip netns exec sw11 nc 192.168.1.2 8000 <<< h
the issue exist from the first version of ovn-2021, ovn-2021-21.03.0-21:
[root@wsfd-advnetlab16 bz2061593]# rpm -qa | grep -E "openvswitch2.15|ovn-2021"
ovn-2021-21.03.0-21.el8fdp.x86_64
ovn-2021-central-21.03.0-21.el8fdp.x86_64
openvswitch2.15-2.15.0-80.el8fdp.x86_64
ovn-2021-host-21.03.0-21.el8fdp.x86_64
+ ip netns exec sw01 nc -k -l 8000
+ ovn-nbctl lr-nat-add R1 snat 172.16.1.21 192.168.2.0/24
+ ip netns exec sw11 nc 192.168.1.2 8000
h
+ ip netns exec sw11 ping 192.168.1.2 -c 1
PING 192.168.1.2 (192.168.1.2) 56(84) bytes of data.
64 bytes from 192.168.1.2: icmp_seq=1 ttl=63 time=0.430 ms
--- 192.168.1.2 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.430/0.430/0.430/0.000 ms
+ ovn-nbctl lr-nat-add R1 dnat_and_snat 172.16.1.2 192.168.1.2
+ ip netns exec sw11 nc 192.168.1.2 8000
h
Ncat: Connection reset by peer.
<=== connection failed
the issue also exist on ovn2.13-20.12.0-9:
+ ovn-nbctl lr-nat-add R1 snat 172.16.1.21 192.168.2.0/24
+ ip netns exec sw11 nc 192.168.1.2 8000
h
+ ip netns exec sw11 ping 192.168.1.2 -c 1
PING 192.168.1.2 (192.168.1.2) 56(84) bytes of data.
64 bytes from 192.168.1.2: icmp_seq=1 ttl=63 time=0.522 ms
--- 192.168.1.2 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.522/0.522/0.522/0.000 ms
+ ovn-nbctl lr-nat-add R1 dnat_and_snat 172.16.1.2 192.168.1.2
+ ip netns exec sw11 nc 192.168.1.2 8000
h
Ncat: Connection reset by peer.
[root@wsfd-advnetlab16 bz2061593]# rpm -qa | grep ovn2.13
ovn2.13-host-20.12.0-9.el8fdp.x86_64
ovn2.13-central-20.12.0-9.el8fdp.x86_64
ovn2.13-20.12.0-9.el8fdp.x86_64
upstream fix:http://patchwork.ozlabs.org/project/ovn/patch/20220401163402.3977720-1-xsimonar@redhat.com/ Moving this to MODIFIED since this is now fixed. Hi xsimonar, will this bug be fixed on ovn-2021 This issue is fixed in ovn-2021-21.12.0-80.el8fdp This issue is fixed in ovn-2021-21.12.0-80.el8fdp This issue is fixed in ovn-2021-21.12.0-80.el8fdp This issue is fixed in ovn-2021-21.12.0-80.el8fdp This issue is fixed in ovn-2021-21.12.0-80.el8fdp This issue is fixed in ovn-2021-21.12.0-80.el8fdp This issue is fixed in ovn-2021-21.12.0-80.el8fdp This issue is fixed in ovn-2021-21.12.0-80.el8fdp This issue is fixed in ovn-2021-21.12.0-80.el8fdp This issue has been cloned at https://bugzilla.redhat.com/show_bug.cgi?id=2101832 Verified on ovn-2021-21.12.0-82.el8:
+ ovn-nbctl lsp-add public public1 -- lsp-set-addresses public1 unknown -- lsp-set-type public1 localnet -- lsp-set-options public1 network_name=phynet
+ sleep 2
+ ip netns exec sw01 nc -k -l 8000
+ ovn-nbctl lr-nat-add R1 snat 172.16.1.21 192.168.2.0/24
+ ip netns exec sw11 nc 192.168.1.2 8000
h
+ ip netns exec sw11 ping 192.168.1.2 -c 1
PING 192.168.1.2 (192.168.1.2) 56(84) bytes of data.
64 bytes from 192.168.1.2: icmp_seq=1 ttl=63 time=0.209 ms
--- 192.168.1.2 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.209/0.209/0.209/0.000 ms
+ ovn-nbctl lr-nat-add R1 dnat_and_snat 172.16.1.2 192.168.1.2
+ ip netns exec sw11 nc 192.168.1.2 8000
h
+ ip netns exec sw11 nc 192.168.1.2 8000
h
+ ip netns exec sw11 nc 192.168.1.2 8000
h
[root@dell-per730-20 bz2061593]# rpm -qa | grep -E "openvswitch2.17|ovn-2021"
openvswitch2.17-2.17.0-31.el8fdp.x86_64
ovn-2021-central-21.12.0-82.el8fdp.x86_64
ovn-2021-21.12.0-82.el8fdp.x86_64
ovn-2021-host-21.12.0-82.el8fdp.x86_64
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (ovn-2021 bug fix and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2022:5787 |
Description of problem: For the below OVN topology, TCP/UDP traffuc from sw1-p1-ew (20.0.0.3) to sw0-p1-ew (10.0.0.3) is broken. Seems like this is broken since the very early versions of OVN. #ovn-nbctl show switch e06d35b2-aad1-4ad6-96d7-34e26017e576 (sw0) port sw0-lr0 type: router addresses: ["00:00:00:00:ff:01"] router-port: lr0-sw0 port sw0-p1-ew addresses: ["50:54:00:00:00:03 10.0.0.3"] switch 9bcb67b2-8b35-48ed-be53-72dc5a6727de (sw1) port sw1-lr0 type: router addresses: ["00:00:00:00:ff:02"] router-port: lr0-sw1 port sw1-p1-ew addresses: ["40:54:00:00:00:03 20.0.0.3"] switch 2ee51c7f-5945-4ab8-95ad-113393142cb2 (public) port public-lr0 type: router router-port: lr0-public router b2b41177-5986-4739-8d29-0ab37866f02b (lr0) port lr0-sw1 mac: "00:00:00:00:ff:02" networks: ["20.0.0.1/24"] port lr0-sw0 mac: "00:00:00:00:ff:01" networks: ["10.0.0.1/24"] port lr0-public mac: "00:00:20:20:12:13" networks: ["172.168.0.100/24"] nat a1962373-bb13-4940-b11f-ca66f1f322a6 external ip: "172.168.0.110" logical ip: "10.0.0.3" type: "dnat_and_snat" nat c5e9eb71-6704-4bb3-ab6e-4167f941eacf external ip: "172.168.0.100" logical ip: "20.0.0.0/24" type: "snat" #ovn-sbctl show Chassis hv1 hostname: fedora Encap geneve ip: "169.0.0.1" options: {csum="true"} Port_Binding lr0-sw0 Port_Binding sw1-p1-ew Port_Binding sw0-p1-ew Port_Binding sw1-lr0 Port_Binding lr0-sw1 Port_Binding lr0-public Port_Binding public-lr0 Port_Binding sw0-lr0 # Steps to reproduce. Create the above topology. # Start nc server on sw0-p1 #ip netns exec n1 nc -l -k -v 192.168.1.11 8000 Start nc client on sw1-p1 #ip netns exec n2 nc 192.168.1.11 8000 The initial connection goes through. But any subsequent causes connection reset. [root@wsfd-advnetlab085 numans]# ip netns exec n2 nc 192.168.1.11 8000 test Ncat: Connection reset by peer. The issue is not seen when the dnat_and_snat entry - 172.168.0.110 - 10.0.0.3 is deleted. Version-Release number of selected component (if applicable): How reproducible: Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Additional info: