Bug 1928164 - Traffic to outside the cluster redirected when OVN is used and NodePort service is configured
Summary: Traffic to outside the cluster redirected when OVN is used and NodePort servi...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Networking
Version: 4.6
Hardware: x86_64
OS: Linux
high
high
Target Milestone: ---
: 4.8.0
Assignee: Alexander Constantinescu
QA Contact: Arti Sood
URL:
Whiteboard:
: 1934737 (view as bug list)
Depends On:
Blocks: 1946696
TreeView+ depends on / blocked
 
Reported: 2021-02-12 14:23 UTC by Lucas López Montero
Modified: 2023-09-15 01:01 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1946696 (view as bug list)
Environment:
Last Closed: 2021-07-27 22:44:28 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2021:2438 0 None None None 2021-07-27 22:45:11 UTC

Description Lucas López Montero 2021-02-12 14:23:15 UTC
Description of problem:

As explained on this issue [1] (fix here [2]); when OVN is being used in a cluster, its local node firewalls are redirecting the traffic to the port configured on NodePort without filtering the destination IP.


How reproducible:


Steps to Reproduce:
oc new-project nodeport-issue
oc new-app https://github.com/openshift/ruby-hello-world
oc expose deployment ruby-hello-world --type=NodePort --name=ruby-hello-world-nodeport
oc describe services ruby-hello-world-nodeport | grep "NodePort:"

Login to one on the nodes and execute:
curl -4 -v http://www.google.de:<NodePort>

Actual results:

The output contains something like "Welcome to an OpenShift v3 Demo App".


Expected results:

Traffic should be directed to www.google.de in the example above



[1] https://github.com/ovn-org/ovn-kubernetes/issues/1981
[2] https://github.com/ovn-org/ovn-kubernetes/pull/2002

Comment 3 Alexander Constantinescu 2021-03-05 09:13:49 UTC
This made it in with the latest downstream merge: https://github.com/openshift/ovn-kubernetes/pull/440 so setting to MODIFIED

Comment 5 Alexander Constantinescu 2021-03-10 16:29:12 UTC
*** Bug 1934737 has been marked as a duplicate of this bug. ***

Comment 6 Arti Sood 2021-03-15 22:07:08 UTC
Verification:-

Build used for verification
oc version
Client Version: 4.7.0
Server Version: 4.8.0-0.nightly-2021-03-14-134919
Kubernetes Version: v1.20.0+e1bc274

Cluster Information

[asood@asood ~]$ oc get network -o jsonpath='{.items[*].status.networkType}'
OVNKubernetes

oc get --namespace openshift-ingress-operator ingresscontrollers/default --output jsonpath='{.status.endpointPublishingStrategy.type}'
NodePortService

Created the app and exposed the deployment.


[asood@asood ~]$ oc describe services ruby-hello-world-nodeport | grep "NodePort:"
NodePort:                 <unset>  31729/TCP


oc get nodes
NAME                                  STATUS   ROLES    AGE     VERSION
asood-03-15-2-s7gqc-compute-0         Ready    worker   5h14m   v1.20.0+e1bc274
asood-03-15-2-s7gqc-compute-1         Ready    worker   5h14m   v1.20.0+e1bc274
asood-03-15-2-s7gqc-compute-2         Ready    worker   5h14m   v1.20.0+e1bc274
asood-03-15-2-s7gqc-control-plane-0   Ready    master   5h26m   v1.20.0+e1bc274
asood-03-15-2-s7gqc-control-plane-1   Ready    master   5h26m   v1.20.0+e1bc274
asood-03-15-2-s7gqc-control-plane-2   Ready    master   5h26m   v1.20.0+e1bc274


oc debug node/asood-03-15-2-s7gqc-compute-0
Starting pod/asood-03-15-2-s7gqc-compute-0-debug ...
To use host binaries, run `chroot /host`
Pod IP: 10.0.97.115
If you don't see a command prompt, try pressing enter.
sh-4.4# chroot /host
sh-4.4# curl -4 -v http://www.google.de:31729
* Rebuilt URL to: http://www.google.de:31729/
*   Trying 172.217.7.195...
* TCP_NODELAY set


Traffic seems to be redirected but do not receive a response.


@aconstan  Should a response be received from www.google.de when curl is issued ?

Comment 7 Josef Meier 2021-03-16 08:26:16 UTC
In our case we could not even pull Docker Images from external registries with this bug.

Comment 8 Alexander Constantinescu 2021-03-16 09:27:47 UTC
Hi 

I don't know what google.de:$NODE_PORT is supposed to do, it obviously should not return a response as I don't think Google has anything that will listen on a port of your choice and return a response. 

Why don't you just use an external node (boostrap maybe?), launch a server running on a dedicated port on that node, then create a new service inside the cluster using the same node port and test that you can connect to the EXTERNAL_NODE:PORT without having the internal service interfere?

/Alex

Comment 9 Arti Sood 2021-03-16 18:52:32 UTC
Marking it verified based on comment#6

Comment 14 errata-xmlrpc 2021-07-27 22:44:28 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.8.2 bug fix and security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:2438

Comment 16 Red Hat Bugzilla 2023-09-15 01:01:11 UTC
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 500 days


Note You need to log in before you can comment on or make changes to this bug.