Bug 1477323 - Nodes public IP address is added in OVS rules for dns resolution in openstack env
Nodes public IP address is added in OVS rules for dns resolution in openstack...
Status: CLOSED NOTABUG
Product: OpenShift Container Platform
Classification: Red Hat
Component: Networking (Show other bugs)
3.6.1
Unspecified Linux
unspecified Severity medium
: ---
: 3.7.0
Assigned To: Dan Williams
Meng Bo
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2017-08-01 15:20 EDT by Weibin Liang
Modified: 2017-08-16 10:18 EDT (History)
5 users (show)

See Also:
Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2017-08-16 10:18:02 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Weibin Liang 2017-08-01 15:20:24 EDT
Description of problem:
According to the description from https://github.com/openshift/origin/pull/14924. The OVS rules should add the nodes local IP allowing for dns resolution when dns is accessible on the node, this works in AWS env, but in Openstack, the nodes
public IP address is added in OVS, then Bug 1458849 still happens.

Version-Release number of selected component (if applicable):
oc v3.6.173.0.1


How reproducible:
100%

Steps to Reproduce:
1.Create openshift cluster in openstack env
2.ovs-ofctl dump-flows br0 -O openflow13 | grep tcp | grep tp_dst=53

Actual results:

Test result form running in Openstack env:(18.175.183 is node public IP)
(ovs-ofctl dump-flows br0 -O openflow13 | grep tcp | grep tp_dst=53  || docker exec openvswitch ovs-ofctl dump-flows br0 -O openflow13
      grep tcp
      grep tp_dst=53 )` @ssh://root@host-8-175-183.host.centralci.eng.rdu2.redhat.com
       cookie=0x0, duration=2288.804s, table=60, n_packets=0, n_bytes=0, priority=100,tcp,nw_dst=172.30.0.1,tp_dst=53 actions=load:0->NXM_NX_REG1[],load:0x2->NXM_NX_REG2[],goto_table:80
       cookie=0x0, duration=2289.112s, table=100, n_packets=0, n_bytes=0, priority=51,tcp,nw_dst=10.8.175.183,tp_dst=53 actions=output:2
      

Test results from running in AWS env: ( 172.18.3.66 is node local IP)
(ovs-ofctl dump-flows br0 -O openflow13 | grep udp | grep tp_dst=53  || docker exec openvswitch ovs-ofctl dump-flows br0 -O openflow13
      grep udp
      grep tp-dst=53 )` @ssh://root@ec2-54-145-184-202.compute-1.amazonaws.com
       cookie=0x0, duration=5987.446s, table=60, n_packets=0, n_bytes=0, priority=100,udp,nw_dst=172.30.0.1,tp_dst=53 actions=load:0->NXM_NX_REG1[],load:0x2->NXM_NX_REG2[],goto_table:80
       cookie=0x0, duration=11417.326s, table=100, n_packets=418, n_bytes=37425, priority=51,udp,nw_dst=172.18.3.66,tp_dst=53 actions=output:2


Expected results:
In openstack env, nw_dst should be node local ip not public ip

Additional info:
Comment 1 hongli 2017-08-01 22:16:50 EDT
One workaround for this bug is adding a rule to allow node local IP (e.g. 172.16.0.0/12) in the egressnetworkpolicy if deny 0.0.0.0/0 also existed.

apiVersion: v1
items:
- apiVersion: v1
  kind: EgressNetworkPolicy
  metadata:
    name: policy-test
  spec:
    egress:
    - to:
        dnsName: www.facebook.com
      type: Allow
    - to:
        cidrSelector: 172.16.0.0/12
      type: Allow
    - to:
        cidrSelector: 0.0.0.0/0
      type: Deny
Comment 2 Dan Williams 2017-08-15 14:00:05 EDT
The IP address that's wrong there (10.8.175.183) comes directly from the node config file's "nodeIP" option.  For the OpenStack setup, what is that set to?

If it's not set to anything then OpenShift has no way to determine the correct IP if the node has multiple IPs.  OpenShift will attempt to reverse-DNS the node's hostname and if that fails, pick the first IP address from the NIC with the default route.

So in any case, if the node has multiple IP addresses, the node config YAML should have the nodeIP option set to the private address.
Comment 3 Weibin Liang 2017-08-16 10:18:02 EDT
After updating nodeIP with node private local IP and restart openvswitch, the OVS
rule show correct rule with private local IP address.

[root@host-8-241-83 ~]# systemctl restart openvswitch
[root@host-8-241-83 ~]# ovs-ofctl dump-flows br0 -O openflow13 | grep tcp | grep tp_dst=53
 cookie=0x0, duration=1.813s, table=60, n_packets=0, n_bytes=0, priority=100,tcp,nw_dst=172.30.0.1,tp_dst=53 actions=load:0->NXM_NX_REG1[],load:0x2->NXM_NX_REG2[],goto_table:80
 cookie=0x0, duration=2.413s, table=100, n_packets=0, n_bytes=0, priority=51,tcp,nw_dst=172.16.120.30,tp_dst=53 actions=output:2
[root@host-8-241-83 ~]#

Note You need to log in before you can comment on or make changes to this bug.