Description of problem: According to the description from https://github.com/openshift/origin/pull/14924. The OVS rules should add the nodes local IP allowing for dns resolution when dns is accessible on the node, this works in AWS env, but in Openstack, the nodes public IP address is added in OVS, then Bug 1458849 still happens. Version-Release number of selected component (if applicable): oc v3.6.173.0.1 How reproducible: 100% Steps to Reproduce: 1.Create openshift cluster in openstack env 2.ovs-ofctl dump-flows br0 -O openflow13 | grep tcp | grep tp_dst=53 Actual results: Test result form running in Openstack env:(18.175.183 is node public IP) (ovs-ofctl dump-flows br0 -O openflow13 | grep tcp | grep tp_dst=53 || docker exec openvswitch ovs-ofctl dump-flows br0 -O openflow13 grep tcp grep tp_dst=53 )` @ssh://root.centralci.eng.rdu2.redhat.com cookie=0x0, duration=2288.804s, table=60, n_packets=0, n_bytes=0, priority=100,tcp,nw_dst=172.30.0.1,tp_dst=53 actions=load:0->NXM_NX_REG1[],load:0x2->NXM_NX_REG2[],goto_table:80 cookie=0x0, duration=2289.112s, table=100, n_packets=0, n_bytes=0, priority=51,tcp,nw_dst=10.8.175.183,tp_dst=53 actions=output:2 Test results from running in AWS env: ( 172.18.3.66 is node local IP) (ovs-ofctl dump-flows br0 -O openflow13 | grep udp | grep tp_dst=53 || docker exec openvswitch ovs-ofctl dump-flows br0 -O openflow13 grep udp grep tp-dst=53 )` @ssh://root.amazonaws.com cookie=0x0, duration=5987.446s, table=60, n_packets=0, n_bytes=0, priority=100,udp,nw_dst=172.30.0.1,tp_dst=53 actions=load:0->NXM_NX_REG1[],load:0x2->NXM_NX_REG2[],goto_table:80 cookie=0x0, duration=11417.326s, table=100, n_packets=418, n_bytes=37425, priority=51,udp,nw_dst=172.18.3.66,tp_dst=53 actions=output:2 Expected results: In openstack env, nw_dst should be node local ip not public ip Additional info:
One workaround for this bug is adding a rule to allow node local IP (e.g. 172.16.0.0/12) in the egressnetworkpolicy if deny 0.0.0.0/0 also existed. apiVersion: v1 items: - apiVersion: v1 kind: EgressNetworkPolicy metadata: name: policy-test spec: egress: - to: dnsName: www.facebook.com type: Allow - to: cidrSelector: 172.16.0.0/12 type: Allow - to: cidrSelector: 0.0.0.0/0 type: Deny
The IP address that's wrong there (10.8.175.183) comes directly from the node config file's "nodeIP" option. For the OpenStack setup, what is that set to? If it's not set to anything then OpenShift has no way to determine the correct IP if the node has multiple IPs. OpenShift will attempt to reverse-DNS the node's hostname and if that fails, pick the first IP address from the NIC with the default route. So in any case, if the node has multiple IP addresses, the node config YAML should have the nodeIP option set to the private address.
After updating nodeIP with node private local IP and restart openvswitch, the OVS rule show correct rule with private local IP address. [root@host-8-241-83 ~]# systemctl restart openvswitch [root@host-8-241-83 ~]# ovs-ofctl dump-flows br0 -O openflow13 | grep tcp | grep tp_dst=53 cookie=0x0, duration=1.813s, table=60, n_packets=0, n_bytes=0, priority=100,tcp,nw_dst=172.30.0.1,tp_dst=53 actions=load:0->NXM_NX_REG1[],load:0x2->NXM_NX_REG2[],goto_table:80 cookie=0x0, duration=2.413s, table=100, n_packets=0, n_bytes=0, priority=51,tcp,nw_dst=172.16.120.30,tp_dst=53 actions=output:2 [root@host-8-241-83 ~]#