Description of problem: Configured openshift-sdn as the default network for OpenShift 4.x and also configured openshift-sdn through multus. The secondary network interface is created without an issue but the IP associated with the secondary one is not routable. Version-Release number of selected component (if applicable): How reproducible: Steps to Reproduce: 1. Create NetworkAttachmentDefintion: cat <<EOF | kubectl create -f - apiVersion: "k8s.cni.cncf.io/v1" kind: NetworkAttachmentDefinition metadata: name: openshift-sdn-extra spec: config: '{ "cniVersion": "0.3.1", "name": "openshift-sdn", "type": "openshift-sdn" }' EOF 2. Create two pods to talk to each other: cat <<EOF | kubectl create -f - apiVersion: v1 kind: Pod metadata: name: my-application annotations: k8s.v1.cni.cncf.io/networks: openshift-sdn-extra spec: containers: - name: samplepod command: ["/bin/bash", "-c", "sleep 2000000000000"] image: dougbtv/centos-network EOF cat <<EOF | kubectl create -f - apiVersion: v1 kind: Pod metadata: name: my-database annotations: k8s.v1.cni.cncf.io/networks: openshift-sdn-extra spec: containers: - name: samplepod command: ["/bin/bash", "-c", "sleep 2000000000000"] image: dougbtv/centos-network EOF 3. Check the IP addresses of the secondary network interfaces on each pod: # oc exec my-application -- ip a # oc exec my-database -- ip a 4. Try to ping the secondary networking interface IP from one pod to another: # oc exec my-application -- ping x.x.x.x # oc exec my-database -- ping y.y.y.y Actual results: Ping works fine for primary network interface while it fails for the second: # oc exec my-application -- ip a ... 3: eth0@if21: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 8951 qdisc noqueue state UP link/ether 0a:58:0a:83:00:10 brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 10.131.0.16/23 brd 10.131.1.255 scope global eth0 valid_lft forever preferred_lft forever inet6 fe80::f461:9ff:fe5a:582b/64 scope link valid_lft forever preferred_lft forever 5: net1@if22: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 8951 qdisc noqueue state UP link/ether 0a:58:0a:83:00:11 brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 10.131.0.17/23 brd 10.131.1.255 scope global net1 valid_lft forever preferred_lft forever inet6 fe80::14b3:edff:feb3:6292/64 scope link valid_lft forever preferred_lft forever # oc exec my-database -- ping 10.131.0.16 PING 10.131.0.16 (10.131.0.16) 56(84) bytes of data. 64 bytes from 10.131.0.16: icmp_seq=1 ttl=64 time=0.711 ms # oc exec my-database -- ping 10.131.0.17 PING 10.131.0.17 (10.131.0.17) 56(84) bytes of data. From 10.129.2.9 icmp_seq=1 Destination Host Unreachable Expected results: I would expect both IP addresses to be routable.
We definitely don't support two openshift-sdn interfaces - I'm surprised this worked at all...