Bug 1703604

Description of problem: Configured openshift-sdn as the default network for OpenShift 4.x and also configured openshift-sdn through multus. The secondary network interface is created without an issue but the IP associated with the secondary one is not routable. 


Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1. Create NetworkAttachmentDefintion:
cat <<EOF | kubectl create -f -
apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
  name: openshift-sdn-extra
spec:
  config: '{
  "cniVersion": "0.3.1",
  "name": "openshift-sdn",
  "type": "openshift-sdn"
  }'
EOF

2. Create two pods to talk to each other:
cat <<EOF | kubectl create -f -
apiVersion: v1
kind: Pod
metadata:
  name: my-application
  annotations:
    k8s.v1.cni.cncf.io/networks: openshift-sdn-extra
spec:
  containers:
  - name: samplepod
    command: ["/bin/bash", "-c", "sleep 2000000000000"]
    image: dougbtv/centos-network
EOF


cat <<EOF | kubectl create -f -
apiVersion: v1
kind: Pod
metadata:
  name: my-database
  annotations:
    k8s.v1.cni.cncf.io/networks: openshift-sdn-extra
spec:
  containers:
  - name: samplepod
    command: ["/bin/bash", "-c", "sleep 2000000000000"]
    image: dougbtv/centos-network
EOF

3. Check the IP addresses of the secondary network interfaces on each pod:
# oc exec my-application -- ip a
# oc exec my-database -- ip a

4. Try to ping the secondary networking interface IP from one pod to another:
# oc exec my-application -- ping x.x.x.x
# oc exec my-database -- ping y.y.y.y

Actual results: Ping works fine for primary network interface while it fails for the second:
# oc exec my-application -- ip a
...
3: eth0@if21: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 8951 qdisc noqueue state UP 
    link/ether 0a:58:0a:83:00:10 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 10.131.0.16/23 brd 10.131.1.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::f461:9ff:fe5a:582b/64 scope link 
       valid_lft forever preferred_lft forever
5: net1@if22: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 8951 qdisc noqueue state UP 
    link/ether 0a:58:0a:83:00:11 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 10.131.0.17/23 brd 10.131.1.255 scope global net1
       valid_lft forever preferred_lft forever
    inet6 fe80::14b3:edff:feb3:6292/64 scope link 
       valid_lft forever preferred_lft forever

# oc exec my-database -- ping 10.131.0.16
PING 10.131.0.16 (10.131.0.16) 56(84) bytes of data.
64 bytes from 10.131.0.16: icmp_seq=1 ttl=64 time=0.711 ms
# oc exec my-database -- ping 10.131.0.17
PING 10.131.0.17 (10.131.0.17) 56(84) bytes of data.
From 10.129.2.9 icmp_seq=1 Destination Host Unreachable


Expected results: I would expect both IP addresses to be routable.