Bug 1898616

*** Bug 1913062 has been marked as a duplicate of this bug. ***

https://github.com/openshift/whereabouts-cni/pull/37 merged 5 day ago

Verification fail in 4.6.0-0.nightly-2021-03-21-131139

Will re test it at the end of this week.

Tested and verified in 4.7.0-0.nightly-2021-03-25-091845

Test passed in 4.7.0-0.nightly-2021-03-25-091845,
but still failed in 4.6.0-0.nightly-2021-03-25-094616, first address is not `caa5::8000`

[weliang@weliang Config]$ oc describe pod | grep caa5
                        "caa5::8001"
                        "caa5::8001"
  Normal  AddedInterface  15s   multus             Add net1 [caa5::8001/112] from test/whereabouts-excludeipv6
                        "caa5::8002"
                        "caa5::8002"
  Normal  AddedInterface  14s   multus             Add net1 [caa5::8002/112] from test/whereabouts-excludeipv6
                        "caa5::8003"
                        "caa5::8003"
  Normal  AddedInterface  14s   multus             Add net1 [caa5::8003/112] from test/whereabouts-excludeipv6
                        "caa5::8005"
                        "caa5::8005"
  Normal  AddedInterface  13s   multus             Add net1 [caa5::8005/112] from test/whereabouts-excludeipv6
                        "caa5::8004"
                        "caa5::8004"
  Normal  AddedInterface  14s   multus             Add net1 [caa5::8004/112] from test/whereabouts-excludeipv6
[weliang@weliang Config]$

I've deployed a fresh 4.6.31, then:

* Edited the cluster network to add an additional network:

```
...
spec:
  additionalNetworks:
  - name: test-network-1
    namespace: test-1
    rawCNIConfig: '{ "cniVersion": "0.3.1", "name": "test-network-1", "type": "ipvlan", "master": "ens5", "mode": "l2", "ipam": { "type": "whereabouts", "range": "caa5::0/112", "exclude": ["caa5::0/113"] } }'
    type: Raw
...
```

* Created the 'test-1' project

```
oc new-project test-1
```

* Created a hello-openshift pod as:

```
apiVersion: v1
kind: Pod
metadata:
  annotations:
    k8s.v1.cni.cncf.io/networks: test-network-1
  labels:
    name: hello-openshift
  name: hello-openshift
  namespace: test-1
spec:
  containers:
  - image: openshift/hello-openshift
    imagePullPolicy: IfNotPresent
    name: hello-openshift
    ports:
    - containerPort: 8080
      protocol: TCP
    volumeMounts:
    - mountPath: /tmp
      name: tmp
  volumes:
  - emptyDir: {}
    name: tmp
```

* The describe seems to show the proper IPs:

```
$ oc describe pod hello-openshift 
Name:         hello-openshift
Namespace:    test-1
Priority:     0
Node:         ip-10-0-176-175.eu-west-3.compute.internal/10.0.176.175
Start Time:   Thu, 10 Jun 2021 10:42:58 +0000
Labels:       name=hello-openshift
Annotations:  k8s.v1.cni.cncf.io/network-status:
                [{
                    "name": "",
                    "interface": "eth0",
                    "ips": [
                        "10.129.2.57"
                    ],
                    "default": true,
                    "dns": {}
                },{
                    "name": "test-1/test-network-1",
                    "interface": "net1",
                    "ips": [
                        "caa5::8000"
                    ],
                    "mac": "0a:a5:48:ad:9a:88",
                    "dns": {}
                }]
              k8s.v1.cni.cncf.io/networks: test-network-1
              k8s.v1.cni.cncf.io/networks-status:
                [{
                    "name": "",
                    "interface": "eth0",
                    "ips": [
                        "10.129.2.57"
                    ],
                    "default": true,
                    "dns": {}
                },{
                    "name": "test-1/test-network-1",
                    "interface": "net1",
                    "ips": [
                        "caa5::8000"
                    ],
                    "mac": "0a:a5:48:ad:9a:88",
                    "dns": {}
                }]
              openshift.io/scc: anyuid
Status:       Running
IP:           10.129.2.57
IPs:
  IP:  10.129.2.57
Containers:
  hello-openshift:
    Container ID:   cri-o://d7116eb428e06988e814bffef6a261672881e7ebf65f726921a79c9451df7a32
    Image:          openshift/hello-openshift
    Image ID:       docker.io/openshift/hello-openshift@sha256:aaea76ff622d2f8bcb32e538e7b3cd0ef6d291953f3e7c9f556c1ba5baf47e2e
    Port:           8080/TCP
    Host Port:      0/TCP
    State:          Running
      Started:      Thu, 10 Jun 2021 10:43:00 +0000
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /tmp from tmp (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-4ktzq (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             True 
  ContainersReady   True 
  PodScheduled      True 
Volumes:
  tmp:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:     
    SizeLimit:  <unset>
  default-token-4ktzq:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-4ktzq
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                 node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type    Reason          Age    From               Message
  ----    ------          ----   ----               -------
  Normal  Scheduled       5m47s  default-scheduler  Successfully assigned test-1/hello-openshift to ip-10-0-176-175.eu-west-3.compute.internal
  Normal  AddedInterface  5m46s  multus             Add eth0 [10.129.2.57/23]
  Normal  AddedInterface  5m46s  multus             Add net1 [caa5::8000/112] from test-1/test-network-1
  Normal  Pulled          5m46s  kubelet            Container image "openshift/hello-openshift" already present on machine
  Normal  Created         5m46s  kubelet            Created container hello-openshift
  Normal  Started         5m46s  kubelet            Started container hello-openshift
```

That looks verified to me, thanks Edwardo!

(In reply to Douglas Smith from comment #13)
> That looks verified to me, thanks Edwardo!

Great!

Anything else I can do to move forward the BZ? Thanks.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.6.47 bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:3737