Bug 1812678

Summary: [4.4.z] Incorrect RBAC for Whereabouts should be updated to ippools.whereabouts.cni.cncf.io
Product: OpenShift Container Platform Reporter: Douglas Smith <dosmith>
Component: NetworkingAssignee: Douglas Smith <dosmith>
Networking sub component: multus QA Contact: Weibin Liang <weliang>
Status: CLOSED ERRATA Docs Contact:
Severity: high    
Priority: high CC: bbennett, cswanson, weliang, william.caban, zzhao
Version: 4.4   
Target Milestone: ---   
Target Release: 4.4.z   
Hardware: All   
OS: All   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 1812676
: 1812680 (view as bug list) Environment:
Last Closed: 2020-06-02 11:17:47 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1812680    
Bug Blocks: 1812676    

Description Douglas Smith 2020-03-11 20:21:46 UTC
+++ This bug was initially created as a clone of Bug #1812676 +++

Description of problem: The RBAC for the ippools.whereabouts.cni.cncf.io for whereabouts IPAM CNI is incorrect.


Version-Release number of selected component (if applicable):


How reproducible: always


Steps to Reproduce: Use whereabouts IPAM CNI

Actual results:

```
  Warning  FailedCreatePodSandBox  6s         kubelet, ip-10-0-136-158.us-west-2.compute.internal  Failed to create pod sandbox: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_samplepod_openshift-multus_37058433-2564-42f2-aa91-d1b11f4c8bb5_0(7f6354c73261945d7d3c29aad3dd48b94aec7248d92b4650ea8554cc14755153): Multus: [openshift-multus/samplepod]: error adding container to network "whereaboutsexample": delegateAdd: error invoking DelegateAdd - "macvlan": error in getting result from AddNetwork: Error assigning IP: ippools.whereabouts.cni.cncf.io is forbidden: User "system:serviceaccount:openshift-multus:multus" cannot list resource "ippools" in API group "whereabouts.cni.cncf.io" in the namespace "openshift-multus"

```


Expected results: No error.


Additional info: This is the offending line @ https://github.com/openshift/cluster-network-operator/pull/526/files#diff-44eeae854395120fe566c1e3ddd5429bR88

This was found while diagnosing https://bugzilla.redhat.com/show_bug.cgi?id=1812245 which is also related to the change of CRD namespace for Whereabouts IPAM CNI.

Comment 1 zhaozhanqi 2020-04-10 02:57:27 UTC
there is workaround from https://bugzilla.redhat.com/show_bug.cgi?id=1812676#c1

Comment 5 zhaozhanqi 2020-04-17 07:36:34 UTC
Try to verified this bug on 4.4.0-0.nightly-2020-04-16-231032

found the pod cannot be created with error: 
  Successfully assigned z1/pod-macvlan-bridge-whereabouts to ip-10-0-145-193.us-east-2.compute.internal
  Warning  FailedCreatePodSandBox  15m                 kubelet, ip-10-0-145-193.us-east-2.compute.internal  Failed to create pod sandbox: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_pod-macvlan-bridge-whereabouts_z1_633fafe8-faaf-4ca5-a419-9e78ac704064_0(2a13058892f599d9588b4f227f1d54d86db930c2e71323926b098a1334ea3302): Multus: [z1/pod-macvlan-bridge-whereabouts]: error adding container to network "whereabouts": delegateAdd: error invoking DelegateAdd - "macvlan": error in getting result from AddNetwork: Error assigning IP: no matches for kind "IPPool" in version "whereabouts.cni.k8s.io/v1alpha1"
  Warning  FailedCreatePodSandBox  15m                 kubelet, ip-10-0-145-193.us-east-2.compute.internal  Failed to create pod sandbox: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_pod-macvlan-bridge-whereabouts_z1_633fafe8-faaf-4ca5-a419-9e78ac704064_0(50b5b34cccd8ba20737f841dde36dc92e347ecd177f817a8feee87682c8165e5): Multus: [z1/pod-macvlan-bridge-whereabouts]: error adding container to network "whereabouts": delegateAdd: error invoking DelegateAdd - "macvlan": error in getting result from AddNetwork: Error assigning IP: no matches for kind "IPPool" in version "whereabouts.cni.k8s.io/v1alpha1"


# oc get net-attach-def -n z1 macvlan-bridge-whereabouts -o yaml
apiVersion: k8s.cni.cncf.io/v1
kind: NetworkAttachmentDefinition
metadata:
  creationTimestamp: "2020-04-17T07:08:59Z"
  generation: 1
  name: macvlan-bridge-whereabouts
  namespace: z1
  resourceVersion: "24477"
  selfLink: /apis/k8s.cni.cncf.io/v1/namespaces/z1/network-attachment-definitions/macvlan-bridge-whereabouts
  uid: 0ac647cf-630e-4b7d-96e1-4cbe9a681b44
spec:
  config: '{ "cniVersion": "0.3.0", "name": "whereabouts", "type": "macvlan", "master":
    "ens3", "mode": "bridge", "ipam": { "type": "whereabouts", "range": "192.168.2.225/28"
    } }'



cat pod.yaml:

apiVersion: v1
kind: Pod
metadata:
  name: pod-name
  annotations:
    k8s.v1.cni.cncf.io/networks: macvlan-bridge-whereabouts
spec:
  containers:
  - name: pod-name
    image: aosqe/hello-openshift

Comment 6 Douglas Smith 2020-04-22 20:49:15 UTC
There's a pending patch for this for 4.4.z @ https://github.com/openshift/whereabouts-cni/pull/8

In order to work around this in 4.4 until that patch lands, you'll likely need to apply a few YAML files. One for a CRD, and one for RBAC that lines up with that CRD.

```
oc apply -f https://raw.githubusercontent.com/dougbtv/whereabouts/d3c8d8e28b4483d8c4e7be533eb1084ceabebbc0/doc/whereabouts.cni.k8s.io_ippools.yaml
oc apply -f https://gist.githubusercontent.com/dougbtv/3006e9ab1ec7517138de7ddc24341f51/raw/3213f7045c8ccaa16cd4b242d5eee0904422472c/rbac.yml
```

Comment 8 Douglas Smith 2020-05-05 18:16:35 UTC
Just an update and clarification...

The change for this merged in https://github.com/openshift/cluster-network-operator/pull/528

However, it depends on this PR to merge before it can be tested: https://github.com/openshift/whereabouts-cni/pull/8

Comment 11 zhaozhanqi 2020-05-09 02:58:24 UTC
Move the status to 'Post' since https://github.com/openshift/whereabouts-cni/pull/8 still not be merged.

Comment 13 zhaozhanqi 2020-05-20 08:28:05 UTC
Move this bug to ON_QA since PR  https://github.com/openshift/whereabouts-cni/pull/8 already merged

Comment 14 zhaozhanqi 2020-05-20 08:33:04 UTC
Verified this bug on 4.4.0-0.nightly-2020-05-18-164758

1. Create NAD from testdata/networking/multus-cni/NetworkAttachmentDefinitions/whereabouts-macvlan.yaml
2. create test pod testdata/networking/multus-cni/Pods/generic_multus_pod.yaml
3. pod can be running well and the second ip is correct.

Comment 17 errata-xmlrpc 2020-06-02 11:17:47 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:2310