Description of problem: Issue: Upgrade from OCP 4.7.31/32/16 to 4.7.33 is failing MULTUS IPVLAN Upgrade from 4.7.16 to 4.7.31/32 was successful. Version-Release number of selected component (if applicable): 4.7.33 How reproducible: Upgrade from OCP 4.7.31/32/16 to 4.7.33 is failing with MULTUS IPVLAN Steps to Reproduce: Upgrade from OCP 4.7.31/32/16 to 4.7.33 is failing with MULTUS IPVLAN Actual results: Failed to create sandbox Expected results: It should upgrade with MULTUS
I believe that the customer may be experiencing this bug: To work around it, you may apply the RBAC rules manually, create a file called `wb-rbac.yml` with these contents: https://bugzilla.redhat.com/show_bug.cgi?id=1990932 ``` --- kind: RoleBinding apiVersion: rbac.authorization.k8s.io/v1 metadata: name: multus-whereabouts namespace: openshift-multus roleRef: apiGroup: rbac.authorization.k8s.io kind: Role name: whereabouts-cni subjects: - kind: ServiceAccount name: multus namespace: openshift-multus --- apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata: namespace: openshift-multus name: whereabouts-cni rules: - apiGroups: - coordination.k8s.io resources: - leases verbs: - '*' ``` And then `oc create -f wb-rbac.yml` -- this should give the Whereabouts IPAM CNI plugin access to the lease CRs. You may also wish to restart the pods referencing net-attach-defs with Whereabouts IPAM, otherwise, it should clear on the next crash loop.
Of note, the fix for this was merged in https://github.com/openshift/cluster-network-operator/pull/1186/files for 4.7.z, however, I'm unsure of which z-stream this appears in.
Looks like it was in errata for 4.7.34. Customer is updating to 4.7.33 so the need to go up one more.
regarding my comment: > You may also wish to restart the pods referencing net-attach-defs with Whereabouts IPAM, otherwise, it should clear on the next crash loop. I mean the pods that are the workload pods, NOT the pods in the `openshift-multus` namespace. That is, the customer pods, the pods that have the "k8s.v1.cni.cncf.io/networks" annotation. This is NOT required, but, would speed up the process instead of waiting for the crashloopbackoff timeout. So, completely optional.
It's my current diagnosis that this was a dupe of 1990932. *** This bug has been marked as a duplicate of bug 1990932 ***