Bug 2085095

Summary: error adding pod to CNI network "multis-cni-network"
Product: OpenShift Container Platform Reporter: Stephen Benjamin <stbenjam>
Component: NetworkingAssignee: Douglas Smith <dosmith>
Networking sub component: multus QA Contact: Weibin Liang <weliang>
Status: CLOSED DUPLICATE Docs Contact:
Severity: unspecified    
Priority: unspecified    
Version: 4.11   
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-05-12 19:25:51 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Stephen Benjamin 2022-05-12 19:15:19 UTC 
`pods should successfully create sandboxes by adding pod to network` is one of the top flaking tests for both SDN and OVN -- about 20%.

Example run:

https://prow.ci.openshift.org/view/gcs/origin-ci-test/logs/periodic-ci-openshift-release-master-ci-4.11-e2e-aws-ovn-upgrade/1524736033537658880

It fails with the following error message (no additional details after the final colon):

ns/openshift-kube-controller-manager pod/revision-pruner-8-ip-10-0-184-220.us-west-2.compute.internal node/ip-10-0-184-220.us-west-2.compute.internal - 451.57 seconds after deletion - reason/FailedCreatePodSandBox Failed to create pod sandbox: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_revision-pruner-8-ip-10-0-184-220.us-west-2.compute.internal_openshift-kube-controller-manager_a747d716-e0ba-4c59-b6ed-600ba11b17d7_0(fea8ca1cc99cb8688c48ef5156dd23e656df1c02233c1538e68c4ff553aeed36): error adding pod openshift-kube-controller-manager_revision-pruner-8-ip-10-0-184-220.us-west-2.compute.internal to CNI network "multus-cni-network": plugin type="multus" name="multus-cni-network" failed (add): 

Some more context in the Loki logs:

https://grafana-loki.ci.openshift.org/explore?orgId=1&left=%5B%221652366400000%22,%221652366520000%22,%22Grafana%20Cloud%22,%7B%22expr%22:%22%7Binvoker%3D%5C%22openshift-internal-ci%2Fperiodic-ci-openshift-release-master-ci-4.11-e2e-aws-ovn-upgrade%2F1524736033537658880%5C%22%7D%20%7C%20unpack%20%7C~%20%5C%22revision-pruner-8-ip-10-0-184-220.us-west-2.compute.internal%5C%22%22,%22refId%22:%22A%22,%22instant%22:false,%22range%22:true,%22maxLines%22:null%7D%5D