Bug 1781242

Summary: 'failed to run CNI IPAM ADD: netplugin failed but error parsing its diagnostic message \"\": unexpected end of JSON input' when starting a pod
Product: OpenShift Container Platform Reporter: Vadim Rutkovsky <vrutkovs>
Component: NetworkingAssignee: Feng Pan <fpan>
Networking sub component: multus QA Contact: Weibin Liang <weliang>
Status: CLOSED DUPLICATE Docs Contact:
Severity: unspecified    
Priority: unspecified CC: bbennett, nagrawal
Version: 4.4   
Target Milestone: ---   
Target Release: 4.4.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-12-10 15:00:17 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Vadim Rutkovsky 2019-12-09 15:43:34 UTC 
Description of problem:

https://prow.svc.ci.openshift.org/view/gcs/origin-ci-test/pr-logs/pull/openshift_machine-config-operator/1314/pull-ci-openshift-machine-config-operator-master-e2e-aws-scaleup-rhel7/742

kube-state-metrics pod failed to start. events (https://storage.googleapis.com/origin-ci-test/pr-logs/pull/openshift_machine-config-operator/1314/pull-ci-openshift-machine-config-operator-master-e2e-aws-scaleup-rhel7/742/artifacts/e2e-aws-scaleup-rhel7/events.json) has the error message:

Failed create pod sandbox: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kube-state-metrics-665b78b6b4-wjdsl_openshift-monitoring_5b8a01c4-9780-4e02-b5fc-15cbc3cb1028_0(e6932fd85f8c138a263bb32cc0d28339c28572fd31687c438ebed3ae4114e99a): Multus: error adding pod to network \"openshift-sdn\": delegateAdd: error invoking DelegateAdd - \"openshift-sdn\": error in getting result from AddNetwork: CNI request failed with status 400: 'failed to run IPAM for e6932fd85f8c138a263bb32cc0d28339c28572fd31687c438ebed3ae4114e99a: failed to run CNI IPAM ADD: netplugin failed but error parsing its diagnostic message \"\": unexpected end of JSON input\n'

Version-Release number of selected component (if applicable):
Various 4.4 runs on AWS and Azure:
https://prow.svc.ci.openshift.org/view/gcs/origin-ci-test/logs/release-openshift-origin-installer-e2e-aws-upgrade/12105
https://prow.svc.ci.openshift.org/view/gcs/origin-ci-test/logs/release-openshift-origin-installer-e2e-aws-serial-4.4/12
https://prow.svc.ci.openshift.org/view/gcs/origin-ci-test/logs/release-openshift-ocp-installer-e2e-azure-serial-4.4/151

Doesn't seem to affect 4.3
Comment 1 Vadim Rutkovsky 2019-12-10 15:00:17 UTC 
Dec 09 14:47:59 ip-10-0-155-39 systemd-coredump[3758]: Process 3752 (host-local) of user 0 dumped core.
                                                       
  Stack trace of thread 3752:
  #0  0x000000000044e838 n/a (/host/var/opt/cni/bin/host-local)
  #1  0x000000000044ebcd n/a (/host/var/opt/cni/bin/host-local)
  #2  0x0000000000428a0e n/a (/host/var/opt/cni/bin/host-local)
  #3  0x00000000004287ca n/a (/host/var/opt/cni/bin/host-local)
  #4  0x00000000004388b9 n/a (/host/var/opt/cni/bin/host-local)
  #5  0x0000000000451e00 n/a (/host/var/opt/cni/bin/host-local)

*** This bug has been marked as a duplicate of bug 1780387 ***