Bug 1899717

Summary: CI test_pull_request_origin_extended_conformance_install-release-3.11: network plugin is not ready: cni config uninitialized
Product: OpenShift Container Platform Reporter: Russell Teague <rteague>
Component: NetworkingAssignee: Juan Luis de Sousa-Valadas <jdesousa>
Networking sub component: openshift-sdn QA Contact: zhaozhanqi <zzhao>
Status: CLOSED DUPLICATE Docs Contact:
Severity: medium    
Priority: high CC: aconstan, danw
Version: 3.11.0   
Target Milestone: ---   
Target Release: 4.7.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-11-30 20:07:45 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1870050    

Description Russell Teague 2020-11-19 20:20:05 UTC
Description of problem:
CI job test_pull_request_origin_extended_conformance_install-release-3.11 is failing continually.

https://prow.ci.openshift.org/job-history/origin-ci-test/pr-logs/directory/test_pull_request_origin_extended_conformance_install-release-3.11

Example:
https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/25578/test_pull_request_origin_extended_conformance_install-release-3.11/2470

During installation at the point where the node service is restarted, the node does not return to a Ready status.  The kubelet is failing to start with the error:

"message": "runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized", 
"reason": "KubeletNotReady", 
"status": "False", 
"type": "Ready"

Node logs show:
Nov 19 15:04:14 ip-172-18-7-176.ec2.internal origin-node[15185]: W1119 15:04:14.130564   15185 cni.go:172] Unable to update cni config: No networks found in /etc/cni/net.d
Nov 19 15:04:14 ip-172-18-7-176.ec2.internal origin-node[15185]: I1119 15:04:14.130782   15185 kubelet.go:2098] Container runtime status: Runtime Conditions: RuntimeReady=true reason: message:, NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
Nov 19 15:04:14 ip-172-18-7-176.ec2.internal origin-node[15185]: E1119 15:04:14.130823   15185 kubelet.go:2101] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized

NOTE:
The base AMI was recently updated when it was discovered that an old version of docker was installed.  There may be other package changes that occurred.

Comment 1 Dan Winship 2020-11-30 20:07:45 UTC

*** This bug has been marked as a duplicate of bug 1902067 ***