Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1768835

Summary: Log Error - windows kubelet report error unable to read config path "c:\\k\\etc\\kubernetes\\manifests"
Product: OpenShift Container Platform Reporter: gaoshang <sgao>
Component: Windows ContainersAssignee: ravig <rgudimet>
Status: CLOSED ERRATA QA Contact: gaoshang <sgao>
Severity: low Docs Contact:
Priority: unspecified    
Version: 4.3.0CC: aos-bugs, aravindh, gmarkley, rgudimet
Target Milestone: ---   
Target Release: 4.4.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-05-04 11:14:43 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description gaoshang 2019-11-05 11:00:53 UTC
Description of problem:

After bootstrapped a windows node by wmcb.exe, windows kubelet continuously report error: unable to read config path "c:\\k\\etc\\kubernetes\\manifests", after adding required folder this error disappears.

PS C:\k> Get-Content .\kubelet.log -tail 1 -wait
...
E1105 02:39:44.792917    3988 file.go:104] Unable to read config path "c:\\k\\etc\\kubernetes\\manifests": path does not exist, ignoring
E1105 02:39:44.792917    3988 file.go:104] Unable to read config path "c:\\k\\etc\\kubernetes\\manifests": path does not exist, ignoring
E1105 02:39:44.792917    3988 file.go:104] Unable to read config path "c:\\k\\etc\\kubernetes\\manifests": path does not exist, ignoring
...

Version-Release number of selected component (if applicable):

OCP version: 4.2 GA
# ./openshift-install version
./openshift-install v4.2.0
built from commit 90ccb37ac1f85ae811c50a29f9bb7e779c5045fb
release image quay.io/openshift-release-dev/ocp-release@sha256:c5337afd85b94c93ec513f21c8545e3f9e36a227f55d41bc1dfb8fcc3f2be129

windows-machine-config-operator version:
# git tag
0.1

windows instance:
MicrosoftWindowsServer:WindowsServer:2019-Datacenter-with-Containers:latest

How reproducible:
Always

Steps to Reproduce:
1. Install OCP 4.2 GA in Azure
2. Launch and configure windows instance
3. Bootstrap windows node via wmcb
> .\wmcb.exe initialize-kubelet --ignition-file worker.ign --kubelet-path kubelet.exe
{"level":"info","ts":1572947207.8846965,"logger":"wmcb","msg":"Bootstrapping completed successfully"}
4. Check log file in C:\k\kubelet.log

Actual results:

windows kubelet continuously report error: unable to read config path "c:\\k\\etc\\kubernetes\\manifests"

PS C:\k> Get-Content .\kubelet.log -tail 1 -wait
...
E1105 02:39:44.792917    3988 file.go:104] Unable to read config path "c:\\k\\etc\\kubernetes\\manifests": path does not exist, ignoring
E1105 02:39:44.792917    3988 file.go:104] Unable to read config path "c:\\k\\etc\\kubernetes\\manifests": path does not exist, ignoring
E1105 02:39:44.792917    3988 file.go:104] Unable to read config path "c:\\k\\etc\\kubernetes\\manifests": path does not exist, ignoring


Expected results:

No such error reported.


Additional info:

Comment 1 ravig 2019-11-05 14:11:18 UTC
What's the output of `oc get nodes`? Are you seeing the node to be in not ready state? This is a known error in the kubelet when it is starting up but the manifests field is not a mandatory one.

Comment 2 gaoshang 2019-11-06 03:14:29 UTC
(In reply to ravig from comment #1)
> What's the output of `oc get nodes`? Are you seeing the node to be in not
> ready state? This is a known error in the kubelet when it is starting up but
> the manifests field is not a mandatory one.

Yes, windows node is in ready state and not affected. It's a low level log message bug, just want to make sure it is tracked somewhere.

# oc get nodes
NAME                                          STATUS     ROLES    AGE    VERSION
sgao-cluster-win-bg7pz-master-0               Ready      master   27h    v1.14.6+c07e432da
sgao-cluster-win-bg7pz-master-1               Ready      master   27h    v1.14.6+c07e432da
sgao-cluster-win-bg7pz-master-2               Ready      master   27h    v1.14.6+c07e432da
sgao-cluster-win-bg7pz-worker-eastus1-4l29r   Ready      worker   27h    v1.14.6+c07e432da
sgao-cluster-win-bg7pz-worker-eastus2-wvk59   Ready      worker   27h    v1.14.6+c07e432da
sgao-cluster-win-bg7pz-worker-eastus3-k2xqp   Ready      worker   27h    v1.14.6+c07e432da
winnode                                       Ready      <none>   151m   v1.14.0

Comment 3 gaoshang 2019-12-13 12:48:22 UTC
This bug has been verified and passed on OCP 4.3.0-0.nightly-2019-12-12-021332

Version-Release number of selected component (if applicable):
# oc get clusterversion
NAME VERSION AVAILABLE PROGRESSING SINCE STATUS
version 4.3.0-0.nightly-2019-12-12-021332 True False 5h16m Cluster version is 4.3.0-0.nightly-2019-12-12-021332
windows-machine-config-operator repo:
# git show
commit c040adf75fd8ef7f2656f58f0c8b75b014212104

Steps to Reproduce:
1, Install OCP 4.3 on azure and configure OVNKubernetes network
2, Install windows instance
3, Bootstrap windows node with wsu ansible
4, RDP into the windows node, check log file in C:\k\kubelet.log, error not exist anymore, check folder C:\k\etc\kubernetes\manifests exist

PS C:\k> Get-Content .\kubelet.log -Tail 0 -Wait
...

PS C:\k\etc\kubernetes\manifests> pwd

Path

C:\k\etc\kubernetes\manifests

Comment 5 errata-xmlrpc 2020-05-04 11:14:43 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:0581