Bug 1768835 - Log Error - windows kubelet report error unable to read config path "c:\\k\\etc\\kubernetes\\manifests"
Summary: Log Error - windows kubelet report error unable to read config path "c:\\k\\e...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Windows Containers
Version: 4.3.0
Hardware: Unspecified
OS: Unspecified
unspecified
low
Target Milestone: ---
: 4.4.0
Assignee: ravig
QA Contact: gaoshang
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-11-05 11:00 UTC by gaoshang
Modified: 2020-05-04 11:15 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-05-04 11:14:43 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift windows-machine-config-bootstrapper pull 105 0 None closed Bug 1768835: Fix pod manifest directory creation 2020-05-05 12:50:08 UTC
Red Hat Product Errata RHBA-2020:0581 0 None None None 2020-05-04 11:15:11 UTC

Description gaoshang 2019-11-05 11:00:53 UTC
Description of problem:

After bootstrapped a windows node by wmcb.exe, windows kubelet continuously report error: unable to read config path "c:\\k\\etc\\kubernetes\\manifests", after adding required folder this error disappears.

PS C:\k> Get-Content .\kubelet.log -tail 1 -wait
...
E1105 02:39:44.792917    3988 file.go:104] Unable to read config path "c:\\k\\etc\\kubernetes\\manifests": path does not exist, ignoring
E1105 02:39:44.792917    3988 file.go:104] Unable to read config path "c:\\k\\etc\\kubernetes\\manifests": path does not exist, ignoring
E1105 02:39:44.792917    3988 file.go:104] Unable to read config path "c:\\k\\etc\\kubernetes\\manifests": path does not exist, ignoring
...

Version-Release number of selected component (if applicable):

OCP version: 4.2 GA
# ./openshift-install version
./openshift-install v4.2.0
built from commit 90ccb37ac1f85ae811c50a29f9bb7e779c5045fb
release image quay.io/openshift-release-dev/ocp-release@sha256:c5337afd85b94c93ec513f21c8545e3f9e36a227f55d41bc1dfb8fcc3f2be129

windows-machine-config-operator version:
# git tag
0.1

windows instance:
MicrosoftWindowsServer:WindowsServer:2019-Datacenter-with-Containers:latest

How reproducible:
Always

Steps to Reproduce:
1. Install OCP 4.2 GA in Azure
2. Launch and configure windows instance
3. Bootstrap windows node via wmcb
> .\wmcb.exe initialize-kubelet --ignition-file worker.ign --kubelet-path kubelet.exe
{"level":"info","ts":1572947207.8846965,"logger":"wmcb","msg":"Bootstrapping completed successfully"}
4. Check log file in C:\k\kubelet.log

Actual results:

windows kubelet continuously report error: unable to read config path "c:\\k\\etc\\kubernetes\\manifests"

PS C:\k> Get-Content .\kubelet.log -tail 1 -wait
...
E1105 02:39:44.792917    3988 file.go:104] Unable to read config path "c:\\k\\etc\\kubernetes\\manifests": path does not exist, ignoring
E1105 02:39:44.792917    3988 file.go:104] Unable to read config path "c:\\k\\etc\\kubernetes\\manifests": path does not exist, ignoring
E1105 02:39:44.792917    3988 file.go:104] Unable to read config path "c:\\k\\etc\\kubernetes\\manifests": path does not exist, ignoring


Expected results:

No such error reported.


Additional info:

Comment 1 ravig 2019-11-05 14:11:18 UTC
What's the output of `oc get nodes`? Are you seeing the node to be in not ready state? This is a known error in the kubelet when it is starting up but the manifests field is not a mandatory one.

Comment 2 gaoshang 2019-11-06 03:14:29 UTC
(In reply to ravig from comment #1)
> What's the output of `oc get nodes`? Are you seeing the node to be in not
> ready state? This is a known error in the kubelet when it is starting up but
> the manifests field is not a mandatory one.

Yes, windows node is in ready state and not affected. It's a low level log message bug, just want to make sure it is tracked somewhere.

# oc get nodes
NAME                                          STATUS     ROLES    AGE    VERSION
sgao-cluster-win-bg7pz-master-0               Ready      master   27h    v1.14.6+c07e432da
sgao-cluster-win-bg7pz-master-1               Ready      master   27h    v1.14.6+c07e432da
sgao-cluster-win-bg7pz-master-2               Ready      master   27h    v1.14.6+c07e432da
sgao-cluster-win-bg7pz-worker-eastus1-4l29r   Ready      worker   27h    v1.14.6+c07e432da
sgao-cluster-win-bg7pz-worker-eastus2-wvk59   Ready      worker   27h    v1.14.6+c07e432da
sgao-cluster-win-bg7pz-worker-eastus3-k2xqp   Ready      worker   27h    v1.14.6+c07e432da
winnode                                       Ready      <none>   151m   v1.14.0

Comment 3 gaoshang 2019-12-13 12:48:22 UTC
This bug has been verified and passed on OCP 4.3.0-0.nightly-2019-12-12-021332

Version-Release number of selected component (if applicable):
# oc get clusterversion
NAME VERSION AVAILABLE PROGRESSING SINCE STATUS
version 4.3.0-0.nightly-2019-12-12-021332 True False 5h16m Cluster version is 4.3.0-0.nightly-2019-12-12-021332
windows-machine-config-operator repo:
# git show
commit c040adf75fd8ef7f2656f58f0c8b75b014212104

Steps to Reproduce:
1, Install OCP 4.3 on azure and configure OVNKubernetes network
2, Install windows instance
3, Bootstrap windows node with wsu ansible
4, RDP into the windows node, check log file in C:\k\kubelet.log, error not exist anymore, check folder C:\k\etc\kubernetes\manifests exist

PS C:\k> Get-Content .\kubelet.log -Tail 0 -Wait
...

PS C:\k\etc\kubernetes\manifests> pwd

Path

C:\k\etc\kubernetes\manifests

Comment 5 errata-xmlrpc 2020-05-04 11:14:43 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:0581


Note You need to log in before you can comment on or make changes to this bug.