Bug 1814433

Summary:	Kubelet throws Failed to list Resource error
Product:	OpenShift Container Platform	Reporter:	vhire <vhire>
Component:	Windows Containers	Assignee:	vhire <vhire>
Status:	CLOSED ERRATA	QA Contact:	gaoshang <sgao>
Severity:	low	Docs Contact:
Priority:	unspecified
Version:	4.5	CC:	aos-bugs, aravindh, gmarkley, rgudimet
Target Milestone:	---
Target Release:	4.6.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2020-10-27 15:57:28 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description vhire@redhat.com 2020-03-17 20:43:42 UTC

Description of problem:
After bootstrapping the kubelet, the kubelet logs show errors

Version-Release number of selected component (if applicable):


How reproducible:
Always

Steps to Reproduce:
1.Create Windows node
2.Run WSU
3.Open C:\k\kubelet.log

Actual results:
E0316 20:46:22.628679    1456 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: pods is forbidden: User "system:anonymous" cannot list resource "pods" in API group "" at the cluster scope
E0316 20:46:22.654198    1456 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/kubelet.go:459: Failed to list *v1.Node: nodes "ip-<>.ec2.internal" is forbidden: User "system:anonymous" cannot list resource "nodes" in API group "" at the cluster scope
E0316 20:46:22.654198    1456 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/kubelet.go:450: Failed to list *v1.Service: services is forbidden: User "system:anonymous" cannot list resource "services" in API group "" at the cluster scope

Expected results:
No Error after certificate signing request is issued in kubelet


Additional info:

Comment 1 Aravindh Puthiyaparambil 2020-05-18 15:28:16 UTC

This will be addressed in https://issues.redhat.com/browse/WINC-134

Comment 2 Aravindh Puthiyaparambil 2020-06-29 19:35:58 UTC

*** Bug 1814441 has been marked as a duplicate of this bug. ***

Comment 3 Aravindh Puthiyaparambil 2020-06-29 19:37:07 UTC

@gaoshang please confirm that the fix for this bug also fixes 1814433.

Comment 4 Aravindh Puthiyaparambil 2020-06-29 19:37:50 UTC

Sorry I meant https://bugzilla.redhat.com/show_bug.cgi?id=1814441

Comment 7 gaoshang 2020-07-11 17:49:56 UTC

Bug 1814441 has been fixed, "unable to read existing bootstrap client config" do not exist in kubelet.log anymore.

PS C:\k\log> Get-Content .\kubelet.log | Select-String -Pattern "^E.*unable to read existing bootstrap client config"
PS C:\k\log>

Comment 8 gaoshang 2020-07-11 18:03:25 UTC

About this bug, after certificate signing request is issued in kubelet, error do not exist anymore, only exist before it, thanks.

PS C:\k\log> Get-Content .\kubelet.log | Select-String -Pattern "^E.*Failed to list"
--- before certificate signing request is issued in kubelet ---
E0711 15:20:42.931703    3500 reflector.go:178] k8s.io/kubernetes/pkg/kubelet/kubelet.go:517: Failed to list *v1.Service: services is forbidden: User "system:anonymous" cannot list resource "services" in API    
group "" at the cluster scope
"csidrivers" in API group "storage.k8s.io" at the cluster scope
E0711 15:20:43.405575    3500 reflector.go:178] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User "system:anonymous" cannot list      
resource "runtimeclasses" in API group "node.k8s.io" at the cluster scope
E0711 15:20:44.047141    3500 reflector.go:178] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: pods is forbidden: User "system:anonymous" cannot list resource "pods" in API group  
"" at the cluster scope
E0711 15:20:44.088110    3500 reflector.go:178] k8s.io/kubernetes/pkg/kubelet/kubelet.go:526: Failed to list *v1.Node: nodes "ip-10-0-55-138.us-east-2.compute.internal" is forbidden: User "system:anonymous"     
cannot list resource "nodes" in API group "" at the cluster scope
E0711 15:20:44.115391    3500 reflector.go:178] k8s.io/kubernetes/pkg/kubelet/kubelet.go:517: Failed to list *v1.Service: services is forbidden: User "system:anonymous" cannot list resource "services" in API    
group "" at the cluster scope
E0711 15:20:44.807702    3500 reflector.go:178] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User "system:anonymous" cannot list resource      
"csidrivers" in API group "storage.k8s.io" at the cluster scope
E0711 15:20:44.932150    3500 reflector.go:178] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User "system:anonymous" cannot list      
resource "runtimeclasses" in API group "node.k8s.io" at the cluster scope
E0711 15:20:46.015830    3500 reflector.go:178] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: pods is forbidden: User "system:anonymous" cannot list resource "pods" in API group  
"" at the cluster scope
E0711 15:20:46.165014    3500 reflector.go:178] k8s.io/kubernetes/pkg/kubelet/kubelet.go:517: Failed to list *v1.Service: services is forbidden: User "system:anonymous" cannot list resource "services" in API    
group "" at the cluster scope
E0711 15:20:47.048955    3500 reflector.go:178] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User "system:anonymous" cannot list      
resource "runtimeclasses" in API group "node.k8s.io" at the cluster scope
E0711 15:20:47.124032    3500 reflector.go:178] k8s.io/kubernetes/pkg/kubelet/kubelet.go:526: Failed to list *v1.Node: nodes "ip-10-0-55-138.us-east-2.compute.internal" is forbidden: User "system:anonymous"     
cannot list resource "nodes" in API group "" at the cluster scope
E0711 15:20:47.953861    3500 reflector.go:178] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User "system:anonymous" cannot list resource      
"csidrivers" in API group "storage.k8s.io" at the cluster scope
E0711 15:20:49.370222    3500 reflector.go:178] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: pods is forbidden: User "system:anonymous" cannot list resource "pods" in API group  
"" at the cluster scope
E0711 15:20:49.979704    3500 reflector.go:178] k8s.io/kubernetes/pkg/kubelet/kubelet.go:517: Failed to list *v1.Service: services is forbidden: User "system:anonymous" cannot list resource "services" in API    
group "" at the cluster scope
E0711 15:20:52.276179    3500 reflector.go:178] k8s.io/kubernetes/pkg/kubelet/kubelet.go:526: Failed to list *v1.Node: nodes "ip-10-0-55-138.us-east-2.compute.internal" is forbidden: User "system:anonymous"     
cannot list resource "nodes" in API group "" at the cluster scope
E0711 15:20:53.385430    3500 reflector.go:178] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User "system:anonymous" cannot list      
resource "runtimeclasses" in API group "node.k8s.io" at the cluster scope
E0711 15:20:53.969072    3500 reflector.go:178] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User "system:anonymous" cannot list resource      
"csidrivers" in API group "storage.k8s.io" at the cluster scope
E0711 15:20:57.045320    3500 reflector.go:178] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: pods is forbidden: User "system:anonymous" cannot list resource "pods" in API group  
"" at the cluster scope
E0711 15:20:58.220458    3500 reflector.go:178] k8s.io/kubernetes/pkg/kubelet/kubelet.go:517: Failed to list *v1.Service: services is forbidden: User "system:anonymous" cannot list resource "services" in API    
group "" at the cluster scope
E0711 15:20:59.794952    3500 reflector.go:178] k8s.io/kubernetes/pkg/kubelet/kubelet.go:526: Failed to list *v1.Node: nodes "ip-10-0-55-138.us-east-2.compute.internal" is forbidden: User "system:anonymous"     
cannot list resource "nodes" in API group "" at the cluster scope
E0711 15:21:04.923838    3500 reflector.go:178] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User "system:anonymous" cannot list      
resource "runtimeclasses" in API group "node.k8s.io" at the cluster scope
E0711 15:21:06.710091    3500 reflector.go:178] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User "system:anonymous" cannot list resource      
"csidrivers" in API group "storage.k8s.io" at the cluster scope
E0711 15:21:17.547702    3500 reflector.go:178] k8s.io/kubernetes/pkg/kubelet/kubelet.go:517: Failed to list *v1.Service: services is forbidden: User "system:anonymous" cannot list resource "services" in API    
group "" at the cluster scope
E0711 15:21:18.151443    3500 reflector.go:178] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User "system:anonymous" cannot list      
resource "runtimeclasses" in API group "node.k8s.io" at the cluster scope
E0711 15:21:19.158667    3500 reflector.go:178] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: pods is forbidden: User "system:anonymous" cannot list resource "pods" in API group  
"" at the cluster scope
--- after certificate signing request is issued in kubelet ---

Comment 10 errata-xmlrpc 2020-10-27 15:57:28 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.6 GA Images), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:4196