Bug 1814433 - Kubelet throws Failed to list Resource error
Summary: Kubelet throws Failed to list Resource error
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Windows Containers
Version: 4.5
Hardware: Unspecified
OS: Unspecified
unspecified
low
Target Milestone: ---
: 4.6.0
Assignee: vhire@redhat.com
QA Contact: gaoshang
URL:
Whiteboard:
: 1814441 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-03-17 20:43 UTC by vhire@redhat.com
Modified: 2020-10-27 15:57 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-10-27 15:57:28 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift windows-machine-config-bootstrapper pull 216 0 None closed Bug 1814433: [wmcb] Generate Windows specific kubelet config 2021-01-26 16:21:17 UTC
Red Hat Product Errata RHBA-2020:4196 0 None None None 2020-10-27 15:57:31 UTC

Description vhire@redhat.com 2020-03-17 20:43:42 UTC
Description of problem:
After bootstrapping the kubelet, the kubelet logs show errors

Version-Release number of selected component (if applicable):


How reproducible:
Always

Steps to Reproduce:
1.Create Windows node
2.Run WSU
3.Open C:\k\kubelet.log

Actual results:
E0316 20:46:22.628679    1456 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: pods is forbidden: User "system:anonymous" cannot list resource "pods" in API group "" at the cluster scope
E0316 20:46:22.654198    1456 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/kubelet.go:459: Failed to list *v1.Node: nodes "ip-<>.ec2.internal" is forbidden: User "system:anonymous" cannot list resource "nodes" in API group "" at the cluster scope
E0316 20:46:22.654198    1456 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/kubelet.go:450: Failed to list *v1.Service: services is forbidden: User "system:anonymous" cannot list resource "services" in API group "" at the cluster scope

Expected results:
No Error after certificate signing request is issued in kubelet


Additional info:

Comment 1 Aravindh Puthiyaparambil 2020-05-18 15:28:16 UTC
This will be addressed in https://issues.redhat.com/browse/WINC-134

Comment 2 Aravindh Puthiyaparambil 2020-06-29 19:35:58 UTC
*** Bug 1814441 has been marked as a duplicate of this bug. ***

Comment 3 Aravindh Puthiyaparambil 2020-06-29 19:37:07 UTC
@gaoshang please confirm that the fix for this bug also fixes 1814433.

Comment 4 Aravindh Puthiyaparambil 2020-06-29 19:37:50 UTC
Sorry I meant https://bugzilla.redhat.com/show_bug.cgi?id=1814441

Comment 7 gaoshang 2020-07-11 17:49:56 UTC
Bug 1814441 has been fixed, "unable to read existing bootstrap client config" do not exist in kubelet.log anymore.

PS C:\k\log> Get-Content .\kubelet.log | Select-String -Pattern "^E.*unable to read existing bootstrap client config"
PS C:\k\log>

Comment 8 gaoshang 2020-07-11 18:03:25 UTC
About this bug, after certificate signing request is issued in kubelet, error do not exist anymore, only exist before it, thanks.

PS C:\k\log> Get-Content .\kubelet.log | Select-String -Pattern "^E.*Failed to list"
--- before certificate signing request is issued in kubelet ---
E0711 15:20:42.931703    3500 reflector.go:178] k8s.io/kubernetes/pkg/kubelet/kubelet.go:517: Failed to list *v1.Service: services is forbidden: User "system:anonymous" cannot list resource "services" in API    
group "" at the cluster scope
"csidrivers" in API group "storage.k8s.io" at the cluster scope
E0711 15:20:43.405575    3500 reflector.go:178] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User "system:anonymous" cannot list      
resource "runtimeclasses" in API group "node.k8s.io" at the cluster scope
E0711 15:20:44.047141    3500 reflector.go:178] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: pods is forbidden: User "system:anonymous" cannot list resource "pods" in API group  
"" at the cluster scope
E0711 15:20:44.088110    3500 reflector.go:178] k8s.io/kubernetes/pkg/kubelet/kubelet.go:526: Failed to list *v1.Node: nodes "ip-10-0-55-138.us-east-2.compute.internal" is forbidden: User "system:anonymous"     
cannot list resource "nodes" in API group "" at the cluster scope
E0711 15:20:44.115391    3500 reflector.go:178] k8s.io/kubernetes/pkg/kubelet/kubelet.go:517: Failed to list *v1.Service: services is forbidden: User "system:anonymous" cannot list resource "services" in API    
group "" at the cluster scope
E0711 15:20:44.807702    3500 reflector.go:178] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User "system:anonymous" cannot list resource      
"csidrivers" in API group "storage.k8s.io" at the cluster scope
E0711 15:20:44.932150    3500 reflector.go:178] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User "system:anonymous" cannot list      
resource "runtimeclasses" in API group "node.k8s.io" at the cluster scope
E0711 15:20:46.015830    3500 reflector.go:178] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: pods is forbidden: User "system:anonymous" cannot list resource "pods" in API group  
"" at the cluster scope
E0711 15:20:46.165014    3500 reflector.go:178] k8s.io/kubernetes/pkg/kubelet/kubelet.go:517: Failed to list *v1.Service: services is forbidden: User "system:anonymous" cannot list resource "services" in API    
group "" at the cluster scope
E0711 15:20:47.048955    3500 reflector.go:178] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User "system:anonymous" cannot list      
resource "runtimeclasses" in API group "node.k8s.io" at the cluster scope
E0711 15:20:47.124032    3500 reflector.go:178] k8s.io/kubernetes/pkg/kubelet/kubelet.go:526: Failed to list *v1.Node: nodes "ip-10-0-55-138.us-east-2.compute.internal" is forbidden: User "system:anonymous"     
cannot list resource "nodes" in API group "" at the cluster scope
E0711 15:20:47.953861    3500 reflector.go:178] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User "system:anonymous" cannot list resource      
"csidrivers" in API group "storage.k8s.io" at the cluster scope
E0711 15:20:49.370222    3500 reflector.go:178] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: pods is forbidden: User "system:anonymous" cannot list resource "pods" in API group  
"" at the cluster scope
E0711 15:20:49.979704    3500 reflector.go:178] k8s.io/kubernetes/pkg/kubelet/kubelet.go:517: Failed to list *v1.Service: services is forbidden: User "system:anonymous" cannot list resource "services" in API    
group "" at the cluster scope
E0711 15:20:52.276179    3500 reflector.go:178] k8s.io/kubernetes/pkg/kubelet/kubelet.go:526: Failed to list *v1.Node: nodes "ip-10-0-55-138.us-east-2.compute.internal" is forbidden: User "system:anonymous"     
cannot list resource "nodes" in API group "" at the cluster scope
E0711 15:20:53.385430    3500 reflector.go:178] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User "system:anonymous" cannot list      
resource "runtimeclasses" in API group "node.k8s.io" at the cluster scope
E0711 15:20:53.969072    3500 reflector.go:178] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User "system:anonymous" cannot list resource      
"csidrivers" in API group "storage.k8s.io" at the cluster scope
E0711 15:20:57.045320    3500 reflector.go:178] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: pods is forbidden: User "system:anonymous" cannot list resource "pods" in API group  
"" at the cluster scope
E0711 15:20:58.220458    3500 reflector.go:178] k8s.io/kubernetes/pkg/kubelet/kubelet.go:517: Failed to list *v1.Service: services is forbidden: User "system:anonymous" cannot list resource "services" in API    
group "" at the cluster scope
E0711 15:20:59.794952    3500 reflector.go:178] k8s.io/kubernetes/pkg/kubelet/kubelet.go:526: Failed to list *v1.Node: nodes "ip-10-0-55-138.us-east-2.compute.internal" is forbidden: User "system:anonymous"     
cannot list resource "nodes" in API group "" at the cluster scope
E0711 15:21:04.923838    3500 reflector.go:178] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User "system:anonymous" cannot list      
resource "runtimeclasses" in API group "node.k8s.io" at the cluster scope
E0711 15:21:06.710091    3500 reflector.go:178] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User "system:anonymous" cannot list resource      
"csidrivers" in API group "storage.k8s.io" at the cluster scope
E0711 15:21:17.547702    3500 reflector.go:178] k8s.io/kubernetes/pkg/kubelet/kubelet.go:517: Failed to list *v1.Service: services is forbidden: User "system:anonymous" cannot list resource "services" in API    
group "" at the cluster scope
E0711 15:21:18.151443    3500 reflector.go:178] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User "system:anonymous" cannot list      
resource "runtimeclasses" in API group "node.k8s.io" at the cluster scope
E0711 15:21:19.158667    3500 reflector.go:178] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: pods is forbidden: User "system:anonymous" cannot list resource "pods" in API group  
"" at the cluster scope
--- after certificate signing request is issued in kubelet ---

Comment 10 errata-xmlrpc 2020-10-27 15:57:28 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.6 GA Images), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:4196


Note You need to log in before you can comment on or make changes to this bug.