Bug 1969268

Summary:	/etc/kubelet/kubelet.conf created in json format after automatic change in KubeletConfig
Product:	OpenShift Container Platform	Reporter:	Ravi Trivedi <travi>
Component:	Node	Assignee:	Qi Wang <qiwan>
Node sub component:	Kubelet	QA Contact:	MinLi <minmli>
Status:	CLOSED NOTABUG	Docs Contact:
Severity:	unspecified
Priority:	unspecified	CC:	aos-bugs, ddelcian, harpatil, rphillips, wking
Version:	4.7	Keywords:	ServiceDeliveryImpact
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2021-07-07 15:00:20 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Ravi Trivedi 2021-06-08 05:58:42 UTC

Description of problem:
There was a spike in load on etcd which caused etcd instances to be unavailable for some time. There was no etcd leader for a brief while. This has perhaps caused caused the KubeletConfig to trigger a change in /etc/kubelet/kubelet.conf to be populated with json content. This BZ is to investigate into this unexpected change.

Version-Release number of selected component (if applicable):
OCP 4.7.2

Actual results:
- /etc/kubelet/kubelet.conf on master nodes created with json content after automatic change in KubeletConfig

Expected results:
- /etc/kubelet/kubelet.conf to be reconciled in correct format 

Additional info:
- This is a GCP cluster on OSD.
- The unexpected change is seen only on the master nodes. 
- Around the time of change, etcd instances faced outage as well.

Comment 2 Qi Wang 2021-06-30 18:04:35 UTC

From the muster-gather logs, the "systemReserved":"cpu" changed from 500m to 1000m since there's a custom-kubelet kubeletconfig and it has been successfully rolled out at "2021-06-07T22:01:40Z". Not sure how the no etcd leader would affect the MCO. 
@harpatil Is it possible the changes caused by the auto node sizing?

Comment 3 Harshal Patil 2021-07-01 11:58:35 UTC

Auto node sizing is only available in 4.8+. Also, auto node sizing only deals with the systemd service file of the kubelet and has no role to play in actually rendering the kubelet.conf itself.

Comment 4 Qi Wang 2021-07-01 16:05:51 UTC

@Ravi Trivedi Is there a way to reproduce this issue?

Comment 7 Red Hat Bugzilla 2023-09-15 01:09:26 UTC

The needinfo request[s] on this closed bug have been removed as they have been unresolved for 500 days