Bug 1690754 - Enabling unsafe sysctls in kubeletConfig is not taking affect
Summary: Enabling unsafe sysctls in kubeletConfig is not taking affect
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Node
Version: 4.2.0
Hardware: All
OS: Linux
Target Milestone: ---
: 4.2.0
Assignee: Ryan Phillips
QA Contact: Sunil Choudhary
Depends On:
TreeView+ depends on / blocked
Reported: 2019-03-20 07:49 UTC by Sunil Choudhary
Modified: 2019-10-16 06:28 UTC (History)
9 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Last Closed: 2019-10-16 06:27:58 UTC
Target Upstream Version:

Attachments (Terms of Use)

System ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2019:2922 None None None 2019-10-16 06:28:10 UTC

Description Sunil Choudhary 2019-03-20 07:49:27 UTC
Description of problem:

Not able to allow unsafe sysctl on node via kubeletConfig.

Version-Release number of selected component (if applicable):
$ oc version --short
Client Version: v4.0.22
Server Version: v1.12.4+befe71b

$ oc get clusterversion
NAME      VERSION                             AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.0.0-0.nightly-2019-03-19-004004   True        False         20h     Cluster version is 4.0.0-0.nightly-2019-03-19-004004

How reproducible: Always

Steps to Reproduce:

Following doc https://github.com/openshift/openshift-docs/blob/enterprise-4.0/modules/create-a-kubeletconfig-crd-to-edit-kubelet-parameters.adoc

$ cat set-sysctl-worker.yaml 
apiVersion: machineconfiguration.openshift.io/v1
kind: KubeletConfig
  name: set-sysctl-worker
  machineConfigSelector: 01-worker-kubelet
      - "kernel.msg*,net.ipv4.route.min_pmtu"

$ oc apply -f set-sysctl-worker.yaml

$ oc get kubeletconfig
NAME                AGE
set-sysctl-worker   21m

Actual results:
Checking on node, unsafe sysctl argument is not set nor machineconfig created for custom kubelet.

$ oc get machineconfig
NAME                                                        GENERATEDBYCONTROLLER       IGNITIONVERSION   CREATED
00-master                                                   4.0.22-201903181722-dirty   2.2.0             20h
00-master-ssh                                               4.0.22-201903181722-dirty   2.2.0             20h
00-worker                                                   4.0.22-201903181722-dirty   2.2.0             20h
00-worker-ssh                                               4.0.22-201903181722-dirty   2.2.0             20h
01-master-container-runtime                                 4.0.22-201903181722-dirty   2.2.0             20h
01-master-kubelet                                           4.0.22-201903181722-dirty   2.2.0             20h
01-worker-container-runtime                                 4.0.22-201903181722-dirty   2.2.0             20h
01-worker-kubelet                                           4.0.22-201903181722-dirty   2.2.0             20h
99-master-36c9fa32-4a34-11e9-9943-02c9a2a649ac-registries   4.0.22-201903181722-dirty   2.2.0             20h
99-worker-36d2f096-4a34-11e9-9943-02c9a2a649ac-registries   4.0.22-201903181722-dirty   2.2.0             20h
master-bb7e23c9982c911eb7be482741d983fd                     4.0.22-201903181722-dirty   2.2.0             20h
worker-65e80488f420def832f1903ef9efc0c4                     4.0.22-201903181722-dirty   2.2.0             20h

cat /etc/kubernetes/kubelet.conf
kind: KubeletConfiguration
apiVersion: kubelet.config.k8s.io/v1beta1
cgroupDriver: systemd
clusterDomain: cluster.local
maxPods: 250
rotateCertificates: true
runtimeRequestTimeout: 10m
serializeImagePulls: false
staticPodPath: /etc/kubernetes/manifests
  cpu: 500m
  memory: 500Mi
  RotateKubeletServerCertificate: true
serverTLSBootstrap: true

Expected results:

Should be able to enable unsafe sysctl.

Additional info:

Comment 1 weiwei jiang 2019-03-20 09:44:48 UTC
FYI, I did not see any things about allowed-unsafe-sysctls in https://github.com/kubernetes/kubelet/blob/release-1.12/config/v1beta1/types.go.

But you can give a try with "experimental-allowed-unsafe-sysctls" according to https://v1-12.docs.kubernetes.io/docs/reference/command-line-tools-reference/kubelet/ since the command line still support this.

Comment 2 MinLi 2019-03-20 10:05:17 UTC
I think current problem is dynamically kubelet-config don't take effect.
I try for kubelet config item "maxPods", reconfig not work.

Comment 5 Ryan Phillips 2019-08-02 15:57:45 UTC
Pick to origin: https://github.com/openshift/origin/pull/23538

Comment 6 Seth Jennings 2019-08-02 16:44:16 UTC
Just ran into a major issue here in that the vendored kube for the MCO, which will need this change to the kubelet API, is pointing to upstream and not to our fork.  Thus we are currently not able to get this change into the MCO.  Discussions on next steps next week (8/5).

Comment 7 Seth Jennings 2019-08-06 18:54:04 UTC

Comment 8 Ryan Phillips 2019-08-07 14:05:02 UTC
PR's have been merged. Code will be in the next 4.2 release.

Comment 11 MinLi 2019-08-09 02:29:30 UTC
Btw, "worker" machineconfigpool should be labeled first before create kubeletconfig
# oc label machineconfigpool worker custom-kubelet=sysctl

Comment 12 errata-xmlrpc 2019-10-16 06:27:58 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.