Description of problem: Kubernetes control-plane node initialization (kubeadm init) from fresh install won't work due to systemd kubelet service failing. The second way to experience the same issue is to upgrade kubernetes-node-12.5-2.fc30 to kubernetes-node-1.15.2-1.fc30 Version-Release number of selected component (if applicable): cat /etc/fedora-release Fedora release 30 (Thirty) uname -a Linux qubit 5.2.9-200.fc30.x86_64 #1 SMP Fri Aug 16 21:37:45 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux dnf list --installed | grep kube kubernetes.x86_64 1.15.2-1.fc30 kubernetes-client.x86_64 1.15.2-1.fc30 kubernetes-kubeadm.x86_64 1.15.2-1.fc30 kubernetes-master.x86_64 1.15.2-1.fc30 kubernetes-node.x86_64 1.15.2-1.fc30 How reproducible: Steps to Reproduce: 1. dnf install kubernetes-master kubernetes-node kubernetes-kubeadm kubernetes-client docker 2. systemctl enable kubelet 3. kubeadm init --pod-network-cidr=10.244.0.0/16 Note: docker and kubelet should have same cgroup driver systemd docker info | grep Cgroup Cgroup Driver: systemd Actual results: kubeadm init --pod-network-cidr=10.244.0.0/16 [init] Using Kubernetes version: v1.15.3 [preflight] Running pre-flight checks [WARNING SystemVerification]: this Docker version is not on the list of validated versions: 19.03.1. Latest validated version: 18.09 [preflight] Pulling images required for setting up a Kubernetes cluster [preflight] This might take a minute or two, depending on the speed of your internet connection [preflight] You can also perform this action in beforehand using 'kubeadm config images pull' [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env" [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml" [kubelet-start] Activating the kubelet service [certs] Using certificateDir folder "/etc/kubernetes/pki" [certs] Generating "front-proxy-ca" certificate and key [certs] Generating "front-proxy-client" certificate and key [certs] Generating "ca" certificate and key [certs] Generating "apiserver" certificate and key [certs] apiserver serving cert is signed for DNS names [qubit kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 192.168.43.88] [certs] Generating "apiserver-kubelet-client" certificate and key [certs] Generating "etcd/ca" certificate and key [certs] Generating "etcd/server" certificate and key [certs] etcd/server serving cert is signed for DNS names [qubit localhost] and IPs [192.168.43.88 127.0.0.1 ::1] [certs] Generating "etcd/peer" certificate and key [certs] etcd/peer serving cert is signed for DNS names [qubit localhost] and IPs [192.168.43.88 127.0.0.1 ::1] [certs] Generating "etcd/healthcheck-client" certificate and key [certs] Generating "apiserver-etcd-client" certificate and key [certs] Generating "sa" key and public key [kubeconfig] Using kubeconfig folder "/etc/kubernetes" [kubeconfig] Writing "admin.conf" kubeconfig file [kubeconfig] Writing "kubelet.conf" kubeconfig file [kubeconfig] Writing "controller-manager.conf" kubeconfig file [kubeconfig] Writing "scheduler.conf" kubeconfig file [control-plane] Using manifest folder "/etc/kubernetes/manifests" [control-plane] Creating static Pod manifest for "kube-apiserver" [control-plane] Creating static Pod manifest for "kube-controller-manager" [control-plane] Creating static Pod manifest for "kube-scheduler" [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests" [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s [kubelet-check] Initial timeout of 40s passed. [kubelet-check] It seems like the kubelet isn't running or healthy. [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused. [kubelet-check] It seems like the kubelet isn't running or healthy. [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused. [kubelet-check] It seems like the kubelet isn't running or healthy. [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused. [kubelet-check] It seems like the kubelet isn't running or healthy. [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused. [kubelet-check] It seems like the kubelet isn't running or healthy. [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused. Unfortunately, an error has occurred: timed out waiting for the condition This error is likely caused by: - The kubelet is not running - The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled) If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands: - 'systemctl status kubelet' - 'journalctl -xeu kubelet' Additionally, a control plane component may have crashed or exited when started by the container runtime. To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker. Here is one example how you may list all Kubernetes containers running in docker: - 'docker ps -a | grep kube | grep -v pause' Once you have found the failing container, you can inspect its logs with: - 'docker logs CONTAINERID' error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster Log output for kubelet: journalctl -u kubelet.service -f Sep 03 14:15:28 qubit systemd[1]: Started Kubernetes Kubelet Server. Sep 03 14:15:28 qubit kubelet[8101]: Flag --fail-swap-on has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 03 14:15:28 qubit kubelet[8101]: Flag --pod-manifest-path has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 03 14:15:28 qubit kubelet[8101]: Usage: Sep 03 14:15:28 qubit kubelet[8101]: kubelet [flags] Sep 03 14:15:28 qubit kubelet[8101]: Flags: Sep 03 14:15:28 qubit kubelet[8101]: --address 0.0.0.0 The IP address for the Kubelet to serve on (set to 0.0.0.0 for all IPv4 interfaces and `::` for all IPv6 interfaces) (default 0.0.0.0) (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.) Sep 03 14:15:28 qubit kubelet[8101]: --allowed-unsafe-sysctls strings Comma-separated whitelist of unsafe sysctls or unsafe sysctl patterns (ending in *). Use these at your own risk. Sysctls feature gate is enabled by default. Sep 03 14:15:28 qubit kubelet[8101]: --alsologtostderr log to standard error as well as files Sep 03 14:15:28 qubit kubelet[8101]: --anonymous-auth Enables anonymous requests to the Kubelet server. Requests that are not rejected by another authentication method are treated as anonymous requests. Anonymous requests have a username of system:anonymous, and a group name of system:unauthenticated. (default true) (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.) Sep 03 14:15:28 qubit kubelet[8101]: --application-metrics-count-limit int Max number of application metrics to store (per container) (default 100) (DEPRECATED: This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed.) Sep 03 14:15:28 qubit kubelet[8101]: --authentication-token-webhook Use the TokenReview API to determine authentication for bearer tokens. (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.) Sep 03 14:15:28 qubit kubelet[8101]: --authentication-token-webhook-cache-ttl duration The duration to cache responses from the webhook token authenticator. (default 2m0s) (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.) Sep 03 14:15:28 qubit kubelet[8101]: --authorization-mode string Authorization mode for Kubelet server. Valid options are AlwaysAllow or Webhook. Webhook mode uses the SubjectAccessReview API to determine authorization. (default "AlwaysAllow") (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.) Sep 03 14:15:28 qubit kubelet[8101]: --authorization-webhook-cache-authorized-ttl duration The duration to cache 'authorized' responses from the webhook authorizer. (default 5m0s) (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.) Sep 03 14:15:28 qubit kubelet[8101]: --authorization-webhook-cache-unauthorized-ttl duration The duration to cache 'unauthorized' responses from the webhook authorizer. (default 30s) (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.) Sep 03 14:15:28 qubit kubelet[8101]: --azure-container-registry-config string Path to the file containing Azure container registry configuration information. Sep 03 14:15:28 qubit kubelet[8101]: --boot-id-file string Comma-separated list of files to check for boot-id. Use the first one that exists. (default "/proc/sys/kernel/random/boot_id") (DEPRECATED: This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed.) Sep 03 14:15:28 qubit kubelet[8101]: --bootstrap-checkpoint-path string <Warning: Alpha feature> Path to the directory where the checkpoints are stored Sep 03 14:15:28 qubit kubelet[8101]: --bootstrap-kubeconfig string Path to a kubeconfig file that will be used to get client certificate for kubelet. If the file specified by --kubeconfig does not exist, the bootstrap kubeconfig is used to request a client certificate from the API server. On success, a kubeconfig file referencing the generated client certificate and key is written to the path specified by --kubeconfig. The client certificate and key file will be stored in the directory pointed by --cert-dir. Sep 03 14:15:28 qubit kubelet[8101]: --cert-dir string The directory where the TLS certs are located. If --tls-cert-file and --tls-private-key-file are provided, this flag will be ignored. (default "/var/lib/kubelet/pki") Sep 03 14:15:28 qubit kubelet[8101]: --cgroup-driver string Driver that the kubelet uses to manipulate cgroups on the host. Possible values: 'cgroupfs', 'systemd' (default "cgroupfs") (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.) Sep 03 14:15:28 qubit kubelet[8101]: --cgroup-root string Optional root cgroup to use for pods. This is handled by the container runtime on a best effort basis. Default: '', which means use the container runtime default. (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.) Sep 03 14:15:28 qubit kubelet[8101]: --cgroups-per-qos Enable creation of QoS cgroup hierarchy, if true top level QoS and pod cgroups are created. (default true) (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.) Sep 03 14:15:28 qubit kubelet[8101]: --chaos-chance float If > 0.0, introduce random client errors and latency. Intended for testing. Sep 03 14:15:28 qubit kubelet[8101]: --client-ca-file string If set, any request presenting a client certificate signed by one of the authorities in the client-ca-file is authenticated with an identity corresponding to the CommonName of the client certificate. (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.) Sep 03 14:15:28 qubit kubelet[8101]: --cloud-config string The path to the cloud provider configuration file. Empty string for no configuration file. Sep 03 14:15:28 qubit kubelet[8101]: --cloud-provider string The provider for cloud services. Specify empty string for running with no cloud provider. If set, the cloud provider determines the name of the node (consult cloud provider documentation to determine if and how the hostname is used). Sep 03 14:15:28 qubit kubelet[8101]: --cluster-dns strings Comma-separated list of DNS server IP address. This value is used for containers DNS server in case of Pods with "dnsPolicy=ClusterFirst". Note: all DNS servers appearing in the list MUST serve the same set of records otherwise name resolution within the cluster may not work correctly. There is no guarantee as to which DNS server may be contacted for name resolution. (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.) Sep 03 14:15:28 qubit kubelet[8101]: --cluster-domain string Domain for this cluster. If set, kubelet will configure all containers to search this domain in addition to the host's search domains (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.) Sep 03 14:15:28 qubit kubelet[8101]: --cni-bin-dir string <Warning: Alpha feature> A comma-separated list of full paths of directories in which to search for CNI plugin binaries. This docker-specific flag only works when container-runtime is set to docker. (default "/opt/cni/bin") Sep 03 14:15:28 qubit kubelet[8101]: --cni-conf-dir string <Warning: Alpha feature> The full path of the directory in which to search for CNI config files. This docker-specific flag only works when container-runtime is set to docker. (default "/etc/cni/net.d") Sep 03 14:15:28 qubit kubelet[8101]: --config string The Kubelet will load its initial configuration from this file. The path may be absolute or relative; relative paths start at the Kubelet's current working directory. Omit this flag to use the built-in default configuration values. Command-line flags override configuration from this file. Sep 03 14:15:28 qubit kubelet[8101]: --container-hints string location of the container hints file (default "/etc/cadvisor/container_hints.json") (DEPRECATED: This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed.) Sep 03 14:15:28 qubit kubelet[8101]: --container-log-max-files int32 <Warning: Beta feature> Set the maximum number of container log files that can be present for a container. The number must be >= 2. This flag can only be used with --container-runtime=remote. (default 5) (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.) Sep 03 14:15:28 qubit kubelet[8101]: --container-log-max-size string <Warning: Beta feature> Set the maximum size (e.g. 10Mi) of container log file before it is rotated. This flag can only be used with --container-runtime=remote. (default "10Mi") (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.) Sep 03 14:15:28 qubit kubelet[8101]: --container-runtime string The container runtime to use. Possible values: 'docker', 'remote', 'rkt (deprecated)'. (default "docker") Sep 03 14:15:28 qubit kubelet[8101]: --container-runtime-endpoint string [Experimental] The endpoint of remote runtime service. Currently unix socket endpoint is supported on Linux, while npipe and tcp endpoints are supported on windows. Examples:'unix:///var/run/dockershim.sock', 'npipe:////./pipe/dockershim' (default "unix:///var/run/dockershim.sock") Sep 03 14:15:28 qubit kubelet[8101]: --containerd string containerd endpoint (default "/run/containerd/containerd.sock") (DEPRECATED: This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed.) Sep 03 14:15:28 qubit kubelet[8101]: --containerized Running kubelet in a container. (DEPRECATED: This feature will be removed in a later release.) Sep 03 14:15:28 qubit kubelet[8101]: --contention-profiling Enable lock contention profiling, if profiling is enabled (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.) Sep 03 14:15:28 qubit kubelet[8101]: --cpu-cfs-quota Enable CPU CFS quota enforcement for containers that specify CPU limits (default true) (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.) Sep 03 14:15:28 qubit kubelet[8101]: --cpu-cfs-quota-period duration Sets CPU CFS quota period value, cpu.cfs_period_us, defaults to Linux Kernel default (default 100ms) (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.) Sep 03 14:15:28 qubit kubelet[8101]: --cpu-manager-policy string CPU Manager policy to use. Possible values: 'none', 'static'. Default: 'none' (default "none") (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.) Sep 03 14:15:28 qubit kubelet[8101]: --cpu-manager-reconcile-period NodeStatusUpdateFrequency <Warning: Alpha feature> CPU Manager reconciliation period. Examples: '10s', or '1m'. If not supplied, defaults to NodeStatusUpdateFrequency (default 10s) (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.) Sep 03 14:15:28 qubit kubelet[8101]: --docker string docker endpoint (default "unix:///var/run/docker.sock") (DEPRECATED: This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed.) Sep 03 14:15:28 qubit kubelet[8101]: --docker-endpoint string Use this for the docker endpoint to communicate with. This docker-specific flag only works when container-runtime is set to docker. (default "unix:///var/run/docker.sock") Sep 03 14:15:28 qubit kubelet[8101]: --docker-env-metadata-whitelist string a comma-separated list of environment variable keys that needs to be collected for docker containers (DEPRECATED: This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed.) Sep 03 14:15:28 qubit kubelet[8101]: --docker-only Only report docker containers in addition to root stats (DEPRECATED: This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed.) Sep 03 14:15:28 qubit kubelet[8101]: --docker-root string DEPRECATED: docker root is read from docker info (this is a fallback, default: /var/lib/docker) (default "/var/lib/docker") Sep 03 14:15:28 qubit kubelet[8101]: --docker-tls use TLS to connect to docker (DEPRECATED: This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed.) Sep 03 14:15:28 qubit kubelet[8101]: --docker-tls-ca string path to trusted CA (default "ca.pem") (DEPRECATED: This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed.) Sep 03 14:15:28 qubit kubelet[8101]: --docker-tls-cert string path to client certificate (default "cert.pem") (DEPRECATED: This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed.) Sep 03 14:15:28 qubit kubelet[8101]: --docker-tls-key string path to private key (default "key.pem") (DEPRECATED: This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed.) Sep 03 14:15:28 qubit kubelet[8101]: --dynamic-config-dir string The Kubelet will use this directory for checkpointing downloaded configurations and tracking configuration health. The Kubelet will create this directory if it does not already exist. The path may be absolute or relative; relative paths start at the Kubelet's current working directory. Providing this flag enables dynamic Kubelet configuration. The DynamicKubeletConfig feature gate must be enabled to pass this flag; this gate currently defaults to true because the feature is beta. Sep 03 14:15:28 qubit kubelet[8101]: --enable-cadvisor-json-endpoints Enable cAdvisor json /spec and /stats/* endpoints. (default true) Sep 03 14:15:28 qubit kubelet[8101]: --enable-controller-attach-detach Enables the Attach/Detach controller to manage attachment/detachment of volumes scheduled to this node, and disables kubelet from executing any attach/detach operations (default true) (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.) Sep 03 14:15:28 qubit kubelet[8101]: --enable-debugging-handlers Enables server endpoints for log collection and local running of containers and commands (default true) (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.) Sep 03 14:15:28 qubit kubelet[8101]: --enable-load-reader Whether to enable cpu load reader (DEPRECATED: This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed.) Sep 03 14:15:28 qubit kubelet[8101]: --enable-server Enable the Kubelet's server (default true) Sep 03 14:15:28 qubit kubelet[8101]: --enforce-node-allocatable strings A comma separated list of levels of node allocatable enforcement to be enforced by kubelet. Acceptable options are 'none', 'pods', 'system-reserved', and 'kube-reserved'. If the latter two options are specified, '--system-reserved-cgroup' and '--kube-reserved-cgroup' must also be set, respectively. If 'none' is specified, no additional options should be set. See https://kubernetes.io/docs/tasks/administer-cluster/reserve-compute-resources/ for more details. (default [pods]) (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.) Sep 03 14:15:28 qubit kubelet[8101]: --event-burst int32 Maximum size of a bursty event records, temporarily allows event records to burst to this number, while still not exceeding event-qps. Only used if --event-qps > 0 (default 10) (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.) Sep 03 14:15:28 qubit kubelet[8101]: --event-qps int32 If > 0, limit event creations per second to this value. If 0, unlimited. (default 5) (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.) Sep 03 14:15:28 qubit kubelet[8101]: --event-storage-age-limit string Max length of time for which to store events (per type). Value is a comma separated list of key values, where the keys are event types (e.g.: creation, oom) or "default" and the value is a duration. Default is applied to all non-specified event types (default "default=0") (DEPRECATED: This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed.) Sep 03 14:15:28 qubit kubelet[8101]: --event-storage-event-limit string Max number of events to store (per type). Value is a comma separated list of key values, where the keys are event types (e.g.: creation, oom) or "default" and the value is an integer. Default is applied to all non-specified event types (default "default=0") (DEPRECATED: This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed.) Sep 03 14:15:28 qubit kubelet[8101]: --eviction-hard mapStringString A set of eviction thresholds (e.g. memory.available<1Gi) that if met would trigger a pod eviction. (default imagefs.available<15%,memory.available<100Mi,nodefs.available<10%,nodefs.inodesFree<5%) (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.) Sep 03 14:15:28 qubit kubelet[8101]: --eviction-max-pod-grace-period int32 Maximum allowed grace period (in seconds) to use when terminating pods in response to a soft eviction threshold being met. If negative, defer to pod specified value. (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.) Sep 03 14:15:28 qubit kubelet[8101]: --eviction-minimum-reclaim mapStringString A set of minimum reclaims (e.g. imagefs.available=2Gi) that describes the minimum amount of resource the kubelet will reclaim when performing a pod eviction if that resource is under pressure. (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.) Sep 03 14:15:28 qubit kubelet[8101]: --eviction-pressure-transition-period duration Duration for which the kubelet has to wait before transitioning out of an eviction pressure condition. (default 5m0s) (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.) Sep 03 14:15:28 qubit kubelet[8101]: --eviction-soft mapStringString A set of eviction thresholds (e.g. memory.available<1.5Gi) that if met over a corresponding grace period would trigger a pod eviction. (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.) Sep 03 14:15:28 qubit kubelet[8101]: --eviction-soft-grace-period mapStringString A set of eviction grace periods (e.g. memory.available=1m30s) that correspond to how long a soft eviction threshold must hold before triggering a pod eviction. (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.) Sep 03 14:15:28 qubit kubelet[8101]: --exit-on-lock-contention Whether kubelet should exit upon lock-file contention. Sep 03 14:15:28 qubit kubelet[8101]: --experimental-allocatable-ignore-eviction When set to 'true', Hard Eviction Thresholds will be ignored while calculating Node Allocatable. See https://kubernetes.io/docs/tasks/administer-cluster/reserve-compute-resources/ for more details. [default=false] Sep 03 14:15:28 qubit kubelet[8101]: --experimental-bootstrap-kubeconfig string (DEPRECATED: Use --bootstrap-kubeconfig) Sep 03 14:15:28 qubit kubelet[8101]: --experimental-check-node-capabilities-before-mount [Experimental] if set true, the kubelet will check the underlying node for required components (binaries, etc.) before performing the mount Sep 03 14:15:28 qubit kubelet[8101]: --experimental-kernel-memcg-notification If enabled, the kubelet will integrate with the kernel memcg notification to determine if memory eviction thresholds are crossed rather than polling. Sep 03 14:15:28 qubit kubelet[8101]: --experimental-mounter-path string [Experimental] Path of mounter binary. Leave empty to use the default mount. Sep 03 14:15:28 qubit kubelet[8101]: --fail-swap-on Makes the Kubelet fail to start if swap is enabled on the node. (default true) (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.) Sep 03 14:15:28 qubit kubelet[8101]: --feature-gates mapStringBool A set of key=value pairs that describe feature gates for alpha/experimental features. Options are: Sep 03 14:15:28 qubit kubelet[8101]: APIListChunking=true|false (BETA - default=true) Sep 03 14:15:28 qubit kubelet[8101]: APIResponseCompression=true|false (ALPHA - default=false) Sep 03 14:15:28 qubit kubelet[8101]: AllAlpha=true|false (ALPHA - default=false) Sep 03 14:15:28 qubit kubelet[8101]: AppArmor=true|false (BETA - default=true) Sep 03 14:15:28 qubit kubelet[8101]: AttachVolumeLimit=true|false (BETA - default=true) Sep 03 14:15:28 qubit kubelet[8101]: BalanceAttachedNodeVolumes=true|false (ALPHA - default=false) Sep 03 14:15:28 qubit kubelet[8101]: BlockVolume=true|false (BETA - default=true) Sep 03 14:15:28 qubit kubelet[8101]: BoundServiceAccountTokenVolume=true|false (ALPHA - default=false) Sep 03 14:15:28 qubit kubelet[8101]: CPUManager=true|false (BETA - default=true) Sep 03 14:15:28 qubit kubelet[8101]: CRIContainerLogRotation=true|false (BETA - default=true) Sep 03 14:15:28 qubit kubelet[8101]: CSIBlockVolume=true|false (BETA - default=true) Sep 03 14:15:28 qubit kubelet[8101]: CSIDriverRegistry=true|false (BETA - default=true) Sep 03 14:15:28 qubit kubelet[8101]: CSIInlineVolume=true|false (ALPHA - default=false) Sep 03 14:15:28 qubit kubelet[8101]: CSIMigration=true|false (ALPHA - default=false) Sep 03 14:15:28 qubit kubelet[8101]: CSIMigrationAWS=true|false (ALPHA - default=false) Sep 03 14:15:28 qubit kubelet[8101]: CSIMigrationAzureDisk=true|false (ALPHA - default=false) Sep 03 14:15:28 qubit kubelet[8101]: CSIMigrationAzureFile=true|false (ALPHA - default=false) Sep 03 14:15:28 qubit kubelet[8101]: CSIMigrationGCE=true|false (ALPHA - default=false) Sep 03 14:15:28 qubit kubelet[8101]: CSIMigrationOpenStack=true|false (ALPHA - default=false) Sep 03 14:15:28 qubit kubelet[8101]: CSINodeInfo=true|false (BETA - default=true) Sep 03 14:15:28 qubit kubelet[8101]: CustomCPUCFSQuotaPeriod=true|false (ALPHA - default=false) Sep 03 14:15:28 qubit kubelet[8101]: CustomResourceDefaulting=true|false (ALPHA - default=false) Sep 03 14:15:28 qubit kubelet[8101]: CustomResourcePublishOpenAPI=true|false (BETA - default=true) Sep 03 14:15:28 qubit kubelet[8101]: CustomResourceSubresources=true|false (BETA - default=true) Sep 03 14:15:28 qubit kubelet[8101]: CustomResourceValidation=true|false (BETA - default=true) Sep 03 14:15:28 qubit kubelet[8101]: CustomResourceWebhookConversion=true|false (BETA - default=true) Sep 03 14:15:28 qubit kubelet[8101]: DebugContainers=true|false (ALPHA - default=false) Sep 03 14:15:28 qubit kubelet[8101]: DevicePlugins=true|false (BETA - default=true) Sep 03 14:15:28 qubit kubelet[8101]: DryRun=true|false (BETA - default=true) Sep 03 14:15:28 qubit kubelet[8101]: DynamicAuditing=true|false (ALPHA - default=false) Sep 03 14:15:28 qubit kubelet[8101]: DynamicKubeletConfig=true|false (BETA - default=true) Sep 03 14:15:28 qubit kubelet[8101]: ExpandCSIVolumes=true|false (ALPHA - default=false) Sep 03 14:15:28 qubit kubelet[8101]: ExpandInUsePersistentVolumes=true|false (BETA - default=true) Sep 03 14:15:28 qubit kubelet[8101]: ExpandPersistentVolumes=true|false (BETA - default=true) Sep 03 14:15:28 qubit kubelet[8101]: ExperimentalCriticalPodAnnotation=true|false (ALPHA - default=false) Sep 03 14:15:28 qubit kubelet[8101]: ExperimentalHostUserNamespaceDefaulting=true|false (BETA - default=false) Sep 03 14:15:28 qubit kubelet[8101]: HyperVContainer=true|false (ALPHA - default=false) Sep 03 14:15:28 qubit kubelet[8101]: KubeletPodResources=true|false (BETA - default=true) Sep 03 14:15:28 qubit kubelet[8101]: LocalStorageCapacityIsolation=true|false (BETA - default=true) Sep 03 14:15:28 qubit kubelet[8101]: LocalStorageCapacityIsolationFSQuotaMonitoring=true|false (ALPHA - default=false) Sep 03 14:15:28 qubit kubelet[8101]: MountContainers=true|false (ALPHA - default=false) Sep 03 14:15:28 qubit kubelet[8101]: NodeLease=true|false (BETA - default=true) Sep 03 14:15:28 qubit kubelet[8101]: NonPreemptingPriority=true|false (ALPHA - default=false) Sep 03 14:15:28 qubit kubelet[8101]: PodShareProcessNamespace=true|false (BETA - default=true) Sep 03 14:15:28 qubit kubelet[8101]: ProcMountType=true|false (ALPHA - default=false) Sep 03 14:15:28 qubit kubelet[8101]: QOSReserved=true|false (ALPHA - default=false) Sep 03 14:15:28 qubit kubelet[8101]: RemainingItemCount=true|false (ALPHA - default=false) Sep 03 14:15:28 qubit kubelet[8101]: RequestManagement=true|false (ALPHA - default=false) Sep 03 14:15:28 qubit kubelet[8101]: ResourceLimitsPriorityFunction=true|false (ALPHA - default=false) Sep 03 14:15:28 qubit kubelet[8101]: ResourceQuotaScopeSelectors=true|false (BETA - default=true) Sep 03 14:15:28 qubit kubelet[8101]: RotateKubeletClientCertificate=true|false (BETA - default=true) Sep 03 14:15:28 qubit kubelet[8101]: RotateKubeletServerCertificate=true|false (BETA - default=true) Sep 03 14:15:28 qubit kubelet[8101]: RunAsGroup=true|false (BETA - default=true) Sep 03 14:15:28 qubit kubelet[8101]: RuntimeClass=true|false (BETA - default=true) Sep 03 14:15:28 qubit kubelet[8101]: SCTPSupport=true|false (ALPHA - default=false) Sep 03 14:15:28 qubit kubelet[8101]: ScheduleDaemonSetPods=true|false (BETA - default=true) Sep 03 14:15:28 qubit kubelet[8101]: ServerSideApply=true|false (ALPHA - default=false) Sep 03 14:15:28 qubit kubelet[8101]: ServiceLoadBalancerFinalizer=true|false (ALPHA - default=false) Sep 03 14:15:28 qubit kubelet[8101]: ServiceNodeExclusion=true|false (ALPHA - default=false) Sep 03 14:15:28 qubit kubelet[8101]: StorageVersionHash=true|false (BETA - default=true) Sep 03 14:15:28 qubit kubelet[8101]: StreamingProxyRedirects=true|false (BETA - default=true) Sep 03 14:15:28 qubit kubelet[8101]: SupportNodePidsLimit=true|false (BETA - default=true) Sep 03 14:15:28 qubit kubelet[8101]: SupportPodPidsLimit=true|false (BETA - default=true) Sep 03 14:15:28 qubit kubelet[8101]: Sysctls=true|false (BETA - default=true) Sep 03 14:15:28 qubit kubelet[8101]: TTLAfterFinished=true|false (ALPHA - default=false) Sep 03 14:15:28 qubit kubelet[8101]: TaintBasedEvictions=true|false (BETA - default=true) Sep 03 14:15:28 qubit kubelet[8101]: TaintNodesByCondition=true|false (BETA - default=true) Sep 03 14:15:28 qubit kubelet[8101]: TokenRequest=true|false (BETA - default=true) Sep 03 14:15:28 qubit kubelet[8101]: TokenRequestProjection=true|false (BETA - default=true) Sep 03 14:15:28 qubit kubelet[8101]: ValidateProxyRedirects=true|false (BETA - default=true) Sep 03 14:15:28 qubit kubelet[8101]: VolumePVCDataSource=true|false (ALPHA - default=false) Sep 03 14:15:28 qubit kubelet[8101]: VolumeSnapshotDataSource=true|false (ALPHA - default=false) Sep 03 14:15:28 qubit kubelet[8101]: VolumeSubpathEnvExpansion=true|false (BETA - default=true) Sep 03 14:15:28 qubit kubelet[8101]: WatchBookmark=true|false (ALPHA - default=false) Sep 03 14:15:28 qubit kubelet[8101]: WinDSR=true|false (ALPHA - default=false) Sep 03 14:15:28 qubit kubelet[8101]: WinOverlay=true|false (ALPHA - default=false) Sep 03 14:15:28 qubit kubelet[8101]: WindowsGMSA=true|false (ALPHA - default=false) (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.) Sep 03 14:15:28 qubit kubelet[8101]: --file-check-frequency duration Duration between checking config files for new data (default 20s) (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.) Sep 03 14:15:28 qubit kubelet[8101]: --global-housekeeping-interval duration Interval between global housekeepings (default 1m0s) (DEPRECATED: This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed.) Sep 03 14:15:28 qubit kubelet[8101]: --hairpin-mode string How should the kubelet setup hairpin NAT. This allows endpoints of a Service to loadbalance back to themselves if they should try to access their own Service. Valid values are "promiscuous-bridge", "hairpin-veth" and "none". (default "promiscuous-bridge") (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.) Sep 03 14:15:28 qubit kubelet[8101]: --healthz-bind-address 0.0.0.0 The IP address for the healthz server to serve on (set to 0.0.0.0 for all IPv4 interfaces and `::` for all IPv6 interfaces) (default 127.0.0.1) (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.) Sep 03 14:15:28 qubit systemd[1]: kubelet.service: Main process exited, code=exited, status=255/EXCEPTION Expected results: kubeadm init --pod-network-cidr=10.244.0.0/16 [init] Using Kubernetes version: v1.15.3 [preflight] Running pre-flight checks [WARNING SystemVerification]: this Docker version is not on the list of validated versions: 19.03.1. Latest validated version: 18.09 [preflight] Pulling images required for setting up a Kubernetes cluster [preflight] This might take a minute or two, depending on the speed of your internet connection [preflight] You can also perform this action in beforehand using 'kubeadm config images pull' [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env" [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml" [kubelet-start] Activating the kubelet service [certs] Using certificateDir folder "/etc/kubernetes/pki" [certs] Generating "front-proxy-ca" certificate and key [certs] Generating "front-proxy-client" certificate and key [certs] Generating "etcd/ca" certificate and key [certs] Generating "etcd/server" certificate and key [certs] etcd/server serving cert is signed for DNS names [qubit localhost] and IPs [192.168.43.88 127.0.0.1 ::1] [certs] Generating "etcd/healthcheck-client" certificate and key [certs] Generating "apiserver-etcd-client" certificate and key [certs] Generating "etcd/peer" certificate and key [certs] etcd/peer serving cert is signed for DNS names [qubit localhost] and IPs [192.168.43.88 127.0.0.1 ::1] [certs] Generating "ca" certificate and key [certs] Generating "apiserver" certificate and key [certs] apiserver serving cert is signed for DNS names [qubit kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 192.168.43.88] [certs] Generating "apiserver-kubelet-client" certificate and key [certs] Generating "sa" key and public key [kubeconfig] Using kubeconfig folder "/etc/kubernetes" [kubeconfig] Writing "admin.conf" kubeconfig file [kubeconfig] Writing "kubelet.conf" kubeconfig file [kubeconfig] Writing "controller-manager.conf" kubeconfig file [kubeconfig] Writing "scheduler.conf" kubeconfig file [control-plane] Using manifest folder "/etc/kubernetes/manifests" [control-plane] Creating static Pod manifest for "kube-apiserver" [control-plane] Creating static Pod manifest for "kube-controller-manager" [control-plane] Creating static Pod manifest for "kube-scheduler" [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests" [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s [apiclient] All control plane components are healthy after 39.002197 seconds [upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace [kubelet] Creating a ConfigMap "kubelet-config-1.15" in namespace kube-system with the configuration for the kubelets in the cluster [upload-certs] Skipping phase. Please see --upload-certs [mark-control-plane] Marking the node qubit as control-plane by adding the label "node-role.kubernetes.io/master=''" [mark-control-plane] Marking the node qubit as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule] [kubelet-check] Initial timeout of 40s passed. [bootstrap-token] Using token: s4797h.3veascl0nj36tgj8 [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles [bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials [bootstrap-token] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token [bootstrap-token] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster [bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace [addons] Applied essential addon: CoreDNS [addons] Applied essential addon: kube-proxy Your Kubernetes control-plane has initialized successfully! To start using your cluster, you need to run the following as a regular user: mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config You should now deploy a pod network to the cluster. Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at: https://kubernetes.io/docs/concepts/cluster-administration/addons/ Then you can join any number of worker nodes by running the following on each as root: kubeadm join 192.168.43.88:6443 --token s4797h.3veascl0nj36tgj8 \ --discovery-token-ca-cert-hash sha256:88a617913b447ebb788e4fd52f651838c13d9b30a18db24cddeff7f7a9d902e6 Additional info: The cause of issue is that in kubernetes 15.2 version, parameters should be set via the config file specified by the Kubelet's --config flag. From official k8s repository the systemd service unit has next configuration: https://github.com/kubernetes/kubernetes/blob/master/build/rpms/10-kubeadm.conf # Note: This dropin only works with kubeadm and kubelet v1.11+ [Service] Environment="KUBELET_KUBECONFIG_ARGS=--bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf" Environment="KUBELET_CONFIG_ARGS=--config=/var/lib/kubelet/config.yaml" # This is a file that "kubeadm init" and "kubeadm join" generates at runtime, populating the KUBELET_KUBEADM_ARGS variable dynamically EnvironmentFile=-/var/lib/kubelet/kubeadm-flags.env # This is a file that the user can use for overrides of the kubelet args as a last resort. Preferably, the user should use # the .NodeRegistration.KubeletExtraArgs object in the configuration files instead. KUBELET_EXTRA_ARGS should be sourced from this file. EnvironmentFile=-/etc/sysconfig/kubelet ExecStart= ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS Where it has Environment="KUBELET_CONFIG_ARGS=--config=/var/lib/kubelet/config.yaml" linked to yaml generating by kubadm during initialization.
As packaged in kubernetes-node-1.15.2-1.fc30.x86_64, the kubelet.service is misconfigured. Just plain # dnf install -y /usr/lib/systemd/system/kubelet.service # systemctl start kubelet.service # systemctl status kubelet.service yields ● kubelet.service - Kubernetes Kubelet Server Loaded: loaded (/usr/lib/systemd/system/kubelet.service; disabled; vendor preset: disabled) Active: failed (Result: exit-code) since Thu 2019-09-12 20:09:08 CEST; 4min 2s ago Docs: https://github.com/GoogleCloudPlatform/kubernetes Process: 10018 ExecStart=/usr/bin/kubelet $KUBE_LOGTOSTDERR $KUBE_LOG_LEVEL $KUBELET_API_SERVER $KUBELET_ADDRESS $KUBELET_PORT $KUBELET_HOSTNAME $KUBE_ALLOW_PRIV $KUBELET_ARGS (code=exited, status=255/EXCEPTION) Main PID: 10018 (code=exited, status=255/EXCEPTION) CPU: 125ms Sep 12 20:09:08 machine.example.test systemd[1]: kubelet.service: Main process exited, code=exited, status=255/EXCEPTION Sep 12 20:09:08 machine.example.test systemd[1]: kubelet.service: Failed with result 'exit-code'. Sep 12 20:09:08 machine.example.test systemd[1]: kubelet.service: Consumed 125ms CPU time. Sep 12 20:09:08 machine.example.test systemd[1]: kubelet.service: Service RestartSec=100ms expired, scheduling restart. Sep 12 20:09:08 machine.example.test systemd[1]: kubelet.service: Scheduled restart job, restart counter is at 5. Sep 12 20:09:08 machine.example.test systemd[1]: Stopped Kubernetes Kubelet Server. Sep 12 20:09:08 machine.example.test systemd[1]: kubelet.service: Consumed 125ms CPU time. Sep 12 20:09:08 machine.example.test systemd[1]: kubelet.service: Start request repeated too quickly. Sep 12 20:09:08 machine.example.test systemd[1]: kubelet.service: Failed with result 'exit-code'. Sep 12 20:09:08 machine.example.test systemd[1]: Failed to start Kubernetes Kubelet Server. In the journal, there's Sep 12 20:09:08 machine.example.test audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=kubelet comm="systemd" exe="/usr/lib/syste md/systemd" hostname=? addr=? terminal=? res=success' Sep 12 20:09:08 machine.example.test kubelet[10018]: Flag --address has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https ://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 12 20:09:08 machine.example.test kubelet[10018]: Usage: Sep 12 20:09:08 machine.example.test kubelet[10018]: kubelet [flags] Sep 12 20:09:08 machine.example.test kubelet[10018]: Flags: Sep 12 20:09:08 machine.example.test kubelet[10018]: --address 0.0.0.0 The IP address for the Kubelet to serve on (set to 0.0.0.0 for all IPv4 interfaces and `::` for all IPv6 interfaces) (default 0.0.0.0) (DEPRECATED: This parameter should be set via the config file specified by t he Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.) Sep 12 20:09:08 machine.example.test kubelet[10018]: --allowed-unsafe-sysctls strings Comma-separated whitel [...] Sep 12 18:23:14 kvm-02-guest18.rhts.eng.brq.redhat.com kubelet[14673]: --vmodule moduleSpec Sep 12 18:23:14 kvm-02-guest18.rhts.eng.brq.redhat.com kubelet[14673]: --volume-plugin-dir string Sep 12 18:23:14 kvm-02-guest18.rhts.eng.brq.redhat.com kubelet[14673]: --volume-stats-agg-period duration Sep 12 18:23:14 kvm-02-guest18.rhts.eng.brq.redhat.com kubelet[14673]: F0912 18:23:14.356051 14673 server.go:156] unknown flag: --allow-privileged That --allow-privileged comes from # grep -r allow-privileged /etc/kubernetes /etc/kubernetes/config:KUBE_ALLOW_PRIV="--allow-privileged=false" When I comment out that KUBE_ALLOW_PRIV line in /etc/kubernetes/config, the kubelet.service with its out-of-box configuration at least starts and does not die immediatelly. So the /etc/kubernetes/config is likely the first thing that needs to change to accomodate Kubernetes 1.15.
Hello Jan Pazdzioram, (In reply to Jan Pazdziora from comment #1) > As packaged in kubernetes-node-1.15.2-1.fc30.x86_64, the kubelet.service is > misconfigured. > > > So the /etc/kubernetes/config is likely the first thing that needs to change > to accomodate Kubernetes 1.15. What would be the approach to fix it? I might help to build stable rpm. Shall it be remained as kubernetes v1.15 or it can be switched to latest v1.16 ? Sincerely Alexander
Given that even rawhide only has 1.15 now, and given that different Kubernetes versions bring small incompatibilities, I'd say rebase to 1.16 in rawhide and possibly in Fedora 30, and fix the config for the 1.15 bulid in Fedora 30.
This message is a reminder that Fedora 30 is nearing its end of life. Fedora will stop maintaining and issuing updates for Fedora 30 on 2020-05-26. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as EOL if it remains open with a Fedora 'version' of '30'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora 30 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged change the 'version' to a later Fedora version prior this bug is closed as described in the policy above. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete.
Fedora 30 changed to end-of-life (EOL) status on 2020-05-26. Fedora 30 is no longer maintained, which means that it will not receive any further security or bug fix updates. As a result we are closing this bug. If you can reproduce this bug against a currently maintained version of Fedora please feel free to reopen this bug against that version. If you are unable to reopen this bug, please file a new report against the current release. If you experience problems, please add a comment to this bug. Thank you for reporting this bug and we are sorry it could not be fixed.