Bug 1692131 - CPU Accounting is not enabled by default
Summary: CPU Accounting is not enabled by default
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Node
Version: 4.1.0
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
: 4.1.0
Assignee: Ryan Phillips
QA Contact: Sunil Choudhary
URL:
Whiteboard:
Depends On:
Blocks: 1777173
TreeView+ depends on / blocked
 
Reported: 2019-03-24 14:40 UTC by Derek Carr
Modified: 2019-11-27 03:53 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1777173 (view as bug list)
Environment:
Last Closed: 2019-06-04 10:46:21 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2019:0758 0 None None None 2019-06-04 10:46:28 UTC

Description Derek Carr 2019-03-24 14:40:37 UTC
Description of problem:
kubelet logs show the following message:

475] failed to find cgroups of kubelet - cpu and memory cgroup hierarchy not unified.  cpu: /, memory: /system.slice/kubelet.service

Version-Release number of selected component (if applicable):
4.0.0-0.alpha-2019-03-23-103439

How reproducible:
Always

Steps to Reproduce:
1. install cluster
2. ssh to node
3. journalctl -u kubelet

Actual results:
kubelet does not appear to be in a cpu cgroup

Expected results:
kubelet should be in cpu cgroup

Additional info:
If you cat /proc/<pid>/cgroup for kubelet, we see following:

cat /proc/951/cgroup 
12:freezer:/
11:memory:/system.slice/kubelet.service
10:cpuset:/
9:hugetlb:/
8:pids:/system.slice/kubelet.service
7:devices:/
6:net_cls,net_prio:/
5:rdma:/
4:blkio:/
3:cpu,cpuacct:/
2:perf_event:/
1:name=systemd:/system.slice/kubelet.service

It appears that RHEL CoreOS is not enabling default cpu accounting in systemd.

CPU accounting should be enabled by default.

cat /etc/systemd/system.conf | grep CPUAccounting
#DefaultCPUAccounting=no

In 3.x releases, we always enabled cpu accounting as a drop-in.

Comment 1 Seth Jennings 2019-03-25 22:28:23 UTC
The drop-in is this
https://github.com/openshift/origin/blob/master/contrib/systemd/origin-accounting.conf

I guess we need the MCO to lay this down for us.

Ryan can you do this?

Comment 2 Ryan Phillips 2019-03-26 14:53:57 UTC
I'll look into it.

Comment 4 Ryan Phillips 2019-03-26 15:31:33 UTC
Oops. Sounds like we want to modify the system.conf within the rpm. Looking into it.

Comment 5 Ryan Phillips 2019-04-01 22:08:45 UTC
PR merged https://github.com/openshift/machine-config-operator/pull/581 and verified working.

systemctl show -all|grep -e 'DefaultCPUAccounting\|DefaultBlockIOAccounting\|DefaultMemoryAccounting'

Comment 9 errata-xmlrpc 2019-06-04 10:46:21 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:0758


Note You need to log in before you can comment on or make changes to this bug.