Bug 1581409 - Limit ranges are being applied with cpu-cfs-quota set to false
Summary: Limit ranges are being applied with cpu-cfs-quota set to false
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Node
Version: 3.7.0
Hardware: Unspecified
OS: Linux
unspecified
urgent
Target Milestone: ---
: 3.10.0
Assignee: Seth Jennings
QA Contact: DeShuai Ma
URL:
Whiteboard:
Depends On:
Blocks: 1581861 1581860 1581862 1581863 1582486 1582487
TreeView+ depends on / blocked
 
Reported: 2018-05-22 17:03 UTC by Taneem Ibrahim
Modified: 2018-07-30 19:16 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Fixes an issue where cfs_quota might still be set on a pod even when cpu-cfs-quota is set to false on the node
Clone Of:
: 1581860 1581861 1581862 1581863 1582486 1582487 (view as bug list)
Environment:
Last Closed: 2018-07-30 19:16:14 UTC


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2018:1816 None None None 2018-07-30 19:16:51 UTC

Description Taneem Ibrahim 2018-05-22 17:03:21 UTC
Description of problem:

This issue was originally thought to be resolved as part of https://bugzilla.redhat.com/show_bug.cgi?id=1554471 . However, the same issue is still showing up on v3.7.42 build. 

Version-Release number of selected component (if applicable):

v3.7.42
kubernetes v1.7.6+a08f5eeb62

How reproducible:

Not easily. Certain pods show the cpu.cfs_quota_us set at -1 as expected with cpu-cfs-quota flag set to false on the kubelet but several other pods show actual cpu values 1000,2000, etc.

Steps to Reproduce:

I have not been able to re-produce it yet on my local cluster unfortunately.

Actual results:


Expected results:


Additional info:

- attached node logs, pod yamls from working and non working.
- print out of the cpu.cfs_quota_us from various pods is added as a private comment. below.

Comment 9 DeShuai Ma 2018-05-25 15:52:44 UTC
Verify build https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=16434524

[root@ip-172-18-12-225 test]# rpm -qa|grep openshift
atomic-openshift-master-3.7.46-1.git.1.849c97d.el7.x86_64
atomic-openshift-docker-excluder-3.7.46-1.git.1.849c97d.el7.noarch
tuned-profiles-atomic-openshift-node-3.7.46-1.git.1.849c97d.el7.x86_64
atomic-openshift-excluder-3.7.46-1.git.1.849c97d.el7.noarch
atomic-openshift-clients-3.7.46-1.git.1.849c97d.el7.x86_64
atomic-openshift-node-3.7.46-1.git.1.849c97d.el7.x86_64
atomic-openshift-3.7.46-1.git.1.849c97d.el7.x86_64
atomic-openshift-sdn-ovs-3.7.46-1.git.1.849c97d.el7.x86_64

[root@ip-172-18-12-225 ~]# oc version
oc v3.7.46
kubernetes v1.7.6+a08f5eeb62
features: Basic-Auth GSSAPI Kerberos SPNEGO

Server https://ip-172-18-12-225.ec2.internal:8443
openshift v3.7.46
kubernetes v1.7.6+a08f5eeb62

Verify steps:
//Scenario A cpu-cfs-quota=false
kubeletArguments:
  cpu-cfs-quota:
  - 'false'

1. Create a burstable pod 
[root@ip-172-18-12-225 ~]# oc create -f pod.yaml 
pod "burstable-vqpsr" created
[root@ip-172-18-12-225 ~]# 
[root@ip-172-18-12-225 ~]# oc get po -o yaml|grep -i uid
      openshift.io/scc: anyuid
    uid: 5a2dea60-6031-11e8-89e2-0eb7f09c6860

2. On node the pod located to check the cpu.cfs_quota_us in pod level cgroup.
[root@ip-172-18-5-70 test]# cat /sys/fs/cgroup/cpu/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a2dea60_6031_11e8_89e2_0eb7f09c6860.slice/cpu.cfs_quota_us
-1

//Scenario B cpu-cfs-quota=true
kubeletArguments:
  cpu-cfs-quota:
  - 'true'
1. Create a burstable pod 
[root@ip-172-18-12-225 ~]# oc create -f pod.yaml 
pod "burstable-8tmdv" created
[root@ip-172-18-12-225 ~]# oc get po -o yaml|grep uid
      openshift.io/scc: anyuid
    uid: 9ec90ce2-6031-11e8-89e2-0eb7f09c6860
[root@ip-172-18-12-225 ~]# oc get po
NAME              READY     STATUS    RESTARTS   AGE
burstable-8tmdv   1/1       Running   0          22s

2. On node the pod located to check the cpu.cfs_quota_us in pod level cgroup.
[root@ip-172-18-5-70 test]# cat /sys/fs/cgroup/cpu/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ec90ce2_6031_11e8_89e2_0eb7f09c6860.slice/cpu.cfs_quota_us
100000

//pod.yaml
apiVersion: v1
kind: Pod
metadata:
  generateName: burstable-
spec:
  containers:
  - image: busybox
    name: busybox
    command:
    - sleep
    - "3600"
    resources:
      requests:
        memory: 1500Mi
        cpu: 300m
      limits:
        cpu: 1
        memory: 3Gi
  restartPolicy: Always
  terminationGracePeriodSeconds: 0

Comment 12 DeShuai Ma 2018-06-06 08:03:13 UTC
Verified on openshift v3.10.0-0.60.0 Step can ref comment 9

Comment 15 errata-xmlrpc 2018-07-30 19:16:14 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:1816


Note You need to log in before you can comment on or make changes to this bug.