Bug 1375167

Summary:	Support for Quota QoS scope restrictions on projects
Product:	OpenShift Container Platform	Reporter:	Jaspreet Kaur <jkaur>
Component:	RFE	Assignee:	Derek Carr <decarr>
Status:	CLOSED DEFERRED	QA Contact:	Johnny Liu <jialiu>
Severity:	medium	Docs Contact:
Priority:	medium
Version:	3.2.0	CC:	aos-bugs, decarr, jkaur, jokerman, mbarrett, mmccomas, simon.gunzenreiner
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2018-03-12 13:54:36 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:

Description Jaspreet Kaur 2016-09-12 11:11:00 UTC

1. What is the nature and description of the request ?

We would like to be able to specify on a project level (unmodified through project admin), what QoS scopes (https://docs.openshift.com/enterprise/latest/admin_guide/quota.html#quota-scopes) a project team my define on their pods.

4. Why does the customer need this? (List the business requirements here)
Specifically for "marketing" purposes (but one could also imagine other scenarios) , we would like to provide low-QoS (no SLA) projects where resources are not guaranteed. This would allow us to optimally use a given platform capacity for "sandboxes" (with no QoS), but at the same time, allow for production projects to have ensured capacity.

5. How would the customer like to achieve this? (List the functional requirements here)
Specify which QoS a project might request, on a per project (namespace) level.

6. For each functional requirement listed in question, specify how Red Hat and the customer can test to confirm the requirement is successfully implemented.
- Specify on a project A's namespace that QoS might only be best effort.
-- try to schedule with "guaranteed", must fail
-- schedule a pod in that project A with no or best effort QoS --> success
- Fill OpenShift with pods from other project B which has QoS guaranteed until there is no more free capacity on any node
-- pod in project A should become unscheduled

7. Is there already an existing RFE upstream or in Red Hat bugzilla?
Not that the customer is aware of it.

8. Does the customer have any specific timeline dependencies?
No.

Comment 3 Derek Carr 2017-10-10 22:12:16 UTC

I do not think we need to enhance the product to achieve the customers desired outcome. In order to reach this desired outcome, I think a customer can do the following with the existing quota facilities.

$ oc new-project project-a

# define a quota to allow BestEffort workloads in this project
$ oc create quota besteffort --hard=pods=3 --scopes=BestEffort

# define a quota to not allow workloads that make resource requests
$ oc create quota not-besteffort --hard=pods=0 --scopes=NotBestEffort

# create a deployment that attempts to use resources
# it will be denied based on the not-besteffort quota
$ oc run not-allowed --image=nginx --requests=cpu=1

# create a besteffort pod (will succeed because we allow 3)
$ oc run allowed --image=nginx

The above prevents users in project-a from running anything other than resource scavenging workloads.

Users in project-b can then run workloads that make explicit resource requirements. Those pods will get scheduled to nodes. The pod-cgroup hierarchy introduced in OCP 3.6 will ensure that via CFS shares that BestEffort workloads will get the bare minimum of CPU (i.e. 2 shares). If memory is exhausted on the local node, the pods from project-a will get evicted before pods in project-b (that made resource requirements).

The quota system will not be extended to introduce a scope per QoS tier as the effective difference between Guaranteed and Burstable is not really existent. The scheduler and node only guarantee requests (and Guaranteed pods just have limits equal to their request).

Comment 5 Eric Rich 2018-03-12 13:54:36 UTC

This bug has been identified as a dated (created more than 3 months ago) bug. 
This bug has been triaged (has a trello card linked to it), or reviewed by Engineering/PM and has been put into the product backlog, 
however this bug has not been slated for a currently planned release (3.9, 3.10 or 3.11), which cover our releases for the rest of the calendar year. 

As a result of this bugs age, state on the current roadmap and PM Score (being below 70), this bug is being Closed - Differed, 
as it is currently not part of the products immediate priorities.

Please see: https://docs.google.com/document/d/1zdqF4rB3ea8GmVIZ7qWCVYUaQ7-EexUrQEF0MTwdDkw/edit for more details.