| Summary: | Need to configure AWS device limitation for scheduling | ||
|---|---|---|---|
| Product: | OpenShift Online | Reporter: | Qixuan Wang <qixuan.wang> |
| Component: | Pod | Assignee: | Abhishek Gupta <abhgupta> |
| Status: | CLOSED CURRENTRELEASE | QA Contact: | Qixuan Wang <qixuan.wang> |
| Severity: | medium | Docs Contact: | |
| Priority: | high | ||
| Version: | 3.x | CC: | abhgupta, agoldste, aos-bugs, jokerman, mmccomas |
| Target Milestone: | --- | ||
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | Bug Fix | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2016-05-23 15:08:20 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
Working on it. This is now fixed in INT Tested on dev-preview-int (3.2.0.6)
Confirmed with Ops，we have added “MaxEBSVolumeCount” on master:
[root@dev-preview-int-master-21eb2 ~]# cat /etc/origin/master/scheduler.json
{
"kind": "Policy",
"apiVersion": "v1",
"predicates": [
{"name": "MatchNodeSelector"},
{"name": "PodFitsResources"},
{"name": "PodFitsPorts"},
{"name": "NoDiskConflict"},
{"name" : "MaxEBSVolumeCount"}
],"priorities": [
{"name": "LeastRequestedPriority", "weight": 1},
{"name": "ServiceSpreadingPriority", "weight": 1}
]
}
But we didn't get KUBE_MAX_PD_VOLS in the cluster. Here is an ideal place where it should be.
[root@dev-preview-int-master-21eb2 ~]# cat /etc/sysconfig/atomic-openshift-master
OPTIONS=--loglevel=2
CONFIG_FILE=/etc/origin/master/master-config.yaml
AWS_ACCESS_KEY_ID=xxxx
AWS_SECRET_ACCESS_KEY=xxxx
# Proxy configuration
# Origin uses standard HTTP_PROXY environment variables. Be sure to set
# NO_PROXY for your master
#NO_PROXY=master.example.com
#HTTP_PROXY=http://USER:PASSWORD@IPADDR:PORT
#HTTPS_PROXY=https://USER:PASSWORD@IPADDR:PORT
So I wanna confirm that KUBE_MAX_PD_VOLS=$number is configured? Except for "/etc/sysconfig/atomic-openshift-master", where or how to set the number?
If that environment variable (configuration) is not provided, the default value of 39 EBS volumes is applied. Refer: https://github.com/kubernetes/kubernetes/blob/master/plugin/pkg/scheduler/algorithmprovider/defaults/defaults.go#L113-L120 and https://github.com/kubernetes/kubernetes/blob/master/plugin/pkg/scheduler/algorithmprovider/defaults/defaults.go#L39-L51 If we use DefaultMaxEBSVolumes instead of set a value, I can verify the bug. Thanks. |
Description of problem: We need to add "MaxEBSVolumeCount" as a predicate so AWS EBS limitations is taken into account during scheduling. We also need to give a number to the environment variable "KUBE_MAX_PD_VOLS" to constraint the maximum device attached on each node. Version-Release number of selected component (if applicable): dev-preview-int How reproducible: Always Steps to Reproduce: 1. Check scheduler.json on master 2. Check /etc/sysconfig/atomic-openshift-master on master Actual results: 1. # cat /etc/origin/master/scheduler.json { "kind": "Policy", "apiVersion": "v1", "predicates": [ {"name": "MatchNodeSelector"}, {"name": "PodFitsResources"}, {"name": "PodFitsPorts"}, {"name": "NoDiskConflict"}, {"name": "Region", "argument": {"serviceAffinity" : {"labels" : ["region"]}}} ],"priorities": [ {"name": "LeastRequestedPriority", "weight": 1}, {"name": "ServiceSpreadingPriority", "weight": 1}, {"name": "Zone", "weight" : 2, "argument": {"serviceAntiAffinity" : {"label": "zone"}}} ] } 2. #cat /etc/sysconfig/atomic-openshift-master OPTIONS=--loglevel=2 CONFIG_FILE=/etc/origin/master/master-config.yaml # Proxy configuration # Origin uses standard HTTP_PROXY environment variables. Be sure to set # NO_PROXY for your master #NO_PROXY=master.example.com #HTTP_PROXY=http://USER:PASSWORD@IPADDR:PORT #HTTPS_PROXY=https://USER:PASSWORD@IPADDR:PORT Expected results: 1. We should add {"name": "MaxEBSVolumeCount"} in predicates: 2. We should add "KUBE_MAX_PD_VOLS=$number" in /etc/sysconfig/atomic-openshift-master.(number < 40) Additional info: