Bug 1707785
Summary: | Autoscaling for Memory Utilization is not working | ||
---|---|---|---|
Product: | OpenShift Container Platform | Reporter: | Weinan Liu <weinliu> |
Component: | Node | Assignee: | Joel Smith <joelsmith> |
Status: | CLOSED ERRATA | QA Contact: | Weinan Liu <weinliu> |
Severity: | medium | Docs Contact: | |
Priority: | medium | ||
Version: | 4.1.0 | CC: | aos-bugs, erich, gblomqui, joelsmith, jokerman, mmccomas, schoudha, suchaudh, wsun |
Target Milestone: | --- | Keywords: | Reopened |
Target Release: | 4.3.0 | ||
Hardware: | Unspecified | ||
OS: | Unspecified | ||
Whiteboard: | |||
Fixed In Version: | Doc Type: | If docs needed, set a value | |
Doc Text: | Story Points: | --- | |
Clone Of: | Environment: | ||
Last Closed: | 2020-01-23 11:03:45 UTC | Type: | Bug |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: |
Description
Weinan Liu
2019-05-08 10:55:28 UTC
"DeploymentConfig" is misspelled https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/#api-object This is working fine afiact $ oc new-project demo Now using project "demo" on server "https://api.lab.variantweb.net:6443". You can add applications to this project with the 'new-app' command. For example, try: oc new-app centos/ruby-25-centos7~https://github.com/sclorg/ruby-ex.git to build a new example application in Ruby. $ ls memory.yaml rc.yaml $ cat rc.yaml apiVersion: v1 kind: ReplicationController metadata: labels: run: hello-openshift name: hello-openshift spec: replicas: 1 selector: run: hello-openshift template: metadata: creationTimestamp: null labels: run: hello-openshift spec: containers: - image: openshift/hello-openshift imagePullPolicy: Always name: hello-openshift ports: - containerPort: 8080 protocol: TCP resources: limits: cpu: 500m memory: 512Mi requests: cpu: 100m memory: 256Mi terminationMessagePath: /dev/termination-log dnsPolicy: ClusterFirst restartPolicy: Always securityContext: {} terminationGracePeriodSeconds: 30 $ oc create -f rc.yaml replicationcontroller/hello-openshift created $ cat memory.yaml apiVersion: autoscaling/v2beta1 kind: HorizontalPodAutoscaler metadata: name: resource-memory spec: scaleTargetRef: kind: ReplicationController name: hello-openshift minReplicas: 1 maxReplicas: 3 metrics: - type: Resource resource: name: memory targetAverageValue: 500Mi $ oc create -f memory.yaml horizontalpodautoscaler.autoscaling/resource-memory created $ oc describe hpa resource-memory Name: resource-memory Namespace: demo Labels: <none> Annotations: <none> CreationTimestamp: Thu, 09 May 2019 09:01:48 -0500 Reference: ReplicationController/hello-openshift Metrics: ( current / target ) resource memory on pods: <unknown> / 500Mi Min replicas: 1 Max replicas: 3 ReplicationController pods: 1 current / 1 desired Conditions: Type Status Reason Message ---- ------ ------ ------- AbleToScale True ReadyForNewScale recommended size matches current size ScalingActive True ValidMetricFound the HPA was able to successfully calculate a replica count from memory resource ScalingLimited False DesiredWithinRange the desired count is within the acceptable range Events: Type Reason Age From Message ---- ------ ---- ---- ------- Warning FailedGetResourceMetric 43s (x3 over 73s) horizontal-pod-autoscaler unable to get metrics for resource memory: no metrics returned from resource metrics API Warning FailedComputeMetricsReplicas 43s (x3 over 73s) horizontal-pod-autoscaler failed to get memory utilization: unable to get metrics for resource memory: no metrics returned from resource metrics API The events are transient while the pod metrics start to be reported by cadvisor. If you delay the creation of the hpa, the events won't appear. The hpa is `ReadyForNewScale=True` and `ValidMetricFound=True` and calculated desired count properly `ReplicationController pods: 1 current / 1 desired` What is the problem exactly? [nathan@localhost 0510]$ oc get hpa NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE resource-memory ReplicationController/hello-openshift <unknown>/500Mi 1 10 1 24h Hi Seth, We can not get current memory usage, so the memory autoscaling is not working Joel can you look into why we are getting <unknown> even though ScalingActive=True which suggest we can get the metric? *** Bug 1701469 has been marked as a duplicate of this bug. *** Just tried this on a 4.2 CI build and it works fine $ ogcv NAME VERSION AVAILABLE PROGRESSING SINCE STATUS version 4.2.0-0.ci-2019-06-18-161924 True False 39m Cluster version is 4.2.0-0.ci-2019-06-18-161924 $ oc describe hpa Name: resource-memory Namespace: demo Labels: <none> Annotations: <none> CreationTimestamp: Tue, 18 Jun 2019 14:46:47 -0500 Reference: ReplicationController/hello-openshift Metrics: ( current / target ) resource memory on pods: 1355776 / 512Mi <---- Min replicas: 1 Max replicas: 3 ReplicationController pods: 1 current / 1 desired Conditions: Type Status Reason Message ---- ------ ------ ------- AbleToScale True ReadyForNewScale recommended size matches current size ScalingActive True ValidMetricFound the HPA was able to successfully calculate a replica count from memory resource ScalingLimited False DesiredWithinRange the desired count is within the acceptable range Events: <none> Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:0062 |