Bug 1694788
| Summary: | Evicted pods have poor output in 3.11 console | ||
|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Clayton Coleman <ccoleman> |
| Component: | Management Console | Assignee: | Jon Jackson <jonjacks> |
| Status: | CLOSED ERRATA | QA Contact: | Yadan Pei <yapei> |
| Severity: | high | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 3.11.0 | CC: | aos-bugs, jokerman, mmccomas, smunilla, spadgett, yanpzhan, yapei |
| Target Milestone: | --- | ||
| Target Release: | 3.11.z | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | Bug Fix | |
| Doc Text: |
Cause: Pod status of 'failed' was displayed instead of the reason when a pod was evicted.
Consequence: It was more difficult to tell when a pod was failing because it was evicted. The user had to drill down into the details to see that the reason.
Fix: When a pod is failing due to eviction, show it's status as 'Evicted' instead of 'Failed'
Result: Pod's failing due to eviction now show 'Evicted' in the status field of the pod list and details pages.
|
Story Points: | --- |
| Clone Of: | Environment: | ||
| Last Closed: | 2019-06-04 10:46:44 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
https://www.dropbox.com/s/2kapdpxeuncf7zs/Screenshot%202019-04-01%2013.07.11.png?dl=0 is what 3.11 shows https://github.com/openshift/console/pull/1379 for 3.11 is not merged yet. The 3.11 fix has merged now. Note that we have only back-ported this fix to the 3.11 admin console and have not made any changes to the developer console. 1. Create some Evicted pods
# oc get pods
NAME READY STATUS RESTARTS AGE
django-ex-10-build 0/1 Completed 0 7m
django-ex-11-build 0/1 Completed 0 6m
django-ex-11-wbt5s 0/1 Pending 0 3m
django-ex-11-wvmcd 0/1 Evicted 0 6m
django-ex-12-build 0/1 Evicted 0 5m
django-ex-13-build 0/1 Evicted 0 5m
django-ex-14-build 0/1 Evicted 0 5m
django-ex-15-build 0/1 Evicted 0 5m
django-ex-6-g7tjm 0/1 Evicted 0 47m
django-ex-7-build 0/1 Completed 0 44m
django-ex-8-build 0/1 Completed 0 39m
django-ex-9-build 0/1 Completed 0 8m
2. Check reason
# oc get pods django-ex-11-wvmcd -o yaml
apiVersion: v1
kind: Pod
metadata:
......
status:
message: 'The node was low on resource: ephemeral-storage. Container django-ex was
using 108Ki, which exceeds its request of 0. '
phase: Failed
reason: Evicted
startTime: 2019-04-16T06:14:21Z
3. login to admin console, goes to Workloads -> Pods page, Pod status is shown as 'Evicted', see attachment
4. Verified on openshift v3.11.106
console image:
REPOSITORY TAG IMAGE ID CREATED SIZE
registry.reg-aws.openshift.com:443/openshift3/ose-console v3.11 79d038390108 8 hours ago 254 MB
"io.openshift.build.commit.id": "bfcf8d02df4e0d8f68ba3648b4efac47c5611252",
"io.openshift.build.commit.url": "https://github.com/openshift/console/commit/bfcf8d02df4e0d8f68ba3648b4efac47c5611252",
$ git log bfcf8d02df4e0d8f68ba3648b4efac47c5611252 | grep '#1379'
Merge pull request #1379 from openshift-cherrypick-robot/cherry-pick-1373-to-release-3.11
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:0758 |
An evicted pod looks like: status: phase: Failed message: >- The node was low on resource: ephemeral-storage. Container web was using 832Ki, which exceeds its request of 0. reason: Evicted startTime: '2019-03-28T05:29:41Z' In the 3.11 cluster this was shown as a "Failed" pod with no additional info. We need to show the reason next to failed, and probably show the message in the container status. Note that there is no container status, which probably confused the console. Please verify that 4.1 correctly handles evicted pods (and consider including failure reason in pod status display). The CLI shows "Evicted" instead of "Failed" because this is such an important condition.