Bug 1538215 – [DOCKER] Eviction manager erros in node logs

Bugzilla will be upgraded to version 5.0. The upgrade date is tentatively scheduled for 2 December 2018, pending final testing and feedback.

Bug 1538215 - [DOCKER] Eviction manager erros in node logs

Summary:	[DOCKER] Eviction manager erros in node logs

Status:	CLOSED CURRENTRELEASE

Aliases:	None

Product:	OpenShift Container Platform
Classification:	Red Hat
Component:	Containers (Show other bugs)
Sub Component:
Version:	3.9.0
Hardware:	Unspecified Unspecified

Priority	unspecified Severity medium
Target Milestone:	---
Target Release:	3.9.z
Assigned To:	Antonio Murdaca
QA Contact:	Vikas Laad
Docs Contact:

URL:
Whiteboard:
Keywords:

Depends On:
Blocks:
	Show dependency tree / graph

Reported:	2018-01-24 11:04 EST by Vikas Laad
Modified:	2018-06-18 14:17 EDT (History)
CC List:	7 users (show)

See Also:
Fixed In Version:	openshift v3.9.19
Doc Type:	If docs needed, set a value
Doc Text:
Story Points:	---
Clone Of:
Environment:
Last Closed:	2018-06-18 14:17:56 EDT
Type:	Bug
Regression:	---
Mount Type:	---
Documentation:	---
CRM:
Verified Versions:
Category:	---
oVirt Team:	---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---

Attachments	(Terms of Use)
Add an attachment (proposed patch, testcase, etc.)

External Trackers
Tracker	ID	Priority	Status	Summary	Last Updated
Red Hat Product Errata	RHSA-2018:2013	normal	SHIPPED_LIVE	Important: OpenShift Container Platform 3.9 security, bug fix, and enhancement update	2018-06-27 18:01:43 EDT

Groups: None (edit)

Description Vikas Laad 2018-01-24 11:04:33 EST

Description of problem:
Lots of Eviction Manager errors in node logs as follows

Jan 20 23:11:49 ip-172-31-13-204.us-west-2.compute.internal atomic-openshift-node[5748]: E0120 23:11:49.959266    5748 eviction_manager.go:238] eviction manager: unexpected err: failed to get imageFs stats: failed to get image stats: rpc error: code = Unknown desc = Error response from daemon: layer does not exist

Version-Release number of selected component (if applicable):
openshift v3.9.0-0.20.0
kubernetes v1.9.1+a0ce1bc657
etcd 3.2.8

Steps to Reproduce:
1. Keep creating/building/scaling quickstart apps on OCP cluster
2. See node logs for errors

Actual results:
Eviction errors in node logs

Expected results:
No errors in node logs

Additional info:
Attaching node logs

Comment 7 Antonio Murdaca 2018-02-06 11:29:11 EST

fixed in latest branches for 1.12/1.13

Comment 10 Vikas Laad 2018-04-16 09:43:24 EDT

Verified in following version, did not see the problem after running reliability tests for a week.

openshift v3.9.19
kubernetes v1.9.1+a0ce1bc657
etcd 3.2.16

Client:
 Version:         1.13.1
 API version:     1.26
 Package version: <unknown>
 Go version:      go1.8.3
 Git commit:      774336d/1.13.1
 Built:           Tue Feb 20 13:46:34 2018
 OS/Arch:         linux/amd64

Server:
 Version:         1.13.1
 API version:     1.26 (minimum version 1.12)
 Package version: <unknown>
 Go version:      go1.8.3
 Git commit:      774336d/1.13.1
 Built:           Tue Feb 20 13:46:34 2018
 OS/Arch:         linux/amd64
 Experimental:    false

Note You need to log in before you can comment on or make changes to this bug.