Description of problem: Lots of Eviction Manager errors in node logs as follows Jan 20 23:11:49 ip-172-31-13-204.us-west-2.compute.internal atomic-openshift-node[5748]: E0120 23:11:49.959266 5748 eviction_manager.go:238] eviction manager: unexpected err: failed to get imageFs stats: failed to get image stats: rpc error: code = Unknown desc = Error response from daemon: layer does not exist Version-Release number of selected component (if applicable): openshift v3.9.0-0.20.0 kubernetes v1.9.1+a0ce1bc657 etcd 3.2.8 Steps to Reproduce: 1. Keep creating/building/scaling quickstart apps on OCP cluster 2. See node logs for errors Actual results: Eviction errors in node logs Expected results: No errors in node logs Additional info: Attaching node logs
fixed in latest branches for 1.12/1.13
Verified in following version, did not see the problem after running reliability tests for a week. openshift v3.9.19 kubernetes v1.9.1+a0ce1bc657 etcd 3.2.16 Client: Version: 1.13.1 API version: 1.26 Package version: <unknown> Go version: go1.8.3 Git commit: 774336d/1.13.1 Built: Tue Feb 20 13:46:34 2018 OS/Arch: linux/amd64 Server: Version: 1.13.1 API version: 1.26 (minimum version 1.12) Package version: <unknown> Go version: go1.8.3 Git commit: 774336d/1.13.1 Built: Tue Feb 20 13:46:34 2018 OS/Arch: linux/amd64 Experimental: false