Bug 1392254 - On scale up/down: Unable to inspect docker image while inspecting container - error or log spam
Summary: On scale up/down: Unable to inspect docker image while inspecting container -...
Keywords:
Status: CLOSED DUPLICATE of bug 1389183
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Node
Version: 3.4.0
Hardware: x86_64
OS: Linux
unspecified
medium
Target Milestone: ---
: ---
Assignee: Seth Jennings
QA Contact: DeShuai Ma
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-11-07 00:29 UTC by Mike Fiedler
Modified: 2016-11-08 18:35 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-11-08 18:35:05 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
node log (19.67 KB, application/x-gzip)
2016-11-07 00:30 UTC, Mike Fiedler
no flags Details

Description Mike Fiedler 2016-11-07 00:29:08 UTC
Description of problem:

Running tests which do hundreds of scale up/downs, the logs get spammed with the following E/Error level message on every scaleup:

Nov  6 19:20:56 ip-172-31-24-197 atomic-openshift-node: E1106 19:20:56.502088   13169 docker_manager.go:404] unable to inspect docker image "POD" while inspecting docker container "sha256:1924b34d94a8d8b77aa6e7742dbd17ea5c4587a9ef68b66b6d9bb70138322030": no such image: "sha256:1924b34d94a8d8b77aa6e7742dbd17ea5c4587a9ef68b66b6d9bb70138322030"

The scale up seems to work ok.  Reduce the loglevel on this message?  or investigate as a real Error?

This is with SELinux disabled and using overlayfs as the docker storage


Version-Release number of selected component (if applicable):   3.4.0.22


How reproducible: Always


Steps to Reproduce:
1.  Install 3.4.0.22 with Docker 1.12.3-3 using overlayfs.  Disable SELinux
2.  oc new-app --template=cakephp-mysql-example and wait for it to build
3.  oc scale --replicas=20 dc/cakephp-mysql-example
4.  Watch syslogs for E messages

Actual results:

24 instances of the above message and a successful scaleup.   Potential underlying issue?

Expected results:

Quiet logs for a successful scaleup.

Additional info:

Comment 1 Mike Fiedler 2016-11-07 00:30:46 UTC
Created attachment 1217842 [details]
node log

Comment 2 Mike Fiedler 2016-11-07 00:40:49 UTC
These 24 messages are also accompanied by 24 instances of this error message:

E1106 19:20:58.023842   13169 docker_manager.go:761] Logging security options: {key:seccomp value:unconfined msg:}

Comment 3 Mike Fiedler 2016-11-07 00:45:53 UTC
On scale down an addition 28 instances of the message pop:

Nov  6 19:42:17 ip-172-31-24-197 atomic-openshift-node: E1106 19:42:17.533707   13169 docker_manager.go:404] unable to inspect docker image "cakephp-mysql-example" while inspecting docker container "sha256:1f1ffe3f6f0285686192496d15209831d74eb02959c9e4b6c6debe60b624121b": no such image: "sha256:1f1ffe3f6f0285686192496d15209831d74eb02959c9e4b6c6debe60b624121b"


Scale up 20 and scale down to 0 yields gives us 52 of the above errors.

Comment 4 Seth Jennings 2016-11-08 18:35:05 UTC
This issue is addressed by the PRs (one still pending merge) in
https://bugzilla.redhat.com/show_bug.cgi?id=1389183

https://github.com/openshift/origin/pull/11674
https://github.com/openshift/origin/pull/11793

I was able to recreate on master but not on the branch for PR 11793.

*** This bug has been marked as a duplicate of bug 1389183 ***


Note You need to log in before you can comment on or make changes to this bug.