Bug 1277873

Summary: Error message was reported in docker registry log even when sti-build succeed
Product: OpenShift Container Platform Reporter: Anping Li <anli>
Component: Image RegistryAssignee: Michail Kargakis <mkargaki>
Status: CLOSED NOTABUG QA Contact: Wei Sun <wsun>
Severity: medium Docs Contact:
Priority: medium    
Version: 3.0.0CC: aos-bugs, pweil
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-11-06 01:07:51 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Anping Li 2015-11-04 09:56:33 UTC
Description of problem:
During openshift start-build, error messages was reported in docker registry logs even when the build succeed.

For the environment which running for days, there are lot of  such messages in docker registry logs. if that is a common message, I think an INFO type is ok.

since openshift ex diagnostics reports lot of such messages. which may make customer confused about his docker registry status.



Version-Release number of selected component (if applicable):
openshift version
openshift v3.0.2.905
kubernetes v1.2.0-alpha.1-1107-g4c8e6f4
etcd 2.1.2

How reproducible:
always

Steps to Reproduce:
1. oc new-app nodejs-example
2. waiting build is completed and  succeed and check the docker registry logs

Actual results:
1. There are rrors messages even when the build succeed.
 time="2015-11-04T04:32:59-05:00" level=error msg="error authorizing context: authorization header with basic token required" http.request.host="172.30.17.178:5000" http.request.id=93c16b7a-587d-4589-8a29-8d7776ad915a http.request.method=GET http.request.remoteaddr="10.1.2.1:37916" http.request.uri="/v2/" http.request.useragent="docker/1.8.2 go/go1.4.2 kernel/3.10.0-326.el7.x86_64 os/linux arch/amd64" instance.id=30ee5883-8012-4e2d-bdcf-1659ebd0c0c5 
       time="2015-11-04T04:33:00-05:00" level=error msg="An error occured" err.code=BLOB_UNKNOWN err.detail={sha256:4ed80e60aa8fc9aff9742eb51874a5a32162e8bc40c870675351cab0e8b6525b} err.message="blob unknown to registry" http.request.host="172.30.17.178:5000" http.request.id=69e92cd8-2748-4f85-8525-eddeb84e6a59 http.request.method=HEAD http.request.remoteaddr="10.1.2.1:37928" http.request.uri="/v2/t1/nodejs-example/blobs/sha256:4ed80e60aa8fc9aff9742eb51874a5a32162e8bc40c870675351cab0e8b6525b" http.request.useragent="docker/1.8.2 go/go1.4.2 kernel/3.10.0-326.el7.x86_64 os/linux arch/amd64" instance.id=30ee5883-8012-4e2d-bdcf-1659ebd0c0c5 vars.digest="sha256:4ed80e60aa8fc9aff9742eb51874a5a32162e8bc40c870675351cab0e8b6525b" vars.name="t1/nodejs-example" 
       time="2015-11-04T04:33:03-05:00" level=error msg="An error occured" err.code=BLOB_UNKNOWN err.detail={sha256:dd2e6cb4f96b9fc11f94216ab7ab46b6fff930c47bb05ba9e8948b44d4ba8cc1} err.message="blob unknown to registry" http.request.host="172.30.17.178:5000" http.request.id=06b22d71-f2fc-4fe9-8f4a-d503db2b8f7a http.request.method=HEAD http.request.remoteaddr="10.1.2.1:37940" http.request.uri="/v2/t1/nodejs-example/blobs/sha256:dd2e6cb4f96b9fc11f94216ab7ab46b6fff930c47bb05ba9e8948b44d4ba8cc1" http.request.useragent="docker/1.8.2 go/go1.4.2 kernel/3.10.0-326.el7.x86_64 os/linux arch/amd64" instance.id=30ee5883-8012-4e2d-bdcf-1659ebd0c0c5 vars.digest="sha256:dd2e6cb4f96b9fc11f94216ab7ab46b6fff930c47bb05ba9e8948b44d4ba8cc1" vars.name="t1/nodejs-example" 
       time="2015-11-04T04:33:13-05:00" level=error msg="An error occured" err.code=BLOB_UNKNOWN err.detail={sha256:0f13b7bd55858ad66686fc75a782b7830f1980b99541743b4ffa1ff379114605} err.message="blob unknown to registry" http.request.host="172.30.17.178:5000" http.request.id=a42949be-77dd-40f9-8aaa-4df994d496b9 http.request.method=HEAD http.request.remoteaddr="10.1.2.1:37957" http.request.uri="/v2/t1/nodejs-example/blobs/sha256:0f13b7bd55858ad66686fc75a782b7830f1980b99541743b4ffa1ff379114605" http.request.useragent="docker/1.8.2 go/go1.4.2 kernel/3.10.0-326.el7.x86_64 os/linux arch/amd64" instance.id=30ee5883-8012-4e2d-bdcf-1659ebd0c0c5 vars.digest="sha256:0f13b7bd55858ad66686fc75a782b7830f1980b99541743b4ffa1ff379114605" vars.name="t1/nodejs-example" 
       time="2015-11-04T04:33:29-05:00" level=error msg="error authorizing context: authorization header with basic token required" http.request.host="172.30.17.178:5000" http.request.id=7fb26624-ef98-4793-ba46-02e480fc641e http.request.method=GET http.request.remoteaddr="10.1.2.1:37983" http.request.uri="/v2/" http.request.useragent="docker/1.8.2 go/go1.4.2 kernel/3.10.0-326.el7.x86_64 os/linux arch/amd64" instance.id=30ee5883-8012-4e2d-bdcf-1659ebd0c0c5



Expected results:
No error was reported since sti build completed without error.

Additional info:

Comment 2 Paul Weil 2015-11-04 17:59:17 UTC
Per Michalis' analysis, this doesn't appear to be affecting functionality and is just producing log messages.  Marking this as upcoming release for now.

Comment 3 Anping Li 2015-11-05 01:14:53 UTC
(In reply to Paul Weil from comment #2)
> Per Michalis' analysis, this doesn't appear to be affecting functionality
> and is just producing log messages.  Marking this as upcoming release for
> now.

The system checking tools 'openshift ex diagnostics', it will report lots of such messages. If we leave this as upcoming release. which may make customer confused. Should we introduces that in 3.1 document.

Comment 4 Paul Weil 2015-11-05 17:57:32 UTC
(In reply to Anping Li from comment #3)
> The system checking tools 'openshift ex diagnostics', it will report lots of
> such messages. If we leave this as upcoming release. which may make customer
> confused. Should we introduces that in 3.1 document.

Michalis, please check with Brenton and Clayton on this.

Comment 5 Paul Weil 2015-11-05 20:23:59 UTC
Removing upcoming release based on email conversation.

Comment 6 Paul Weil 2015-11-06 01:05:38 UTC
Confirmed that the BLOB_UNKNOWN error is benign.  It is docker checking for the existence of the blob with a HEAD request prior to uploading.  

Michalis will double check the auth error and I think we can close this out rather than marking for UpcomingRelease.

Comment 7 Paul Weil 2015-11-06 01:07:51 UTC
Also confirmed that the auth error is expected.  It will happen periodically when docker talks to the registry.  Docker will resend when it gets the 401.