Description of problem: The docker image registry at a customer has several recurring errors reported, and it's not clear to me if they should really be considered "errors". I originally reported as https://github.com/openshift/docker-registry-extensions/issues/9 and https://github.com/openshift/docker-registry-extensions/issues/8 but since this is a registry on 3.2 I thought a bug might serve better. In a customer environment with OSE 3.2 they're getting these errors in the registry logs: time="2016-10-10T22:01:04.451940836-04:00" level=error msg="response completed with error" err.code="BLOB_UNKNOWN" err.detail=sha256:398b653eab57fc5b408fdb65ab811ecd555d551da18e98400d14cb0d6249dfe9 err.message="blob unknown to registry" go.version=go1.4.2 http.request.host="172.30.248.117:5000" http.request.id=7956882c-26d4-4a2f-a8e7-aa0396622853 http.request.method=GET http.request.remoteaddr="10.1.2.1:58209" http.request.uri="/v2/orange/php/blobs/sha256:398b653eab57fc5b408fdb65ab811ecd555d551da18e98400d14cb0d6249dfe9" http.request.useragent="docker/1.10.3 go/go1.4.2 git-commit/7ffc8ee-unsupported kernel/3.10.0-327.13.1.el7.x86_64 os/linux arch/amd64" http.response.contenttype="application/json; charset=utf-8" http.response.duration=33.769186ms http.response.status=404 http.response.written=157 instance.id=a20494ee-c862-42a0-8893-577e8ef39ed8 vars.digest="sha256:398b653eab57fc5b408fdb65ab811ecd555d551da18e98400d14cb0d6249dfe9" vars.name="orange/php" I'm not sure what that means but it's in the logs a lot. Something asking about an image (layer) that's no longer in the registry? time="2016-10-10T22:01:42.623690508-04:00" level=error msg="Error retrieving ImageStreamImage admsdev/adms@sha256:43cf02f33d027880a33d27598cbb7b726ff445121a56872f2cc6d18ab8169a37: imagestreams \"adms\" not found" go.version=go1.4.2 http.request.host="172.30.248.117:5000" http.request.id=85ee9f32-bf9f-4653-be7d-49defa531542 http.request.method=GET http.request.remoteaddr="10.1.22.1:53215" http.request.uri="/v2/admsdev/adms/manifests/sha256:43cf02f33d027880a33d27598cbb7b726ff445121a56872f2cc6d18ab8169a37" http.request.useragent="docker/1.10.3 go/go1.4.2 git-commit/7ffc8ee-unsupported kernel/3.10.0-327.13.1.el7.x86_64 os/linux arch/amd64" instance.id=a20494ee-c862-42a0-8893-577e8ef39ed8 vars.name="admsdev/adms" vars.reference="sha256:43cf02f33d027880a33d27598cbb7b726ff445121a56872f2cc6d18ab8169a37" time="2016-10-10T22:01:42.623884544-04:00" level=error msg="response completed with error" err.code="MANIFEST_UNKNOWN" err.detail="imagestreams \"adms\" not found" err.message="manifest unknown" go.version=go1.4.2 http.request.host="172.30.248.117:5000" http.request.id=85ee9f32-bf9f-4653-be7d-49defa531542 http.request.method=GET http.request.remoteaddr="10.1.22.1:53215" http.request.uri="/v2/admsdev/adms/manifests/sha256:43cf02f33d027880a33d27598cbb7b726ff445121a56872f2cc6d18ab8169a37" http.request.useragent="docker/1.10.3 go/go1.4.2 git-commit/7ffc8ee-unsupported kernel/3.10.0-327.13.1.el7.x86_64 os/linux arch/amd64" http.response.contenttype="application/json; charset=utf-8" http.response.duration=17.552851ms http.response.status=404 http.response.written=285 instance.id=a20494ee-c862-42a0-8893-577e8ef39ed8 vars.name="admsdev/adms" vars.reference="sha256:43cf02f33d027880a33d27598cbb7b726ff445121a56872f2cc6d18ab8169a37" Is this a simple "missing imagestream" message or does it point to something going wrong? time="2016-10-11T03:18:23.804470343-04:00" level=error msg="response completed with error" err.code="MANIFEST_UNKNOWN" err.detail="filesystem: Path not found: /docker/registry/v2/repositories/ctm-test/dispatcher-author/_manifests/revisions/sha256/599cb877ac19c22159fc0e3634fc456291f8a6862a6e4c04635d0cd9bcec034f/signatures/sha256" err.message="manifest unknown" go.version=go1.4.2 http.request.host="172.30.248.117:5000" http.request.id=c0a7b9ff-6152-49d4-835c-ccee415f5020 http.request.method=GET http.request.remoteaddr="10.1.6.1:40197" http.request.uri="/v2/ctm-test/dispatcher-author/manifests/sha256:599cb877ac19c22159fc0e3634fc456291f8a6862a6e4c04635d0cd9bcec034f" http.request.useragent="docker/1.10.3 go/go1.4.2 git-commit/7ffc8ee-unsupported kernel/3.10.0-327.13.1.el7.x86_64 os/linux arch/amd64" http.response.contenttype="application/json; charset=utf-8" http.response.duration=48.976977ms http.response.status=404 http.response.written=287 instance.id=c987c3e6-7bb7-43fa-aa16-d1be10d64c1e vars.name="ctm-test/dispatcher-author" vars.reference="sha256:599cb877ac19c22159fc0e3634fc456291f8a6862a6e4c04635d0cd9bcec034f" One that doesn't seem like it should be level=error is: time="2016-10-10T22:05:53.09732915-04:00" level=error msg="error authorizing context: authorization header with basic token required" go.version=go1.4.2 http.request.host="172.30.248.117:5000" http.request.id=dd935353-7781-4f6b-b042-066db87335ec http.request.method=GET http.request.remoteaddr="10.1.10.1:39413" http.request.uri="/v2/" http.request.useragent="docker/1.10.3 go/go1.4.2 git-commit/7ffc8ee-unsupported kernel/3.10.0-327.13.1.el7.x86_64 os/linux arch/amd64" instance.id=a20494ee-c862-42a0-8893-577e8ef39ed8 time="2016-10-10T22:07:26.553102691-04:00" level=error msg="error authorizing context: authorization header with basic token required" go.version=go1.4.2 http.request.host="172.30.248.117:5000" http.request.id=89f60a4d-8adb-4970-bc1c-13588bd17d3b http.request.method=GET http.request.remoteaddr="10.1.13.1:46212" http.request.uri="/v2/" http.request.useragent="docker/1.10.3 go/go1.4.2 git-commit/7ffc8ee-unsupported kernel/3.10.0-327.13.1.el7.x86_64 os/linux arch/amd64" instance.id=a20494ee-c862-42a0-8893-577e8ef39ed8 time="2016-10-10T22:07:56.468117847-04:00" level=error msg="error authorizing context: authorization header with basic token required" go.version=go1.4.2 http.request.host="172.30.248.117:5000" http.request.id=24165fad-ae4a-4632-8929-806ba6cf1685 http.request.method=GET http.request.remoteaddr="10.1.12.1:56036" http.request.uri="/v2/" http.request.useragent="docker/1.10.3 go/go1.4.2 git-commit/7ffc8ee-unsupported kernel/3.10.0-327.22.2.el7.x86_64 os/linux arch/amd64" instance.id=a20494ee-c862-42a0-8893-577e8ef39ed8 time="2016-10-10T22:11:50.090330658-04:00" level=error msg="error authorizing context: authorization header with basic token required" go.version=go1.4.2 http.request.host="172.30.248.117:5000" http.request.id=72c6db2b-c879-48e7-be75-b0df974ec4ff http.request.method=GET http.request.remoteaddr="10.1.10.1:39634" http.request.uri="/v2/" http.request.useragent="docker/1.10.3 go/go1.4.2 git-commit/7ffc8ee-unsupported kernel/3.10.0-327.13.1.el7.x86_64 os/linux arch/amd64" instance.id=a20494ee-c862-42a0-8893-577e8ef39ed8 time="2016-10-10T22:11:57.595052459-04:00" level=error msg="error authorizing context: authorization header with basic token required" go.version=go1.4.2 http.request.host="172.30.248.117:5000" http.request.id=f0d04ba8-3bba-40f4-8577-b1b27b72cd63 http.request.method=GET http.request.remoteaddr="10.1.22.1:54021" http.request.uri="/v2/" http.request.useragent="docker/1.10.3 go/go1.4.2 git-commit/7ffc8ee-unsupported kernel/3.10.0-327.13.1.el7.x86_64 os/linux arch/amd64" instance.id=a20494ee-c862-42a0-8893-577e8ef39ed8 Diagnostics look through the registry logs for common errors and this doesn't seem like it really merits reporting to the user as an error. If these aren't really errors in the operation of the registry, it would be nice if they could either be downgraded from level=error or if there were another level for tagging "real" errors.
These messages are not really an errors, just OpenShift requesting objects that does not really exists anymore in registry (because they were pruned/etc). They can be safely ignored and we will make a patch upstream that will change the log level. Marking this as low priority (as it is not an error, just logging verbosity).
One more just for completeness sake. time="2016-09-29T15:16:19.557367666-04:00" level=error msg="Error getting ImageStreamTag \"latest\": imagestreamtags \"ctxmgr-app-dev:latest\" not found" go.version=go1.4.2 http.request.host="172.30.101.240:5000" http.request.id=d28759d7-ee48-4661-8158-7805819d3227 http.request.method=GET http.request.remoteaddr="10.1.11.1:54187" http.request.uri="/v2/linkctxmgrdev/ctxmgr-app-dev/manifests/latest" http.request.useragent="docker/1.10.3 go/go1.4.2 git-commit/7ffc8ee-unsupported kernel/3.10.0-327.13.1.el7.x86_64 os/linux arch/amd64" instance.id=0294efca-bce9-45dd-9864-13824c318bec vars.name="linkctxmgrdev/ctxmgr-app-dev" vars.reference=latest
Michal, did that upstream patch ever happen?
Michal, bump.
> Michal, did that upstream patch ever happen? I'm afraid not. Closing this one even though it's older since the newer bug has more up-to-date info and idieas on resolution. *** This bug has been marked as a duplicate of bug 1567658 ***
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 1000 days