Description of problem: test: [sig-imageregistry][Feature:ImageAppend] Image append should create images by appending them [Suite:openshift/conformance/parallel] The test sometimes fails due to errors likes error: uploading the source layer sha256:cca21acb641a96561e0cf9a0c1c7b7ffbaaefc92185bd8a9440f6049c838e33b failed: Patch "https://image-registry.openshift-image-registry.svc:5000/v2/e2e-test-image-append-x2z6w/test/blobs/uploads/255c02bc-0686-4885-804c-cabdd3e5a24e?_state=OuRDGVptJzYwiBi7cJ7OLJtE6UUrcgGFxnJqbZXFMOF7Ik5hbWUiOiJlMmUtdGVzdC1pbWFnZS1hcHBlbmQteDJ6NncvdGVzdCIsIlVVSUQiOiIyNTVjMDJiYy0wNjg2LTQ4ODUtODA0Yy1jYWJkZDNlNWEyNGUiLCJPZmZzZXQiOjAsIlN0YXJ0ZWRBdCI6IjIwMjEtMDItMTlUMTI6MDM6MTAuMDY5NTc0MDAxWiJ9": unknown blob (This error can be found in the test output.) In the registry logs I can see: time="2021-02-19T12:03:10.440054187Z" level=error msg="client disconnected during blob PATCH" contentLength=-1 copied=0 error="stream error: stream ID 51; CANCEL" go.version=go1.15.5 http.request.host="image-registry.openshift-image-registry.svc:5000" http.request.id=4b1455da-ceec-4d7a-8ee8-c653a904636f http.request.method=PATCH http.request.remoteaddr="10.128.2.17:60804" http.request.uri="/v2/e2e-test-image-append-x2z6w/test/blobs/uploads/255c02bc-0686-4885-804c-cabdd3e5a24e?_state=OuRDGVptJzYwiBi7cJ7OLJtE6UUrcgGFxnJqbZXFMOF7Ik5hbWUiOiJlMmUtdGVzdC1pbWFnZS1hcHBlbmQteDJ6NncvdGVzdCIsIlVVSUQiOiIyNTVjMDJiYy0wNjg2LTQ4ODUtODA0Yy1jYWJkZDNlNWEyNGUiLCJPZmZzZXQiOjAsIlN0YXJ0ZWRBdCI6IjIwMjEtMDItMTlUMTI6MDM6MTAuMDY5NTc0MDAxWiJ9" http.request.useragent=Go-http-client/2.0 openshift.auth.user="system:serviceaccount:e2e-test-image-append-x2z6w:builder" vars.name=e2e-test-image-append-x2z6w/test vars.uuid=255c02bc-0686-4885-804c-cabdd3e5a24e Version-Release number of selected component (if applicable): I found it in 4.7 and 4.8. How reproducible: Sometimes. Additional info: https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/release-openshift-ocp-installer-e2e-aws-4.8/1362724429691359232 https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/release-openshift-ocp-installer-e2e-azure-4.8/1364144090546966528 https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/release-openshift-ocp-installer-e2e-azure-ovn-4.8/1364562084272017408 https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/release-openshift-ocp-installer-e2e-azure-ovn-4.8/1364484774214766592
oc wanted to mirror the blob sha256:cca21acb641a96561e0cf9a0c1c7b7ffbaaefc92185bd8a9440f6049c838e33b from image-registry.openshift-image-registry.svc:5000/openshift/tools to image-registry.openshift-image-registry.svc:5000/e2e-test-image-append-x2z6w/test. openshift/tools:latest is an image imported from quay.io. From the logs [1] I can see that the registry tried to use pull-through for the blob, but failed due to 429: time="2021-02-19T12:03:10.435117604Z" level=error msg="Error statting blob sha256:cca21acb641a96561e0cf9a0c1c7b7ffbaaefc92185bd8a9440f6049c838e33b in remote repository \"quay.io/openshift-release-dev/ocp-v4.0-art-dev\": Head \"https://quay.io/v2/openshift-release-dev/ocp-v4.0-art-dev/blobs/sha256:cca21acb641a96561e0cf9a0c1c7b7ffbaaefc92185bd8a9440f6049c838e33b\": error parsing HTTP 429 response body: invalid character '<' looking for beginning of value: \"<html>\\r\\n<head><title>429 Too Many Requests</title></head>\\r\\n<body bgcolor=\\\"white\\\">\\r\\n<center><h1>429 Too Many Requests</h1></center>\\r\\n<hr><center>nginx/1.12.1</center>\\r\\n</body>\\r\\n</html>\\r\\n\"" go.version=go1.15.5 http.request.host="image-registry.openshift-image-registry.svc:5000" http.request.id=503b9144-5a74-4646-84a8-9ce4017d45e9 http.request.method=GET http.request.remoteaddr="10.128.2.17:60804" So the registry said "blob unknown to registry" for this blob in openshift/tools. [1]: https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/origin-ci-test/logs/release-openshift-ocp-installer-e2e-aws-4.8/1362724429691359232/artifacts/e2e-aws/pods/openshift-image-registry_image-registry-9c76c4dcf-g9d7m_registry.log
*** This bug has been marked as a duplicate of bug 1923536 ***