Description of problem: Cannot run prune in the registry. It fails with: ERRO[1760] failed to get the image stream dk0502-a/psd2-anusha/psd2-anusha: invalid resource name "psd2-anusha/psd2-anusha": [may not contain '/'] Version-Release number of selected component (if applicable): 3.9 How reproducible: Every single time Steps to Reproduce: 1. oc exec docker-registry-NAME -- /bin/sh -c 'REGISTRY_LOG_LEVEL=info /usr/bin/dockerregistry -prune=check' Actual results: ERRO[1760] failed to get the image stream dk0502-a/psd2-anusha/psd2-anusha: invalid resource name "psd2-anusha/psd2-anusha": [may not contain '/'] Expected results: Prune finishes correctly. Additional info: Some troubleshooting has been done, we found missing references to a previous image which was wrong and was deleted. We also deleted the imagestream (and we will push it back again) but then the problem reproduced with another image so we must confirm what's going on under the hood before continuing deleting images.
Some troubleshooting: $ oc delete is psd2-anusha imagestream "psd2-anusha" deleted $ oc get is NAME DOCKER REPO TAGS UPDATED psd2-karthik-002 docker-registry.default.svc:5000/dk0502-a/psd2-karthik-002 1.0.0 14 months ago psd2-sujit docker-registry.default.svc:5000/dk0502-a/psd2-sujit 1.0.0 13 months ago $ oc get istag NAME DOCKER REF UPDATED IMAGENAME psd2-karthik-002:1.0.0 172.30.154.106:5000/dk0502-a/psd2-karthik-002@sha256:fad833b733493a7ae7a8cb95039b1e563a31838a015f0bd683cd6a5561fec457 14 months ago sha256:fad833b733493a7ae7a8cb95039b1e563a31838a015f0bd683cd6a5561fec457 psd2-sujit:1.0.0 172.30.154.106:5000/dk0502-a/psd2-sujit@sha256:6184d37ebe6cd2de51feb930ac5f628aae3ba7905e6680d188c812719c000302 13 months ago sha256:6184d37ebe6cd2de51feb930ac5f628aae3ba7905e6680d188c812719c000302 So that image stream is no more in dk0502-a project and we expected to succeed but we got the same error: $ oc exec docker-registry-NAME -- /bin/sh -c 'REGISTRY_LOG_LEVEL=info /usr/bin/dockerregistry -prune=check' ... INFO[1759] Would delete manifest link: dk0501-a/virtuoso@sha256:ea4eacba0302d418cb05f11215a5cdd50f51f2422910c9f869f7d39459c58476 go.version=go1.9.4 instance.id=2b65daf4-7012-4cd6-a918-0ce94ad26587 INFO[1759] Would delete manifest link: dk0501-a/virtuoso72@sha256:09c75b8cca710ab86ea46c1b128608bfee88eec80f7da3199d85540560cfd345 go.version=go1.9.4 instance.id=2b65daf4-7012-4cd6-a918-0ce94ad26587 INFO[1759] Would delete manifest link: dk0501-a/virtuoso72@sha256:1f298418e82d72e5339d3c5fb943499d72e7d2627dd7e30caa8eff3faf4c5db5 go.version=go1.9.4 instance.id=2b65daf4-7012-4cd6-a918-0ce94ad26587 INFO[1759] Would delete manifest link: dk0501-a/virtuoso72@sha256:2ae3d1a894a2722a68ae271adffefaf6b4b0fe885fcb6e55e97aaef12ea42b6b go.version=go1.9.4 instance.id=2b65daf4-7012-4cd6-a918-0ce94ad26587 INFO[1760] Would delete manifest link: dk0501-a/virtuoso72@sha256:7400245604988798f638333d573d88eeef386a813d73e065ab08765be7fe93aa go.version=go1.9.4 instance.id=2b65daf4-7012-4cd6-a918-0ce94ad26587 INFO[1760] Would delete manifest link: dk0501-a/virtuoso72@sha256:a59ace00e6759f2f625a1147e0e3d4b948ceb278c260cf9dde0ee933c123d063 go.version=go1.9.4 instance.id=2b65daf4-7012-4cd6-a918-0ce94ad26587 INFO[1760] Would delete manifest link: dk0501-a/virtuoso72@sha256:b069c5798a9644c3727783e96a5dcc391bcb24154f641b20f401dcbefce95b49 go.version=go1.9.4 instance.id=2b65daf4-7012-4cd6-a918-0ce94ad26587 INFO[1760] Would delete manifest link: dk0501-a/virtuoso72@sha256:c81398875cb5bfbb27174dda51c11c512f2e4a75689efe4f9a46124d4378389c go.version=go1.9.4 instance.id=2b65daf4-7012-4cd6-a918-0ce94ad26587 INFO[1760] Would delete manifest link: dk0501-a/virtuoso72@sha256:ee587a9ab56ec595ae495b9364843d245befb73a48a6d5408c1646b62bd91e69 go.version=go1.9.4 instance.id=2b65daf4-7012-4cd6-a918-0ce94ad26587 INFO[1760] Would delete manifest link: dk0501-a/virtuoso72@sha256:f929d02383e05eba1bf40efc7dc31cb626cc43bb2648ea51d4179db314e83a1c go.version=go1.9.4 instance.id=2b65daf4-7012-4cd6-a918-0ce94ad26587 INFO[1760] The image stream dk0502-a/psd2-anusha is not found, will remove the whole repository go.version=go1.9.4 instance.id=2b65daf4-7012-4cd6-a918-0ce94ad26587 INFO[1760] Would delete manifest link: dk0501-a/virtuoso72@sha256:f9a32215c227c3298e3f90440798135469fd5d3e24d20c6b6ec5f1f142c94293 go.version=go1.9.4 instance.id=2b65daf4-7012-4cd6-a918-0ce94ad26587 ERRO[1760] failed to get the image stream dk0502-a/psd2-anusha/psd2-anusha: invalid resource name "psd2-anusha/psd2-anusha": [may not contain '/'] Would delete 0 blobs Would free up 0B of disk space Then we deleted the directory directly from the /registry filesystem and try again: $ oc exec docker-registry-NAME -- /bin/sh -c 'REGISTRY_LOG_LEVEL=info /usr/bin/dockerregistry -prune=check' ... ERRO[2347] failed to get the image stream dk0709-d/jupyter/minimal-notebook: invalid resource name "jupyter/minimal-notebook": [may not contain '/'] Same error with another image. Is there any way to know what is happening and not just keep deleting images?
*** Bug 1752513 has been marked as a duplicate of this bug. ***
Steps: 1. Import some image, it'll create something like /docker/registry/v2/repositories/NAMESPACE/NAME 2. Copy these files as docker/registry/v2/repositories/foo/bar/baz 3. $ oc adm policy add-cluster-role-to-user system:image-pruner system:serviceaccount:openshift-image-registry:registry 4. $ oc exec image-registry-98888df69-j7gs7 -- /bin/sh -c 'REGISTRY_LOG_LEVEL=info /usr/bin/dockerregistry -prune=check' time="2019-10-24T07:15:55.362046985Z" level=info msg="start prune (dry-run mode)" distribution_version=v2.6.0+unknown go.version=go1.12.8 openshift_version=v4.3.0-201910221117+c34a783-dirty time="2019-10-24T07:15:55.749814531Z" level=info msg="Invalid image name foo/bar/ruby-ex, removing whole repository" go.version=go1.12.8 time="2019-10-24T07:15:56.0891229Z" level=info msg="Would delete repository: foo/bar/ruby-ex" go.version=go1.12.8 Would delete 0 blobs Would free up 0B of disk space Use -prune=delete to actually delete the data Verified on 4.3.0-0.nightly-2019-10-23-150053
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:0062