Description of problem: When run oc adm catalog mirror command, there's error message Version-Release number of selected component (if applicable): # oc version Client Version: 4.3.10 How reproducible: Just two steps # oc adm catalog build --appregistry-org=redhat-operators --to=registry.ocp4.example.com:5443/catalog/redhat-operators:20200403 # oc adm catalog mirror --manifests-only registry.ocp4.example.com:5443/catalog/redhat-operators:20200403 registry.ocp4.example.com:5443 I0407 09:06:56.735621 10421 mirror.go:231] wrote database to /tmp/686135596/bundles.db W0407 09:06:56.870712 10421 mirror.go:258] errors during mirroring. the full contents of the catalog may not have been mirrored: couldn't parse image for mirroring (), skipping mirror: invalid reference format I0407 09:06:56.887378 10421 mirror.go:329] wrote mirroring manifests to redhat-operators-manifests Steps to Reproduce: 1. oc adm catalog build --appregistry-org=redhat-operators --to=registry.ocp4.example.com:5443/catalog/redhat-operators:20200403 2. oc adm catalog mirror --manifests-only registry.ocp4.example.com:5443/catalog/redhat-operators:20200403 registry.ocp4.example.com:5443 3. Actual results: I0407 09:06:56.735621 10421 mirror.go:231] wrote database to /tmp/686135596/bundles.db W0407 09:06:56.870712 10421 mirror.go:258] errors during mirroring. the full contents of the catalog may not have been mirrored: couldn't parse image for mirroring (), skipping mirror: invalid reference format I0407 09:06:56.887378 10421 mirror.go:329] wrote mirroring manifests to redhat-operators-manifests Expected results: No error message Additional info: Also tried use oc 4.3.5, the same issue
Were there errors in `oc adm catalog build` that have been elided from the report? I suspect this is a duplicate of: https://bugzilla.redhat.com/show_bug.cgi?id=1820419 Could you please try to create the catalog image with `oc adm catalog build`, but this time, please set the `--from` flag to the correct base image for the version of ocp. For example, for 4.4 the command would be `oc adm catalog build --from=quay.io/openshift/origin-operator-registry:4.4 <other options>`.
# oc adm catalog build --appregistry-org=redhat-operators --from=quay.io/openshift/origin-operator-registry:4.3 --to=registry.ocp4.example.com:5443/catalog/redhat-operators:20200409 ... ... ... INFO[0041] directory dir=/tmp/cache-323547050/manifests-997131483 file=1.0.8 load=package INFO[0041] directory dir=/tmp/cache-323547050/manifests-997131483 file=1.0.9 load=package INFO[0041] directory dir=/tmp/cache-323547050/manifests-997131483 file=1.1.0 load=package INFO[0041] directory dir=/tmp/cache-323547050/manifests-997131483 file=sriov-network-operator load=package INFO[0041] directory dir=/tmp/cache-323547050/manifests-997131483 file=sriov-network-operator-mlpeabrg load=package INFO[0041] directory dir=/tmp/cache-323547050/manifests-997131483 file=4.2 load=package INFO[0041] directory dir=/tmp/cache-323547050/manifests-997131483 file=4.2-s390x load=package INFO[0041] directory dir=/tmp/cache-323547050/manifests-997131483 file=4.3 load=package Uploading ... 17.59MB/s Uploading 8.202MB ... Uploading 1.573kB ... Uploading 76.23MB ... Uploading 402B ... Uploading 87.2MB ... Pushed sha256:52fbd99553adcead706f8de350a29a34e9a1caf6b1aab716094ec0ec8bee3df0 to registry.ocp4.example.com:5443/catalog/redhat-operators:20200409 # oc adm catalog mirror --manifests-only registry.ocp4.example.com:5443/catalog/redhat-operators:20200409 registry.ocp4.example.com:5443 I0409 08:04:48.342110 11331 mirror.go:231] wrote database to /tmp/798897643/bundles.db W0409 08:04:48.347417 11331 mirror.go:258] errors during mirroring. the full contents of the catalog may not have been mirrored: couldn't parse image for mirroring (), skipping mirror: invalid reference format I0409 08:04:48.385816 11331 mirror.go:329] wrote mirroring manifests to redhat-operators-manifests # oc version Client Version: 4.3.10
# oc adm catalog mirror --loglevel=10 --manifests-only registry.ocp4.example.com:5443/catalog/redhat-operators:20200409 registry.ocp4.example.com:5443 I0409 08:15:00.478697 11471 config.go:137] looking for config.json at /root/.docker/config.json I0409 08:15:00.479679 11471 config.go:145] found valid config.json at /root/.docker/config.json I0409 08:15:00.479973 11471 round_trippers.go:423] curl -k -v -XGET 'https://registry.ocp4.example.com:5443/v2/' I0409 08:15:00.524094 11471 round_trippers.go:443] GET https://registry.ocp4.example.com:5443/v2/ 200 OK in 44 milliseconds I0409 08:15:00.524111 11471 round_trippers.go:449] Response Headers: I0409 08:15:00.524115 11471 round_trippers.go:452] Date: Thu, 09 Apr 2020 00:15:00 GMT I0409 08:15:00.524118 11471 round_trippers.go:452] Content-Type: application/json; charset=utf-8 I0409 08:15:00.524121 11471 round_trippers.go:452] Docker-Distribution-Api-Version: registry/2.0 I0409 08:15:00.524124 11471 round_trippers.go:452] Content-Length: 2 I0409 08:15:00.524956 11471 round_trippers.go:423] curl -k -v -XHEAD -H "Accept: application/vnd.docker.distribution.manifest.v2+json" -H "Accept: application/vnd.docker.distribution.manifest.v1+prettyjws" -H "Accept: application/json" -H "Accept: application/vnd.docker.distribution.manifest.list.v2+json" -H "Accept: application/vnd.oci.image.index.v1+json" 'https://registry.ocp4.example.com:5443/v2/catalog/redhat-operators/manifests/20200409' I0409 08:15:00.529317 11471 round_trippers.go:443] HEAD https://registry.ocp4.example.com:5443/v2/catalog/redhat-operators/manifests/20200409 200 OK in 4 milliseconds I0409 08:15:00.529339 11471 round_trippers.go:449] Response Headers: I0409 08:15:00.529343 11471 round_trippers.go:452] Content-Type: application/vnd.docker.distribution.manifest.v2+json I0409 08:15:00.529347 11471 round_trippers.go:452] Docker-Content-Digest: sha256:52fbd99553adcead706f8de350a29a34e9a1caf6b1aab716094ec0ec8bee3df0 I0409 08:15:00.529351 11471 round_trippers.go:452] Docker-Distribution-Api-Version: registry/2.0 I0409 08:15:00.529354 11471 round_trippers.go:452] Etag: "sha256:52fbd99553adcead706f8de350a29a34e9a1caf6b1aab716094ec0ec8bee3df0" I0409 08:15:00.529357 11471 round_trippers.go:452] Content-Length: 1578 I0409 08:15:00.529360 11471 round_trippers.go:452] Date: Thu, 09 Apr 2020 00:15:00 GMT I0409 08:15:00.529448 11471 round_trippers.go:423] curl -k -v -XGET -H "Accept: application/vnd.docker.distribution.manifest.list.v2+json" -H "Accept: application/vnd.docker.distribution.manifest.v2+json" 'https://registry.ocp4.example.com:5443/v2/catalog/redhat-operators/manifests/sha256:52fbd99553adcead706f8de350a29a34e9a1caf6b1aab716094ec0ec8bee3df0' I0409 08:15:00.531338 11471 round_trippers.go:443] GET https://registry.ocp4.example.com:5443/v2/catalog/redhat-operators/manifests/sha256:52fbd99553adcead706f8de350a29a34e9a1caf6b1aab716094ec0ec8bee3df0 200 OK in 1 milliseconds I0409 08:15:00.531355 11471 round_trippers.go:449] Response Headers: I0409 08:15:00.531360 11471 round_trippers.go:452] Docker-Distribution-Api-Version: registry/2.0 I0409 08:15:00.531363 11471 round_trippers.go:452] Etag: "sha256:52fbd99553adcead706f8de350a29a34e9a1caf6b1aab716094ec0ec8bee3df0" I0409 08:15:00.531367 11471 round_trippers.go:452] Content-Length: 1578 I0409 08:15:00.531370 11471 round_trippers.go:452] Date: Thu, 09 Apr 2020 00:15:00 GMT I0409 08:15:00.531373 11471 round_trippers.go:452] Content-Type: application/vnd.docker.distribution.manifest.v2+json I0409 08:15:00.531376 11471 round_trippers.go:452] Docker-Content-Digest: sha256:52fbd99553adcead706f8de350a29a34e9a1caf6b1aab716094ec0ec8bee3df0 I0409 08:15:00.531611 11471 round_trippers.go:423] curl -k -v -XGET -H "Accept-Encoding: identity" 'https://registry.ocp4.example.com:5443/v2/catalog/redhat-operators/blobs/sha256:2160fbe1f35b9a644d25e096178eeecdaf9b77527c9c7f139880d085ca12af7d' I0409 08:15:00.536470 11471 round_trippers.go:443] GET https://registry.ocp4.example.com:5443/v2/catalog/redhat-operators/blobs/sha256:2160fbe1f35b9a644d25e096178eeecdaf9b77527c9c7f139880d085ca12af7d 200 OK in 4 milliseconds I0409 08:15:00.536488 11471 round_trippers.go:449] Response Headers: I0409 08:15:00.536492 11471 round_trippers.go:452] Accept-Ranges: bytes I0409 08:15:00.536495 11471 round_trippers.go:452] Cache-Control: max-age=31536000 I0409 08:15:00.536499 11471 round_trippers.go:452] Content-Type: application/octet-stream I0409 08:15:00.536502 11471 round_trippers.go:452] Docker-Content-Digest: sha256:2160fbe1f35b9a644d25e096178eeecdaf9b77527c9c7f139880d085ca12af7d I0409 08:15:00.536505 11471 round_trippers.go:452] Docker-Distribution-Api-Version: registry/2.0 I0409 08:15:00.536509 11471 round_trippers.go:452] Etag: "sha256:2160fbe1f35b9a644d25e096178eeecdaf9b77527c9c7f139880d085ca12af7d" I0409 08:15:00.536512 11471 round_trippers.go:452] Content-Length: 5504 I0409 08:15:00.536515 11471 round_trippers.go:452] Date: Thu, 09 Apr 2020 00:15:00 GMT I0409 08:15:00.536643 11471 manifest.go:289] Raw image config json: {"id":"","created":"2020-04-09T08:03:10.051942685+08:00","container":"48771e0b0357ad64e672994670a79510d0898d5215e1e701e6e7a4b508d5da6a","container_config":{"Hostname":"operator-registry-build","Env":["OPENSHIFT_BUILD_NAME=base","OPENSHIFT_BUILD_NAMESPACE=ci-op-20g465q9","PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin","container=oci"],"Cmd":["#(imagebuilder)\nsleep 86400"],"Image":"docker-registry.default.svc:5000/ci-op-5w2szq4m/pipeline@sha256:8f4728ddd27d43b8afc0b46f804db14dc2df87c9ddcc95b85a62e64fb03c4d2f","Entrypoint":["/bin/sh","-c"],"Labels":{"architecture":"x86_64","authoritative-source-url":"registry.access.redhat.com","build-date":"2019-10-08T12:52:56.230390","com.redhat.build-host":"cpt-1003.osbs.prod.upshift.rdu2.redhat.com","com.redhat.component":"ubi7-container","com.redhat.license_terms":"https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI","description":"The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly.","distribution-scope":"public","io.k8s.description":"This is the base image from which all OpenShift images inherit.","io.k8s.display-name":"OpenShift Base","io.openshift.build.commit.author":"","io.openshift.build.commit.date":"","io.openshift.build.commit.id":"1be922de56a99f4e254eb9c2fc42d30e5596cf77","io.openshift.build.commit.message":"","io.openshift.build.commit.ref":"master","io.openshift.build.name":"","io.openshift.build.namespace":"","io.openshift.build.source-context-dir":"base/","io.openshift.build.source-location":"https://github.com/openshift/images","io.openshift.tags":"base rhel7","maintainer":"Red Hat, Inc.","name":"ubi7","release":"178","summary":"Provides the latest release of the Red Hat Universal Base Image 7.","url":"https://access.redhat.com/containers/#/registry.access.redhat.com/ubi7/images/7.7-178","vcs-ref":"1be922de56a99f4e254eb9c2fc42d30e5596cf77","vcs-type":"git","vcs-url":"https://github.com/openshift/images","vendor":"Red Hat, Inc.","version":"7.7"}},"docker_version":"1.13.1","config":{"Hostname":"fae65ece521e","User":"1001","ExposedPorts":{"50051/tcp":{}},"Env":["foo=bar","OPENSHIFT_BUILD_NAME=operator-registry","OPENSHIFT_BUILD_NAMESPACE=ci-op-5w2szq4m","PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin","container=oci"],"Cmd":["--database","/bundles.db"],"WorkingDir":"/registry","Entrypoint":["/bin/registry-server"],"Labels":{"architecture":"x86_64","authoritative-source-url":"registry.access.redhat.com","build-date":"2019-10-08T12:52:56.230390","com.redhat.build-host":"cpt-1003.osbs.prod.upshift.rdu2.redhat.com","com.redhat.component":"ubi7-container","com.redhat.license_terms":"https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI","description":"The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly.","distribution-scope":"public","io.k8s.description":"This is a component of OpenShift Operator Lifecycle Manager and is the base for operator catalog API containers.","io.k8s.display-name":"OpenShift Operator Registry","io.openshift.build.commit.author":"","io.openshift.build.commit.date":"","io.openshift.build.commit.id":"36ee3a437614ca1cb781465e590613e01556e756","io.openshift.build.commit.message":"","io.openshift.build.commit.ref":"release-4.3","io.openshift.build.name":"","io.openshift.build.namespace":"","io.openshift.build.source-context-dir":"","io.openshift.build.source-location":"https://github.com/operator-framework/operator-registry","io.openshift.tags":"base rhel7","maintainer":"Odin Team \u003caos-odin\u003e","name":"ubi7","release":"178","summary":"Operator Registry runs in a Kubernetes or OpenShift cluster to provide operator catalog data to Operator Lifecycle Manager.","url":"https://access.redhat.com/containers/#/registry.access.redhat.com/ubi7/images/7.7-178","vcs-ref":"36ee3a437614ca1cb781465e590613e01556e756","vcs-type":"git","vcs-url":"https://github.com/operator-framework/operator-registry","vendor":"Red Hat, Inc.","version":"7.7"}},"architecture":"amd64","size":173262725,"rootfs":{"type":"layers","diff_ids":["sha256:2cab4440f9070965c8454792eee4b122bce5939cc6d21e554469f01e71712a89","sha256:64f2bd9f473b0ae150f07dc65aafb90beeec7ebaa722625055cb8c1ddede0b09","sha256:bd48b9cc9f65a873bde992c97649ace2895af41307502429527f35f1bebda2c0","sha256:86e61694888a7971fcd711f84b1eff117844813adb4ed388d34bdb2d347fd33d","sha256:16b3bcd2591fdcfd8484713c597a4031947bea70cebf67943eb26540dd1421c6","sha256:626e05d48913e4ac8b16fc91167f23a7e6ee0a00390e20a5528a81d733541bc8"]},"history":[{"created":"2019-10-08T12:53:07.414065092Z","comment":"Imported from -"},{"created":"2019-10-08T12:53:18.913068Z"},{"created":"2019-11-08T05:23:58.162756074Z","created_by":"#(imagebuilder)\nsleep 86400"},{"created":"2019-11-08T05:25:30.489079659Z","created_by":"#(imagebuilder)\nsleep 86400"},{"created":"2020-03-10T21:35:37.089968212Z","created_by":"#(imagebuilder)\nsleep 86400"},{"created":"2020-04-09T08:03:10.051942685+08:00"}],"os":"linux"} I0409 08:15:00.536917 11471 extract.go:448] Extracting from layer: distribution.Descriptor{MediaType:"application/vnd.docker.image.rootfs.diff.tar.gzip", Size:76227246, Digest:"sha256:1c9f515fc6ab2b7ebfcaffd8af681b68869d78a3b19c69e87c296363ab1bc2fe", URLs:[]string(nil), Annotations:map[string]string(nil), Platform:(*v1.Platform)(nil)} I0409 08:15:00.536966 11471 extract.go:470] Extracting layer sha256:1c9f515fc6ab2b7ebfcaffd8af681b68869d78a3b19c69e87c296363ab1bc2fe with options &archive.TarOptions{IncludeFiles:[]string(nil), ExcludePatterns:[]string(nil), Compression:0, NoLchown:false, ChownOpts:(*idtools.IDPair)(nil), IncludeSourceDir:false, WhiteoutFormat:0, NoOverwriteDirNonDir:false, RebaseNames:map[string]string(nil), InUserNS:false, Chown:false, AlterHeaders:extract.alterations{extract.removePermissions{}}} I0409 08:15:00.537371 11471 round_trippers.go:423] curl -k -v -XGET -H "Accept-Encoding: identity" 'https://registry.ocp4.example.com:5443/v2/catalog/redhat-operators/blobs/sha256:1c9f515fc6ab2b7ebfcaffd8af681b68869d78a3b19c69e87c296363ab1bc2fe' I0409 08:15:00.553885 11471 round_trippers.go:443] GET https://registry.ocp4.example.com:5443/v2/catalog/redhat-operators/blobs/sha256:1c9f515fc6ab2b7ebfcaffd8af681b68869d78a3b19c69e87c296363ab1bc2fe 200 OK in 16 milliseconds I0409 08:15:00.553901 11471 round_trippers.go:449] Response Headers: I0409 08:15:00.553905 11471 round_trippers.go:452] Cache-Control: max-age=31536000 I0409 08:15:00.553909 11471 round_trippers.go:452] Content-Type: application/octet-stream I0409 08:15:00.553912 11471 round_trippers.go:452] Docker-Content-Digest: sha256:1c9f515fc6ab2b7ebfcaffd8af681b68869d78a3b19c69e87c296363ab1bc2fe I0409 08:15:00.553915 11471 round_trippers.go:452] Docker-Distribution-Api-Version: registry/2.0 I0409 08:15:00.553919 11471 round_trippers.go:452] Etag: "sha256:1c9f515fc6ab2b7ebfcaffd8af681b68869d78a3b19c69e87c296363ab1bc2fe" I0409 08:15:00.553922 11471 round_trippers.go:452] Content-Length: 76227246 I0409 08:15:00.553925 11471 round_trippers.go:452] Date: Thu, 09 Apr 2020 00:15:00 GMT I0409 08:15:00.553928 11471 round_trippers.go:452] Accept-Ranges: bytes I0409 08:15:03.386995 11471 extract.go:448] Extracting from layer: distribution.Descriptor{MediaType:"application/vnd.docker.image.rootfs.diff.tar.gzip", Size:1573, Digest:"sha256:1d2c4ce43b78cb9a97ede7f19ad1406a43ee50532568bda660193e4a404b424f", URLs:[]string(nil), Annotations:map[string]string(nil), Platform:(*v1.Platform)(nil)} I0409 08:15:03.387079 11471 extract.go:470] Extracting layer sha256:1d2c4ce43b78cb9a97ede7f19ad1406a43ee50532568bda660193e4a404b424f with options &archive.TarOptions{IncludeFiles:[]string(nil), ExcludePatterns:[]string(nil), Compression:0, NoLchown:false, ChownOpts:(*idtools.IDPair)(nil), IncludeSourceDir:false, WhiteoutFormat:0, NoOverwriteDirNonDir:false, RebaseNames:map[string]string(nil), InUserNS:false, Chown:false, AlterHeaders:extract.alterations{extract.removePermissions{}}} I0409 08:15:03.387144 11471 round_trippers.go:423] curl -k -v -XGET -H "Accept-Encoding: identity" 'https://registry.ocp4.example.com:5443/v2/catalog/redhat-operators/blobs/sha256:1d2c4ce43b78cb9a97ede7f19ad1406a43ee50532568bda660193e4a404b424f' I0409 08:15:03.393053 11471 round_trippers.go:443] GET https://registry.ocp4.example.com:5443/v2/catalog/redhat-operators/blobs/sha256:1d2c4ce43b78cb9a97ede7f19ad1406a43ee50532568bda660193e4a404b424f 200 OK in 5 milliseconds I0409 08:15:03.393073 11471 round_trippers.go:449] Response Headers: I0409 08:15:03.393077 11471 round_trippers.go:452] Date: Thu, 09 Apr 2020 00:15:03 GMT I0409 08:15:03.393081 11471 round_trippers.go:452] Accept-Ranges: bytes I0409 08:15:03.393090 11471 round_trippers.go:452] Cache-Control: max-age=31536000 I0409 08:15:03.393093 11471 round_trippers.go:452] Content-Type: application/octet-stream I0409 08:15:03.393096 11471 round_trippers.go:452] Docker-Content-Digest: sha256:1d2c4ce43b78cb9a97ede7f19ad1406a43ee50532568bda660193e4a404b424f I0409 08:15:03.393100 11471 round_trippers.go:452] Docker-Distribution-Api-Version: registry/2.0 I0409 08:15:03.393103 11471 round_trippers.go:452] Etag: "sha256:1d2c4ce43b78cb9a97ede7f19ad1406a43ee50532568bda660193e4a404b424f" I0409 08:15:03.393106 11471 round_trippers.go:452] Content-Length: 1573 I0409 08:15:03.393816 11471 extract.go:448] Extracting from layer: distribution.Descriptor{MediaType:"application/vnd.docker.image.rootfs.diff.tar.gzip", Size:402, Digest:"sha256:70e9a2db49ca121dc3fe0385f0139b89f1e75ad5144ff128d7102901eae8d283", URLs:[]string(nil), Annotations:map[string]string(nil), Platform:(*v1.Platform)(nil)} I0409 08:15:03.393879 11471 extract.go:470] Extracting layer sha256:70e9a2db49ca121dc3fe0385f0139b89f1e75ad5144ff128d7102901eae8d283 with options &archive.TarOptions{IncludeFiles:[]string(nil), ExcludePatterns:[]string(nil), Compression:0, NoLchown:false, ChownOpts:(*idtools.IDPair)(nil), IncludeSourceDir:false, WhiteoutFormat:0, NoOverwriteDirNonDir:false, RebaseNames:map[string]string(nil), InUserNS:false, Chown:false, AlterHeaders:extract.alterations{extract.removePermissions{}}} I0409 08:15:03.393930 11471 round_trippers.go:423] curl -k -v -XGET -H "Accept-Encoding: identity" 'https://registry.ocp4.example.com:5443/v2/catalog/redhat-operators/blobs/sha256:70e9a2db49ca121dc3fe0385f0139b89f1e75ad5144ff128d7102901eae8d283' I0409 08:15:03.402422 11471 round_trippers.go:443] GET https://registry.ocp4.example.com:5443/v2/catalog/redhat-operators/blobs/sha256:70e9a2db49ca121dc3fe0385f0139b89f1e75ad5144ff128d7102901eae8d283 200 OK in 8 milliseconds I0409 08:15:03.402444 11471 round_trippers.go:449] Response Headers: I0409 08:15:03.402449 11471 round_trippers.go:452] Content-Length: 402 I0409 08:15:03.402453 11471 round_trippers.go:452] Date: Thu, 09 Apr 2020 00:15:03 GMT I0409 08:15:03.402456 11471 round_trippers.go:452] Accept-Ranges: bytes I0409 08:15:03.402459 11471 round_trippers.go:452] Cache-Control: max-age=31536000 I0409 08:15:03.402462 11471 round_trippers.go:452] Content-Type: application/octet-stream I0409 08:15:03.402465 11471 round_trippers.go:452] Docker-Content-Digest: sha256:70e9a2db49ca121dc3fe0385f0139b89f1e75ad5144ff128d7102901eae8d283 I0409 08:15:03.402469 11471 round_trippers.go:452] Docker-Distribution-Api-Version: registry/2.0 I0409 08:15:03.402472 11471 round_trippers.go:452] Etag: "sha256:70e9a2db49ca121dc3fe0385f0139b89f1e75ad5144ff128d7102901eae8d283" I0409 08:15:03.403050 11471 extract.go:448] Extracting from layer: distribution.Descriptor{MediaType:"application/vnd.docker.image.rootfs.diff.tar.gzip", Size:8202354, Digest:"sha256:fda2c16288874857ab0dc630c3c6822dcd6135927a89e8f1df8765f8dd941c72", URLs:[]string(nil), Annotations:map[string]string(nil), Platform:(*v1.Platform)(nil)} I0409 08:15:03.403114 11471 extract.go:470] Extracting layer sha256:fda2c16288874857ab0dc630c3c6822dcd6135927a89e8f1df8765f8dd941c72 with options &archive.TarOptions{IncludeFiles:[]string(nil), ExcludePatterns:[]string(nil), Compression:0, NoLchown:false, ChownOpts:(*idtools.IDPair)(nil), IncludeSourceDir:false, WhiteoutFormat:0, NoOverwriteDirNonDir:false, RebaseNames:map[string]string(nil), InUserNS:false, Chown:false, AlterHeaders:extract.alterations{extract.removePermissions{}}} I0409 08:15:03.403154 11471 round_trippers.go:423] curl -k -v -XGET -H "Accept-Encoding: identity" 'https://registry.ocp4.example.com:5443/v2/catalog/redhat-operators/blobs/sha256:fda2c16288874857ab0dc630c3c6822dcd6135927a89e8f1df8765f8dd941c72' I0409 08:15:03.425538 11471 round_trippers.go:443] GET https://registry.ocp4.example.com:5443/v2/catalog/redhat-operators/blobs/sha256:fda2c16288874857ab0dc630c3c6822dcd6135927a89e8f1df8765f8dd941c72 200 OK in 22 milliseconds I0409 08:15:03.425561 11471 round_trippers.go:449] Response Headers: I0409 08:15:03.425566 11471 round_trippers.go:452] Date: Thu, 09 Apr 2020 00:15:03 GMT I0409 08:15:03.425570 11471 round_trippers.go:452] Accept-Ranges: bytes I0409 08:15:03.425573 11471 round_trippers.go:452] Cache-Control: max-age=31536000 I0409 08:15:03.425577 11471 round_trippers.go:452] Content-Type: application/octet-stream I0409 08:15:03.425580 11471 round_trippers.go:452] Docker-Content-Digest: sha256:fda2c16288874857ab0dc630c3c6822dcd6135927a89e8f1df8765f8dd941c72 I0409 08:15:03.425584 11471 round_trippers.go:452] Docker-Distribution-Api-Version: registry/2.0 I0409 08:15:03.425587 11471 round_trippers.go:452] Etag: "sha256:fda2c16288874857ab0dc630c3c6822dcd6135927a89e8f1df8765f8dd941c72" I0409 08:15:03.425623 11471 round_trippers.go:452] Content-Length: 8202354 I0409 08:15:03.713526 11471 extract.go:448] Extracting from layer: distribution.Descriptor{MediaType:"application/vnd.docker.image.rootfs.diff.tar.gzip", Size:87195673, Digest:"sha256:432ce72d7eae54fd18a984bc0e15f81d42c2e25fe46df5b428883829db772125", URLs:[]string(nil), Annotations:map[string]string(nil), Platform:(*v1.Platform)(nil)} I0409 08:15:03.713597 11471 extract.go:470] Extracting layer sha256:432ce72d7eae54fd18a984bc0e15f81d42c2e25fe46df5b428883829db772125 with options &archive.TarOptions{IncludeFiles:[]string(nil), ExcludePatterns:[]string(nil), Compression:0, NoLchown:false, ChownOpts:(*idtools.IDPair)(nil), IncludeSourceDir:false, WhiteoutFormat:0, NoOverwriteDirNonDir:false, RebaseNames:map[string]string(nil), InUserNS:false, Chown:false, AlterHeaders:extract.alterations{extract.removePermissions{}}} I0409 08:15:03.713634 11471 round_trippers.go:423] curl -k -v -XGET -H "Accept-Encoding: identity" 'https://registry.ocp4.example.com:5443/v2/catalog/redhat-operators/blobs/sha256:432ce72d7eae54fd18a984bc0e15f81d42c2e25fe46df5b428883829db772125' I0409 08:15:03.726324 11471 round_trippers.go:443] GET https://registry.ocp4.example.com:5443/v2/catalog/redhat-operators/blobs/sha256:432ce72d7eae54fd18a984bc0e15f81d42c2e25fe46df5b428883829db772125 200 OK in 12 milliseconds I0409 08:15:03.726342 11471 round_trippers.go:449] Response Headers: I0409 08:15:03.726346 11471 round_trippers.go:452] Cache-Control: max-age=31536000 I0409 08:15:03.726349 11471 round_trippers.go:452] Content-Type: application/octet-stream I0409 08:15:03.726353 11471 round_trippers.go:452] Docker-Content-Digest: sha256:432ce72d7eae54fd18a984bc0e15f81d42c2e25fe46df5b428883829db772125 I0409 08:15:03.726356 11471 round_trippers.go:452] Docker-Distribution-Api-Version: registry/2.0 I0409 08:15:03.726359 11471 round_trippers.go:452] Etag: "sha256:432ce72d7eae54fd18a984bc0e15f81d42c2e25fe46df5b428883829db772125" I0409 08:15:03.726362 11471 round_trippers.go:452] Content-Length: 87195673 I0409 08:15:03.726365 11471 round_trippers.go:452] Date: Thu, 09 Apr 2020 00:15:03 GMT I0409 08:15:03.726368 11471 round_trippers.go:452] Accept-Ranges: bytes I0409 08:15:05.692225 11471 extract.go:448] Extracting from layer: distribution.Descriptor{MediaType:"application/vnd.docker.image.rootfs.diff.tar.gzip", Size:1635477, Digest:"sha256:ebd3c8e8b51b82d3741437f68163d1b0e084ab5e54606aa473120652557ad099", URLs:[]string(nil), Annotations:map[string]string(nil), Platform:(*v1.Platform)(nil)} I0409 08:15:05.692318 11471 extract.go:470] Extracting layer sha256:ebd3c8e8b51b82d3741437f68163d1b0e084ab5e54606aa473120652557ad099 with options &archive.TarOptions{IncludeFiles:[]string(nil), ExcludePatterns:[]string(nil), Compression:0, NoLchown:false, ChownOpts:(*idtools.IDPair)(nil), IncludeSourceDir:false, WhiteoutFormat:0, NoOverwriteDirNonDir:false, RebaseNames:map[string]string(nil), InUserNS:false, Chown:false, AlterHeaders:extract.alterations{extract.removePermissions{}}} I0409 08:15:05.692388 11471 round_trippers.go:423] curl -k -v -XGET -H "Accept-Encoding: identity" 'https://registry.ocp4.example.com:5443/v2/catalog/redhat-operators/blobs/sha256:ebd3c8e8b51b82d3741437f68163d1b0e084ab5e54606aa473120652557ad099' I0409 08:15:05.701144 11471 round_trippers.go:443] GET https://registry.ocp4.example.com:5443/v2/catalog/redhat-operators/blobs/sha256:ebd3c8e8b51b82d3741437f68163d1b0e084ab5e54606aa473120652557ad099 200 OK in 8 milliseconds I0409 08:15:05.701163 11471 round_trippers.go:449] Response Headers: I0409 08:15:05.701168 11471 round_trippers.go:452] Docker-Content-Digest: sha256:ebd3c8e8b51b82d3741437f68163d1b0e084ab5e54606aa473120652557ad099 I0409 08:15:05.701172 11471 round_trippers.go:452] Docker-Distribution-Api-Version: registry/2.0 I0409 08:15:05.701177 11471 round_trippers.go:452] Etag: "sha256:ebd3c8e8b51b82d3741437f68163d1b0e084ab5e54606aa473120652557ad099" I0409 08:15:05.701181 11471 round_trippers.go:452] Content-Length: 1635477 I0409 08:15:05.701185 11471 round_trippers.go:452] Date: Thu, 09 Apr 2020 00:15:05 GMT I0409 08:15:05.701189 11471 round_trippers.go:452] Accept-Ranges: bytes I0409 08:15:05.701193 11471 round_trippers.go:452] Cache-Control: max-age=31536000 I0409 08:15:05.701197 11471 round_trippers.go:452] Content-Type: application/octet-stream I0409 08:15:05.762594 11471 workqueue.go:143] about to send work queue error: <nil> I0409 08:15:05.762642 11471 mirror.go:231] wrote database to /tmp/080333852/bundles.db I0409 08:15:05.762862 11471 workqueue.go:54] worker 0 stopping I0409 08:15:05.762884 11471 workqueue.go:54] worker 3 stopping I0409 08:15:05.762900 11471 workqueue.go:54] worker 2 stopping I0409 08:15:05.762910 11471 workqueue.go:60] work queue exiting I0409 08:15:05.762919 11471 workqueue.go:54] worker 1 stopping W0409 08:15:05.771733 11471 mirror.go:258] errors during mirroring. the full contents of the catalog may not have been mirrored: couldn't parse image for mirroring (), skipping mirror: invalid reference format I0409 08:15:05.792636 11471 mirror.go:329] wrote mirroring manifests to redhat-operators-manifests
Let me add some info. With catalog source holding only a subset of the full catalog (few operators), which is functioning ok, then the oc adm catalog mirror fails with the same error message. The only wa, is to mirror each image by "oc image mirror..."
It looks like a problem with the image references in certain operators. These six operators include entries in their clusterserviceversion's .spec.relatedImages field that are missing an "image" value: $ sqlite3 /tmp/db-579446650/bundles.db 'select * from related_image' | grep '^|' |3scale-operator.v0.5.0 |3scale-operator.v0.5.1 |3scale-operator.v0.5.2 |amqstreams.v1.4.0 |amqstreams.v1.4.1 |serverless-operator.v1.6.0 Looking at 3scale-operator.v0.5.0.clusterserviceversion.yaml: spec: relatedImages: - name: apicast-gateway-rhel8 image: registry.redhat.io/3scale-amp2/apicast-gateway-rhel8@sha256:21be62a6557846337dc0cf764be63442718fab03b95c198a301363886a9e74f9 - name: backend-rhel7 image: registry.redhat.io/3scale-amp2/backend-rhel7@sha256:ea8a31345d3c2a56b02998b019db2e17f61eeaa26790a07962d5e3b66032d8e5 - name: system-rhel7 image: registry.redhat.io/3scale-amp2/system-rhel7@sha256:93819c324831353bb8f7cb6e9910694b88609c3a20d4c1b9a22d9c2bbfbad16f - name: zync-rhel7 image: registry.redhat.io/3scale-amp2/zync-rhel7@sha256:f4d5c1fdebe306f4e891ddfc4d3045a622d2f01db21ecfc9397cab25c9baa91a - name: memcached-rhel7 image: registry.redhat.io/3scale-amp2/memcached-rhel7@sha256:ff5f3d2d131631d5db8985a5855ff4607e91f0aa86d07dafdcec4f7da13c9e05 - name: redis-32-rhel7 value: registry.redhat.io/rhscl/redis-32-rhel7@sha256:a9bdf52384a222635efc0284db47d12fbde8c3d0fcb66517ba8eefad1d4e9dc9 - name: mysql-57-rhel7 value: registry.redhat.io/rhscl/mysql-57-rhel7@sha256:9a781abe7581cc141e14a7e404ec34125b3e89c008b14f4e7b41e094fd3049fe - name: postgresql-10-rhel7 value: registry.redhat.io/rhscl/postgresql-10-rhel7@sha256:de3ab628b403dc5eed986a7f392c34687bddafee7bdfccfd65cecf137ade3dfd Some of the relatedImages entries accidentally specify a key "value" rather than "image". Those entries are being deserialized as though they specify the empty string as the value for "image": $ sqlite3 /tmp/db-579446650/bundles.db 'select * from related_image where operatorbundle_name="3scale-operator.v0.5.0"' registry.redhat.io/3scale-amp2/3scale-rhel7-operator@sha256:2ba16314ee046b3c3814fe4e356b728da6853743bd72f8651e1a338e8bbf4f81|3scale-operator.v0.5.0 registry.redhat.io/3scale-amp2/system-rhel7@sha256:93819c324831353bb8f7cb6e9910694b88609c3a20d4c1b9a22d9c2bbfbad16f|3scale-operator.v0.5.0 registry.redhat.io/3scale-amp2/zync-rhel7@sha256:f4d5c1fdebe306f4e891ddfc4d3045a622d2f01db21ecfc9397cab25c9baa91a|3scale-operator.v0.5.0 registry.redhat.io/3scale-amp2/memcached-rhel7@sha256:ff5f3d2d131631d5db8985a5855ff4607e91f0aa86d07dafdcec4f7da13c9e05|3scale-operator.v0.5.0 |3scale-operator.v0.5.0 registry.redhat.io/3scale-amp2/apicast-gateway-rhel8@sha256:21be62a6557846337dc0cf764be63442718fab03b95c198a301363886a9e74f9|3scale-operator.v0.5.0 registry.redhat.io/3scale-amp2/backend-rhel7@sha256:ea8a31345d3c2a56b02998b019db2e17f61eeaa26790a07962d5e3b66032d8e5|3scale-operator.v0.5.0 So the error message is correct here -- the image reference "" is invalid. This should probably be failing at build time.
We fixed this for the serverless-operator in 1.7.0, although we didn't have any plans to update the older serverless operator 1.6.0. I'm assuming these are non-fatal errors and the mirroring process just skips over the missing image values? This hasn't impacted our ability to mirror our images in the serverless 1.7.0 release, for example.
That's right. At one point the entire catalog mirror operation would fail if one source image could not be found, but that has since been fixed (https://bugzilla.redhat.com/show_bug.cgi?id=1790798).
The only remaining work here is to tighten validation for bundle validate, so that these issues can be easily detected by the pipelines. Because of that I have lowered severity and moved to 4.6.
Moving to assigned and marking for upcoming sprint.
+1 I'm having the same error message on `oc v4.4.6`; `couldn't parse image for mirroring (), skipping mirror: invalid reference format` when running `oc adm catalog mirror`. Affected customers: Datacom New Zealand
This message does not indicate that the entire catalog could not be mirrored, only that an operator has provided an invalid related image reference. Other operators in the same catalog should still mirror successfully. If a specific, necessary image is not being mirrored due to this issue, the image reference needs to be corrected in a new version of the operator that requires it. This issue only tracks the addition of new validation to help operator authors identify this mistake earlier.
Closing this issue. Empty relatedImage refs are ignored in newer versions of oc (https://github.com/openshift/oc/blob/4ce165465c4a436cffd0722826c138691b3c1919/pkg/cli/admin/catalog/mirrorer.go#L116), and this warning does not indicate a mirroring failure.