Bug 1874053

Created attachment 1713144 [details]
output of oc adm catalog mirror

Description of problem:
I have a disconnected OCP 4.6.fc.2 cluster and I'm mirroring operators from registry-proxy.engineering.redhat.com/rh-osbs/iib-pub-pending:v4.5. Mirroring works fine but installation of operator fails.

Version-Release number of selected component (if applicable):
OCP 4.6.fc.2

How reproducible:
Always

Steps to Reproduce:
1. install disconnected cluster with mirror registry following https://docs.openshift.com/container-platform/4.5/installing/install_config/installing-restricted-networks-preparations.html
2. disable default sources: oc patch operatorhub.config.openshift.io/cluster -p='{"spec":{"disableAllDefaultSources":true}}' --type=merge
3. on a host with the mirror registry
a) add 'registry-proxy.engineering.redhat.com','registry.stage.redhat.io' to insecure registries in /etc/containers/registries.conf
b) run podman login for registry.stage.redhat.io, registry.redhat.io and mirror registry and copy auth.json to current dir
c) mirror catalog with --manifests-only: oc adm catalog mirror  registry-proxy.engineering.redhat.com/rh-osbs/iib-pub-pending:v4.5 `hostname`:55555/olm -a auth.json --insecure --manifests-only
d) get only kiali and jaeger from mapping.txt: cat iib-pub-pending-manifests/mapping.txt | grep -E 'kiali|jaeger' > map.txt
e) mirror images from map.txt: oc image mirror -a auth.json --insecure -f map.txt --filter-by-os=/*
f) previous fails because of registry.redhat.io/openshift-service-mesh/kiali-rhel7 manifest sha256:29acbaf2c5c6909b0a9596060a40a2f304047c361422442fccc674a535fcee6a does not exist in registry.redhat.io -> change it to following in map.txt:
registry-proxy.engineering.redhat.com/rh-osbs/openshift-service-mesh-kiali-rhel7@sha256:29acbaf2c5c6909b0a9596060a40a2f304047c361422442fccc674a535fcee6a=bastion.fbr-46-2-disc.maistra.upshift.redhat.com:55555/olm/openshift-service-mesh-kiali-rhel7:9258190e
g) run step e) again which should be successful
h) mirror IIB itself:
podman inspect --format='{{index .RepoDigests 0}}' registry-proxy.engineering.redhat.com/rh-osbs/iib-pub-pending:v4.5
oc image mirror -a auth.json --insecure registry-proxy.engineering.redhat.com/rh-osbs/iib-pub-pending@sha256:3ea4b0a44c3422219edb7ed5817385ff221be58ffa09ac2ddfc6c331281a42b1 bastion.fbr-46-2-disc.maistra.upshift.redhat.com:55555/olm/my-iib:mytag
i) oc apply -f iib-pub-pending-manifests/imageContentSourcePolicy.yaml
j) wait for the cluster to reconcile
k) create new catalog source with following content:
apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
  name: my-index-catalog
  namespace: openshift-marketplace
spec:
  sourceType: grpc
  image: bastion.fbr-46-2-disc.maistra.upshift.redhat.com:55555/olm/my-iib:mytag
l) go to UI console and install kiali and jaeger operators with all values set to default


Actual results:
Installation never finishes but both operators are marked as installed on Operator Hub page.
Operators are not visible on Installed operators page.
There are no pending pods or errors on Overview page. There is only one event which seems to be related - found more than one head for channel.
No pods are running in redhat-operators NS.
marketplace-operator and my-index-catalog pods are running in openshift-marketplace NS.


Expected results:
Operators are installed.

Additional info:
Attached output of oc adm catalog mirror command, output of oc image mirror command. Content of imageContentSourcePolicy.yaml and edited mapping.txt.