Bug 1909992
Summary: | Fail to pull the bundle image when using the private index image | ||
---|---|---|---|
Product: | OpenShift Container Platform | Reporter: | Jian Zhang <jiazha> |
Component: | OLM | Assignee: | Anik <anbhatta> |
OLM sub component: | OLM | QA Contact: | Jian Zhang <jiazha> |
Status: | CLOSED ERRATA | Docs Contact: | |
Severity: | medium | ||
Priority: | medium | CC: | anbhatta, dsover, fdeutsch, jian.zhang, krizza, kuiwang |
Version: | 4.7 | ||
Target Milestone: | --- | ||
Target Release: | 4.7.0 | ||
Hardware: | Unspecified | ||
OS: | Unspecified | ||
Whiteboard: | |||
Fixed In Version: | Doc Type: | If docs needed, set a value | |
Doc Text: | Story Points: | --- | |
Clone Of: | Environment: | ||
Last Closed: | 2021-02-24 15:47:55 UTC | Type: | Bug |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: |
Description
Jian Zhang
2020-12-22 09:29:40 UTC
Cluster version is 4.7.0-0.nightly-2021-01-18-000316 [root@preserve-olm-env data]# oc -n openshift-operator-lifecycle-manager exec catalog-operator-6d9d94fdb8-wk2vh -- olm --version OLM version: 0.17.0 git commit: cab348020d3dafccfb7eef5ef4e05f7fe402b544 1, Create a pull secret called "secret-cs" in "openshift-marketplace" namespace. [root@preserve-olm-env data]# oc project Using project "openshift-marketplace" on server "https://api.xxia18shared.qe.devcluster.openshift.com:6443". [root@preserve-olm-env data]# oc extract secret/pull-secret -n openshift-config --confirm .dockerconfigjson [root@preserve-olm-env data]# oc create secret generic secret-cs --from-file=.dockerconfigjson=/root/.docker/config.json --type=kubernetes.io/dockerconfigjson secret/secret-cs created 2, Create a CatalogSource CR to consume a private index image that provides etcdoperator. [root@preserve-olm-env data]# oc create -f cs-private.yaml catalogsource.operators.coreos.com/cs-private created [root@preserve-olm-env data]# cat cs-private.yaml apiVersion: operators.coreos.com/v1alpha1 kind: CatalogSource metadata: name: cs-private namespace: openshift-marketplace spec: displayName: OLM Test publisher: Jian sourceType: grpc secrets: - "secret-cs" image: quay.io/jiazha/upstream-opm-builder:etcdindex updateStrategy: registryPoll: interval: 10m [root@preserve-olm-env data]# oc get packagemanifest|grep etcd etcd Community Operators 3h26m etcd OLM Test 3m5s 3, Subscribe to this etcdoperator. [root@preserve-olm-env data]# cat sub-etcd-cluster.yaml apiVersion: operators.coreos.com/v1alpha1 kind: Subscription metadata: name: etcd-private namespace: openshift-operators spec: channel: clusterwide-alpha installPlanApproval: Automatic name: etcd source: cs-private sourceNamespace: openshift-marketplace startingCSV: etcdoperator.v0.9.4-clusterwide 4, The job that unpacks the bundle images works well. [root@preserve-olm-env data]# oc get job NAME COMPLETIONS DURATION AGE ... d3383223ae0bc9c573b86c2d918fe9f6f9988f5771d0a503da2943d2fa37991 1/1 8s 34m [root@preserve-olm-env data]# oc get pods NAME READY STATUS RESTARTS AGE ... d3383223ae0bc9c573b86c2d918fe9f6f9988f5771d0a503da2943d2fajxtbz 0/1 Completed 0 34m ... [root@preserve-olm-env data]# oc get job d3383223ae0bc9c573b86c2d918fe9f6f9988f5771d0a503da2943d2fa37991 -o yaml|grep imagePullSecrets -A1 f:imagePullSecrets: .: {} -- imagePullSecrets: - name: secret-cs 5, Check if this operator can be installed. [root@preserve-olm-env data]# oc get sub -n openshift-operators NAME PACKAGE SOURCE CHANNEL etcd-private etcd cs-private clusterwide-alpha [root@preserve-olm-env data]# oc get ip -n openshift-operators NAME CSV APPROVAL APPROVED install-lm8d4 etcdoperator.v0.9.4-clusterwide Automatic true [root@preserve-olm-env data]# oc get csv -n openshift-operators NAME DISPLAY VERSION REPLACES PHASE etcdoperator.v0.9.4-clusterwide etcd 0.9.4-clusterwide Installing ... [root@preserve-olm-env data]# oc get job d3383223ae0bc9c573b86c2d918fe9f6f9988f5771d0a503da2943d2fa37991 -o yaml|grep image: f:image: {} f:image: {} f:image: {} image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:56493169715122404a9007e9c087ba36a5851f0cbccebd82c2c0a162ef80fdef image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:452e9c8f942f706eed65c0b726b863a2dc796dc029f9736a1596a2fc23e8b29f image: quay.io/jiazha/upstream-opm-builder:etcdbundle [root@preserve-olm-env data]# oc get csv -n openshift-operators NAME DISPLAY VERSION REPLACES PHASE etcdoperator.v0.9.4-clusterwide etcd 0.9.4-clusterwide Failed [root@preserve-olm-env data]# oc get pods -n openshift-operators NAME READY STATUS RESTARTS AGE etcd-operator-65f8576977-n5txr 0/3 ImagePullBackOff 0 19m [root@preserve-olm-env data]# oc get deployment -n openshift-operators NAME READY UP-TO-DATE AVAILABLE AGE etcd-operator 0/1 1 0 20m [root@preserve-olm-env data]# oc get deployment -n openshift-operators etcd-operator -o yaml|grep imagePullSecrets ... This private operator failed to install since we didn't inject the pull-secret to its deployment in the configmap. [root@preserve-olm-env data]# oc get cm d3383223ae0bc9c573b86c2d918fe9f6f9988f5771d0a503da2943d2fa37991 -o yaml|grep imagePullSecrets [root@preserve-olm-env data]# I tried to add the above pull secret(cs-secret) auth to the pull-secret of "openshift-config". But, it still failed to pull this private image. [root@preserve-olm-env data]# cat .dockerconfigjson | jq --compact-output '.auths["quay.io/jiazha"] |= . + {"auth":"xxx"}' > new_dockerconfigjson [root@preserve-olm-env data]# oc set data secret/pull-secret -n openshift-config --from-file=.dockerconfigjson=new_dockerconfigjson secret/pull-secret data updated [root@preserve-olm-env data]# oc get pods NAME READY STATUS RESTARTS AGE etcd-operator-65f8576977-2xlsb 0/3 ImagePullBackOff 0 9s Normal BackOff <invalid> (x2 over <invalid>) kubelet Back-off pulling image "quay.io/jiazha/upstream-opm-builder@sha256:2cce9f8e95c9b4ce19b9ffbb95298b9ba3c4960dc962030d8bce655f3811adb0" Warning Failed <invalid> (x2 over <invalid>) kubelet Error: ImagePullBackOff I change the Status to ASSIGNED since I think we should have an official solution to install the operator using the private image. Hi Anik, Thanks for your information! I understand, the problem here is that I add this pull-secret to the pull-secret of "openshift-config", but it still doesn't work. Anyway, I think this problem is not related to this bug, verify it first. Updates: adding this pull-secret to the pull-secret of "openshift-config", it works after a few minutes. [root@preserve-olm-env data]# oc project Using project "openshift-operators" on server "https://api.hongli-aw47.qe.devcluster.openshift.com:6443". [root@preserve-olm-env data]# oc get csv NAME DISPLAY VERSION REPLACES PHASE etcdoperator.v0.9.4-clusterwide etcd 0.9.4-clusterwide Installing [root@preserve-olm-env data]# oc get pods NAME READY STATUS RESTARTS AGE etcd-operator-65f8576977-pzbf7 0/3 ImagePullBackOff 0 22s [root@preserve-olm-env data]# oc get pods NAME READY STATUS RESTARTS AGE etcd-operator-65f8576977-8mh4h 3/3 Running 0 154m That's a workaround if users want to install the operator that using the private image. [root@preserve-olm-env data]# cat .dockerconfigjson | jq --compact-output '.auths["quay.io/jiazha"] |= . + {"auth":"xxx"}' > new_dockerconfigjson [root@preserve-olm-env data]# oc set data secret/pull-secret -n openshift-config --from-file=.dockerconfigjson=new_dockerconfigjson secret/pull-secret data updated Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: OpenShift Container Platform 4.7.0 security, bug fix, and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2020:5633 |