1821783 – openshift-marketplace pods using images not present in release payload.

Bug 1821783 - openshift-marketplace pods using images not present in release payload.

Summary: openshift-marketplace pods using images not present in release payload.

Keywords:
Status:	CLOSED ERRATA
Alias:	None
Product:	OpenShift Container Platform
Classification:	Red Hat
Component:	OLM
Sub Component:
Version:	4.4
Hardware:	Unspecified
OS:	Unspecified
Priority:	low
Severity:	low
Target Milestone:	---
Target Release:	4.4.0
Assignee:	Evan Cordell
QA Contact:	Jian Zhang
Docs Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+	depends on / blocked

Reported:	2020-04-07 15:35 UTC by Luis Sanchez
Modified:	2020-05-08 04:52 UTC (History)
CC List:	1 user (show)
Fixed In Version:
Doc Type:	If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:	2020-05-04 11:48:34 UTC
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)

Links
System	ID	Private	Priority	Status	Summary	Last Updated
Red Hat Product Errata	RHBA-2020:0581	0	None	None	None	2020-05-04 11:49:00 UTC

Description Luis Sanchez 2020-04-07 15:35:44 UTC

openshift-marketplace/certified-operators and openshift-marketplace/redhat-operators pods not using images from release payload after 4.1->4.2->4.3->4.4 upgrade.


buildlog.txt snippet:

[It] [Top Level] [Feature:Platform] Managed cluster should ensure pods use downstream images from our release image with proper ImagePullPolicy [Suite:openshift/conformance/parallel]
  /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/test/extended/operators/images.go:21
Apr  7 06:34:53.134: INFO: Running 'oc --namespace= --kubeconfig= adm release info --pullspecs -o json --registry-config /tmp/image-pull-secret636389395'
Apr  7 06:34:54.536: INFO: Release Info image=registry.svc.ci.openshift.org/ocp/release:4.4.0-0.nightly-2020-04-04-025830
Apr  7 06:34:54.536: INFO: Release Info number of tags 109

Apr  7 06:37:23.038: INFO: Pods found with invalid container images not present in release payload: openshift-marketplace/certified-operators-797cc58648-47qbj/certified-operators image=quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c517ffebffb8f5ffd3cc04f2491270b59b61079ca889be664945e166f90f420
openshift-marketplace/community-operators-849dd49d8d-tl5t7/community-operators image=quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c517ffebffb8f5ffd3cc04f2491270b59b61079ca889be664945e166f90f420
openshift-marketplace/redhat-operators-ffb9b86c6-dr74f/redhat-operators image=quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c517ffebffb8f5ffd3cc04f2491270b59b61079ca889be664945e166f90f420
Apr  7 06:37:23.038: INFO: Pods found with invalid container image pull policy not equal to IfNotPresent: 
Apr  7 06:37:23.038: INFO: Pods with invalid dowstream images: 
Apr  7 06:37:23.039: FAIL: Pods found with invalid container images not present in release payload: openshift-marketplace/certified-operators-797cc58648-47qbj/certified-operators image=quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c517ffebffb8f5ffd3cc04f2491270b59b61079ca889be664945e166f90f420
openshift-marketplace/community-operators-849dd49d8d-tl5t7/community-operators image=quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c517ffebffb8f5ffd3cc04f2491270b59b61079ca889be664945e166f90f420
openshift-marketplace/redhat-operators-ffb9b86c6-dr74f/redhat-operators image=quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8c517ffebffb8f5ffd3cc04f2491270b59b61079ca889be664945e166f90f420


CI Search:

https://search.svc.ci.openshift.org/?search=Pods+found+with+invalid+container+images+not+present+in+release+payload.*+openshift-marketplace.certified-operators&maxAge=48h&context=2&type=bug%2Bjunit

Comment 1 Jian Zhang 2020-04-10 06:24:33 UTC

mac:~ jianzhang$ oc get clusterversion
NAME      VERSION                             AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.4.0-0.nightly-2020-04-09-220855   True        False         168m    Cluster version is 4.4.0-0.nightly-2020-04-09-220855

mac:~ jianzhang$ oc get pods
NAME                                    READY   STATUS    RESTARTS   AGE
certified-operators-7c78cf8594-flg7d    1/1     Running   0          83s
community-operators-5645f84b57-kwsf8    1/1     Running   0          85s
marketplace-operator-59ffc649dd-2h2vk   1/1     Running   0          84m
quay-applications-d65844df4-lq8zw       1/1     Running   0          84m
redhat-marketplace-5c598d8c9-r2gcf      1/1     Running   0          85s
redhat-operators-748f96cbd8-dqbvv       1/1     Running   0          84s
mac:~ jianzhang$ oc exec certified-operators-7c78cf8594-flg7d   -- cat /etc/redhat-release
Red Hat Enterprise Linux Server release 7.8 (Maipo)
mac:~ jianzhang$ oc exec redhat-operators-748f96cbd8-dqbvv    -- cat /etc/redhat-release
Red Hat Enterprise Linux Server release 7.8 (Maipo)
mac:~ jianzhang$ oc exec community-operators-5645f84b57-kwsf8 -- cat /etc/redhat-release
Red Hat Enterprise Linux Server release 7.8 (Maipo)
mac:~ jianzhang$ oc exec marketplace-operator-59ffc649dd-2h2vk -- cat /etc/redhat-release
Red Hat Enterprise Linux Server release 7.8 (Maipo)
mac:~ jianzhang$ oc exec redhat-marketplace-5c598d8c9-r2gcf -- cat /etc/redhat-release
Red Hat Enterprise Linux Server release 7.8 (Maipo)

As we can see, they did use the downstream image on 4.4.0-0.nightly-2020-04-09-220855 cluster, verify it.

Comment 4 errata-xmlrpc 2020-05-04 11:48:34 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:0581

Comment 5 W. Trevor King 2020-05-08 04:52:56 UTC

I'm still seeing these in CI.  Opened bug 1833207 to track, since this one is already CLOSED ERRATA.

Note You need to log in before you can comment on or make changes to this bug.