Bug 1833019 - need up-to-date messages from imagestreaimport InternalErrors surface in imagestream status
Summary: need up-to-date messages from imagestreaimport InternalErrors surface in imag...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: ImageStreams
Version: 4.4
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: 4.3.z
Assignee: Gabe Montero
QA Contact: XiuJuan Wang
URL:
Whiteboard:
Depends On: 1788700
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-05-07 16:35 UTC by OpenShift BugZilla Robot
Modified: 2020-05-27 17:01 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: stale error messages would exist on failed imagestreamimports Consequence: users would be mislead on what current imagestreamimport problems were because errors from prior import failures were still the only ones surfaced Fix: the logic for updating imagestreamimport error messages was enhanced to better determine that successive errors were in fact from different root causes, and hence update the error messages when appropriate Result: users get better guidance from imagestreamimport errors after successive failed attempts on what is needed to fix problems
Clone Of:
Environment:
Last Closed: 2020-05-27 17:00:46 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift openshift-apiserver pull 106 0 None closed Bug 1833019: include ISI error message changes when determining new generation need 2020-05-22 17:49:05 UTC
Red Hat Product Errata RHBA-2020:2184 0 None None None 2020-05-27 17:01:01 UTC

Description OpenShift BugZilla Robot 2020-05-07 16:35:38 UTC
This is a clone of Bug #1788700. This is the description of that bug:
From Oleg's and my recent debug foray with a customer disconnected install 

1) On an import error, the bump of the condition does not take into account the message as the comment notes at 

https://github.com/openshift/openshift-apiserver/blob/master/pkg/image/apiserver/internalimageutil/helpers.go#L470-L479

Namely

// HasTagCondition returns true if the specified image stream tag has a condition with the same type, status, and
// reason (does not check generation, date, or message).
func HasTagCondition(stream *imageapi.ImageStream, tag string, condition imageapi.TagEventCondition) bool {
	for _, existing := range stream.Status.Tags[tag].Conditions {
		if condition.Type == existing.Type && condition.Status == existing.Status && condition.Reason == existing.Reason {
			return true
		}
	}
	return false
}

So we could do things like change the image registry used, and if it fails in the same way, we won't bump the condition from the prior image registry used.

The above check should be updated to look at condition.Message as well.

Also, Oleg found that the error logging at https://github.com/openshift/openshift-controller-manager/blob/master/pkg/image/controller/imagestream_controller.go#L316

Namely

		klog.V(4).Infof("Import stream %s/%s partial=%t error: %v", stream.Namespace, stream.Name, partial, err)

masks error by default.

Perhaps that was done in a case where imports were failing for a while, and the log got chatty ... but we should revisit bumping down the log level here.

Comment 5 XiuJuan Wang 2020-05-18 10:50:14 UTC
Should test with 4.3.0-0.nightly-2020-05-17-223233, and pass . Please ingore comment #4
1) configure the samplesRegistry to an initial registry that did not have the ruby images

2) then change samplesRegistry to ec2-3-133-161-167.us-east-2.compute.amazonaws.com:5000 ,  the perl images are not there as well.
 oc get is perl  -n openshift -o yaml 
apiVersion: image.openshift.io/v1
kind: ImageStream
metadata:
  annotations:
    openshift.io/display-name: Perl
    openshift.io/image.dockerRepositoryCheck: "2020-05-18T09:29:01Z"
    samples.operator.openshift.io/version: 4.3.0-0.nightly-2020-05-17-223233
  creationTimestamp: "2020-05-18T08:46:37Z"
  generation: 4
  labels:
    samples.operator.openshift.io/managed: "true"
  name: perl
  namespace: openshift
  resourceVersion: "34026"
  selfLink: /apis/image.openshift.io/v1/namespaces/openshift/imagestreams/perl
  uid: 2203d85b-7d65-4e71-9634-feff6d2e97d9
spec:
  lookupPolicy:
    local: false
  tags:
  - annotations:
      description: Build and run Perl 5.24 applications on RHEL 7. For more information
        about using this builder image, including OpenShift considerations, see https://github.com/sclorg/s2i-perl-container/blob/master/5.24/README.md.
      iconClass: icon-perl
      openshift.io/display-name: Perl 5.24
      openshift.io/provider-display-name: Red Hat, Inc.
      sampleRepo: https://github.com/sclorg/dancer-ex.git
      supports: perl:5.24,perl
      tags: builder,perl
      version: "5.24"
    from:
      kind: DockerImage
      name: ec2-3-133-161-167.us-east-2.compute.amazonaws.com:5000/rhscl/perl-524-rhel7:latest
    generation: 4
    importPolicy: {}
    name: "5.24"
    referencePolicy:
      type: Local
  - annotations:
      description: Build and run Perl 5.26 applications on RHEL 7. For more information
        about using this builder image, including OpenShift considerations, see https://github.com/sclorg/s2i-perl-container/blob/master/5.26/README.md.
      iconClass: icon-perl
      openshift.io/display-name: Perl 5.26
      openshift.io/provider-display-name: Red Hat, Inc.
      sampleRepo: https://github.com/sclorg/dancer-ex.git
      supports: perl:5.26,perl
      tags: builder,perl
      version: "5.26"
    from:
      kind: DockerImage
      name: ec2-3-133-161-167.us-east-2.compute.amazonaws.com:5000/rhscl/perl-526-rhel7:latest
    generation: 4
    importPolicy: {}
    name: "5.26"
    referencePolicy:
      type: Local
  - annotations:
      description: |-
        Build and run Perl applications on RHEL 7. For more information about using this builder image, including OpenShift considerations, see https://github.com/sclorg/s2i-perl-container/blob/master/5.26/README.md.

        WARNING: By selecting this tag, your application will automatically update to use the latest version of Perl available on OpenShift, including major version updates.
      iconClass: icon-perl
      openshift.io/display-name: Perl (Latest)
      openshift.io/provider-display-name: Red Hat, Inc.
      sampleRepo: https://github.com/sclorg/dancer-ex.git
      supports: perl
      tags: builder,perl
    from:
      kind: ImageStreamTag
      name: "5.26"
    generation: 1
    importPolicy: {}
    name: latest
    referencePolicy:
      type: Local
status:
  dockerImageRepository: image-registry.openshift-image-registry.svc:5000/openshift/perl
  tags:
  - conditions:
    - generation: 4
      lastTransitionTime: "2020-05-18T09:29:01Z"
      message: dockerimage.image.openshift.io "ec2-3-133-161-167.us-east-2.compute.amazonaws.com:5000/rhscl/perl-524-rhel7:latest"
        not found
      reason: NotFound
      status: "False"
      type: ImportSuccess
    items: null
    tag: "5.24"
  - conditions:
    - generation: 4
      lastTransitionTime: "2020-05-18T09:29:01Z"
      message: dockerimage.image.openshift.io "ec2-3-133-161-167.us-east-2.compute.amazonaws.com:5000/rhscl/perl-526-rhel7:latest"
        not found
      reason: NotFound
      status: "False"
      type: ImportSuccess
    items: null
    tag: "5.26"

Comment 7 errata-xmlrpc 2020-05-27 17:00:46 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:2184


Note You need to log in before you can comment on or make changes to this bug.