Adam - minus the EOL image noise, this is actually another instance of what we talked about yesterday wrt the installer pull credentials not getting pulled in by the build controller to handle TBR images.

But this most likely is a more reproducible case, vs. the e2e you mentioned only failed part of the time.

@Wenjing - you'll notice the image reference is properly resolved from ImageStream status:  Pulling image registry.redhat.io/rhscl/ruby-22-rhel7@sha25...e85960060096b6aff7ee202a5f52e0e18447641b080b1f3879e0901 ...

Closing this as a dup of the other bug.

*** This bug has been marked as a duplicate of bug 1721380 ***

Yes, Gabe! seems pull credential is not used to builder image. Thanks for your analysis.

Have some additional analysis here 

One of the items that is lost when the imagestream spec no longer has the EOL image, even if the EOL images is present in the imagestream status, is the registry pullthrough logic ... i.e. when the IST referencePolicy is "Local" on the IST in the image stream spec.

Rather that doing pull through, it defaults to type:  Source, and attempts to pull from registry.redhat.io directly vs. using the copy in the internal image registry.

The relevant code is at https://github.com/openshift/library-go/blob/master/pkg/image/imageutil/helpers.go#L310-L355

Ben / Adam - are we interested in modifying that code to allow for a tag to only be in status, and enable pull through in that case?

probably not a dupe per discussion.

I think this needs to work or we need to revisit removing those EOL images from our imagestreams.

not advocating this as a long term solution, but a workaround on 4.2 is to re-import the tag so it shows up in the spec:

oc import-image openshift/ruby:2.2 --from="registry.redhat.io/rhscl/ruby-22-rhel7:latest" --confirm=true --insecure=true  --reference-policy='local' -n openshift

for example ... the local ref policy is key ... users can play around with insecure, specific docker tags, as desired.

I'm going to look at augmenting samples opearator to not totally replace the imagestream.spec, and leave existing EOL specs if they have upgraded.

If that proves untenable, we will undo the removal of EOL images.

OK I have a change that is working with local testing, both clean 4.2 install and 4.1 to 4.2 upgrade

PR up shortly

Cluster openshift-samples updated 4.2.0-0.nightly-2019-08-06-062019 and no new build triggered because ruby:2.2 image has no changed in upgraded env, old build still complete, so verified it.

[wewang@Desktop work]$ oc get is ruby -o yaml -n openshift
apiVersion: image.openshift.io/v1
kind: ImageStream
metadata:
  annotations:
    openshift.io/display-name: Ruby
    samples.operator.openshift.io/version: 4.2.0-0.nightly-2019-08-06-062019
  creationTimestamp: "2019-08-06T06:56:25Z"
  generation: 3
  labels:
    samples.operator.openshift.io/managed: "true"
  name: ruby
  namespace: openshift
  resourceVersion: "65016"
  selfLink: /apis/image.openshift.io/v1/namespaces/openshift/imagestreams/ruby
  uid: 4e5e5360-b817-11e9-ac6e-0a580a810009
spec:
  lookupPolicy:
    local: false
  tags:
  - annotations:
      description: Build and run Ruby 2.0 applications on RHEL 7. For more information
        about using this builder image, including OpenShift considerations, see https://github.com/sclorg/s2i-ruby-container/tree/master/2.0/README.md.
      iconClass: icon-ruby
      openshift.io/display-name: Ruby 2.0
      openshift.io/provider-display-name: Red Hat, Inc.
      sampleRepo: https://github.com/sclorg/ruby-ex.git
      supports: ruby:2.0,ruby
      tags: hidden,builder,ruby
      version: "2.0"
    from:
      kind: DockerImage
      name: registry.redhat.io/openshift3/ruby-20-rhel7:latest
    generation: 2
    importPolicy: {}
    name: "2.0"
    referencePolicy:
      type: Local
  - annotations:
      description: Build and run Ruby 2.2 applications on RHEL 7. For more information
        about using this builder image, including OpenShift considerations, see https://github.com/sclorg/s2i-ruby-container/tree/master/2.2/README.md.
      iconClass: icon-ruby
      openshift.io/display-name: Ruby 2.2
      openshift.io/provider-display-name: Red Hat, Inc.
      sampleRepo: https://github.com/sclorg/ruby-ex.git
      supports: ruby:2.2,ruby
      tags: hidden,builder,ruby
      version: "2.2"
    from:
      kind: DockerImage
      name: registry.redhat.io/rhscl/ruby-22-rhel7:latest
    generation: 2
    importPolicy: {}
    name: "2.2"
    referencePolicy:
      type: Local
  - annotations:
      description: Build and run Ruby 2.3 applications on RHEL 7. For more information
        about using this builder image, including OpenShift considerations, see https://github.com/sclorg/s2i-ruby-container/blob/master/2.3/README.md.
      iconClass: icon-ruby
      openshift.io/display-name: Ruby 2.3
      openshift.io/provider-display-name: Red Hat, Inc.
      sampleRepo: https://github.com/sclorg/ruby-ex.git
      supports: ruby:2.3,ruby
      tags: hidden,builder,ruby
      version: "2.3"
    from:
      kind: DockerImage
      name: registry.redhat.io/rhscl/ruby-23-rhel7:latest
    generation: 2
    importPolicy: {}
    name: "2.3"
    referencePolicy:
      type: Local
  - annotations:
      description: Build and run Ruby 2.4 applications on RHEL 7. For more information
        about using this builder image, including OpenShift considerations, see https://github.com/sclorg/s2i-ruby-container/blob/master/2.4/README.md.
      iconClass: icon-ruby
      openshift.io/display-name: Ruby 2.4
      openshift.io/provider-display-name: Red Hat, Inc.
      sampleRepo: https://github.com/sclorg/ruby-ex.git
      supports: ruby:2.4,ruby
      tags: builder,ruby
      version: "2.4"
    from:
      kind: DockerImage
      name: registry.redhat.io/rhscl/ruby-24-rhel7:latest
    generation: 2
    importPolicy: {}
    name: "2.4"
    referencePolicy:
      type: Local
  - annotations:
      description: Build and run Ruby 2.5 applications on RHEL 7. For more information
        about using this builder image, including OpenShift considerations, see https://github.com/sclorg/s2i-ruby-container/blob/master/2.5/README.md.
      iconClass: icon-ruby
      openshift.io/display-name: Ruby 2.5
      openshift.io/provider-display-name: Red Hat, Inc.
      sampleRepo: https://github.com/sclorg/ruby-ex.git
      supports: ruby:2.5,ruby
      tags: builder,ruby
      version: "2.5"
    from:
      kind: DockerImage
      name: registry.redhat.io/rhscl/ruby-25-rhel7:latest
    generation: 2
    importPolicy: {}
    name: "2.5"
    referencePolicy:
      type: Local
  - annotations:
      description: |-
        Build and run Ruby applications on RHEL 7. For more information about using this builder image, including OpenShift considerations, see https://github.com/sclorg/s2i-ruby-container/tree/master/2.5/README.md.

        WARNING: By selecting this tag, your application will automatically update to use the latest version of Ruby available on OpenShift, including major version updates.
      iconClass: icon-ruby
      openshift.io/display-name: Ruby (Latest)
      openshift.io/provider-display-name: Red Hat, Inc.
      sampleRepo: https://github.com/sclorg/ruby-ex.git
      supports: ruby
      tags: builder,ruby
    from:
      kind: ImageStreamTag
      name: "2.5"
    generation: 1
    importPolicy: {}
    name: latest
    referencePolicy:
      type: Local
status:
  dockerImageRepository: image-registry.openshift-image-registry.svc:5000/openshift/ruby
  tags:
  - items:
    - created: "2019-08-06T06:57:00Z"
      dockerImageReference: registry.redhat.io/openshift3/ruby-20-rhel7@sha256:9cfdf4b811ace13d4c555335b249ab831832a384113035512abc9d4d5cc59716
      generation: 2
      image: sha256:9cfdf4b811ace13d4c555335b249ab831832a384113035512abc9d4d5cc59716
    tag: "2.0"
  - items:
    - created: "2019-08-06T06:57:00Z"
      dockerImageReference: registry.redhat.io/rhscl/ruby-22-rhel7@sha256:2e83b9e07e85960060096b6aff7ee202a5f52e0e18447641b080b1f3879e0901
      generation: 2
      image: sha256:2e83b9e07e85960060096b6aff7ee202a5f52e0e18447641b080b1f3879e0901
    tag: "2.2"
  - items:
    - created: "2019-08-06T06:57:00Z"
      dockerImageReference: registry.redhat.io/rhscl/ruby-23-rhel7@sha256:3f205876e1e6d05d693c8fc94e7abf7137f20767c66c636431774f1ea37094d6
      generation: 2
      image: sha256:3f205876e1e6d05d693c8fc94e7abf7137f20767c66c636431774f1ea37094d6
    tag: "2.3"
  - items:
    - created: "2019-08-06T06:57:00Z"
      dockerImageReference: registry.redhat.io/rhscl/ruby-24-rhel7@sha256:f3faf0a8255db207acaba98aa00293e4d2e890b5b63009fc95fafb46f035f94f
      generation: 2
      image: sha256:f3faf0a8255db207acaba98aa00293e4d2e890b5b63009fc95fafb46f035f94f
    tag: "2.4"
  - items:
    - created: "2019-08-06T06:57:00Z"
      dockerImageReference: registry.redhat.io/rhscl/ruby-25-rhel7@sha256:a05e92bbe06aafa013a2b6f5b85dec8b2f546195f17f365d99cc92aa917b7d64
      generation: 2
      image: sha256:a05e92bbe06aafa013a2b6f5b85dec8b2f546195f17f365d99cc92aa917b7d64
    tag: "2.5"
  - items:
    - created: "2019-08-06T06:57:00Z"
      dockerImageReference: registry.redhat.io/rhscl/ruby-25-rhel7@sha256:a05e92bbe06aafa013a2b6f5b85dec8b2f546195f17f365d99cc92aa917b7d64
      generation: 2
      image: sha256:a05e92bbe06aafa013a2b6f5b85dec8b2f546195f17f365d99cc92aa917b7d64
    tag: latest

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:2922