Bug 1281735 - image stream is referencing non-existing images from internal docker-registry
image stream is referencing non-existing images from internal docker-registry
Status: CLOSED ERRATA
Product: OpenShift Container Platform
Classification: Red Hat
Component: Image Registry (Show other bugs)
3.1.0
Unspecified Unspecified
medium Severity medium
: ---
: ---
Assigned To: Maciej Szulik
ge liu
: Reopened
: 1330720 (view as bug list)
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2015-11-13 05:46 EST by Johnny Liu
Modified: 2017-07-24 10 EDT (History)
7 users (show)

See Also:
Fixed In Version:
Doc Type: No Doc Update
Doc Text:
undefined
Story Points: ---
Clone Of:
Environment:
Last Closed: 2017-04-12 15:04:47 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Johnny Liu 2015-11-13 05:46:39 EST
Description of problem:
After installation is finished, image stream is referencing non-existing images from internal docker-registry.

Version-Release number of selected component (if applicable):
atomic-openshift-3.1.0.4-1.git.4.b6c7cd2.el7aos.x86_64

How reproducible:
Always

Steps to Reproduce:
1. Use openshift-ansible to set up an OSE env.
2. The installer would create image stream with the following json file in openshift namespace.
# cat /usr/share/openshift/examples/image-streams/image-streams-rhel7.json 
{
  "kind": "ImageStreamList",
  "apiVersion": "v1",
  "metadata": {},
  "items": [
    {
      "kind": "ImageStream",
      "apiVersion": "v1",
      "metadata": {
        "name": "ruby",
        "creationTimestamp": null
      },
      "spec": {
        "tags": [
          {
            "name": "latest",
            "from": {
              "Kind": "ImageStreamTag",
              "Name": "2.0"
            }
          },
          {
            "name": "2.0",
            "annotations": {
              "description": "Build and run Ruby 2.0 applications",
              "iconClass": "icon-ruby",
              "tags": "builder,ruby",
              "supports": "ruby:2.0,ruby",
              "version": "2.0",
              "sampleRepo": "https://github.com/openshift/ruby-ex.git"
            },
            "from": {
              "Kind": "DockerImage",
              "Name": "registry.access.redhat.com/openshift3/ruby-20-rhel7:latest"
            }
          },
          {
            "name": "2.2",
            "annotations": {
              "description": "Build and run Ruby 2.2 applications",
              "iconClass": "icon-ruby",
              "tags": "builder,ruby",
              "supports": "ruby:2.2,ruby",
              "version": "2.2",
              "sampleRepo": "https://github.com/openshift/ruby-ex.git"
            },
            "from": {
              "Kind": "DockerImage",
              "Name": "registry.access.redhat.com/rhscl/ruby-22-rhel7:latest"
            }
          }
        ]
      }
    },
    {
      "kind": "ImageStream",
      "apiVersion": "v1",
      "metadata": {
        "name": "nodejs",
        "creationTimestamp": null
      },
      "spec": {
        "tags": [
          {
            "name": "latest",
            "from": {
              "Kind": "ImageStreamTag",
              "Name": "0.10"
            }
          },
          {
            "name": "0.10",
            "annotations": {
              "description": "Build and run NodeJS 0.10 applications",
              "iconClass": "icon-nodejs",
              "tags": "builder,nodejs",
              "supports":"nodejs:0.10,nodejs:0.1,nodejs",
              "version": "0.10",
              "sampleRepo": "https://github.com/openshift/nodejs-ex.git"
            },
            "from": {
              "Kind": "DockerImage",
              "Name": "registry.access.redhat.com/openshift3/nodejs-010-rhel7:latest"
            }
          }
        ]
      }
    },
    {
      "kind": "ImageStream",
      "apiVersion": "v1",
      "metadata": {
        "name": "perl",
        "creationTimestamp": null
      },
      "spec": {
        "tags": [
          {
            "name": "latest",
            "from": {
              "Kind": "ImageStreamTag",
              "Name": "5.16"
            }
          },
          {
            "name": "5.16",
            "annotations": {
              "description": "Build and run Perl 5.16 applications",
              "iconClass": "icon-perl",
              "tags": "builder,perl",
              "supports":"perl:5.16,perl",
              "version": "5.16",
              "sampleRepo": "https://github.com/openshift/dancer-ex.git"
            },
            "from": {
              "Kind": "DockerImage",
              "Name": "registry.access.redhat.com/openshift3/perl-516-rhel7:latest"
            }
          },
          {
            "name": "5.20",
            "annotations": {
              "description": "Build and run Perl 5.20 applications",
              "iconClass": "icon-perl",
              "tags": "builder,perl",
              "supports":"perl:5.20,perl",
              "version": "5.20",
              "sampleRepo": "https://github.com/openshift/dancer-ex.git"
            },
            "from": {
              "Kind": "DockerImage",
              "Name": "registry.access.redhat.com/rhscl/perl-520-rhel7:latest"
            }

          }
        ]
      }
    },
    {
      "kind": "ImageStream",
      "apiVersion": "v1",
      "metadata": {
        "name": "php",
        "creationTimestamp": null
      },
      "spec": {
        "tags": [
          {
            "name": "latest",
            "from": {
              "Kind": "ImageStreamTag",
              "Name": "5.5"
            }
          },
          {
            "name": "5.5",
            "annotations": {
              "description": "Build and run PHP 5.5 applications",
              "iconClass": "icon-php",
              "tags": "builder,php",
              "supports":"php:5.5,php",
              "version": "5.5",
              "sampleRepo": "https://github.com/openshift/cakephp-ex.git"              
            },
            "from": {
              "Kind": "DockerImage",
              "Name": "registry.access.redhat.com/openshift3/php-55-rhel7:latest"
            }
          },
          {
            "name": "5.6",
            "annotations": {
              "description": "Build and run PHP 5.6 applications",
              "iconClass": "icon-php",
              "tags": "builder,php",
              "supports":"php:5.6,php",
              "version": "5.6",
              "sampleRepo": "https://github.com/openshift/cakephp-ex.git"
            },
            "from": {
              "Kind": "DockerImage",
              "Name": "registry.access.redhat.com/rhscl/php-56-rhel7:latest"
            }
          }
        ]
      }
    },
    {
      "kind": "ImageStream",
      "apiVersion": "v1",
      "metadata": {
        "name": "python",
        "creationTimestamp": null
      },
      "spec": {
        "tags": [
          {
            "name": "latest",
            "from": {
              "Kind": "ImageStreamTag",
              "Name": "3.3"
            }
          },
          {
            "name": "3.3",
            "annotations": {
              "description": "Build and run Python 3.3 applications",
              "iconClass": "icon-python",
              "tags": "builder,python",
              "supports":"python:3.3,python",
              "version": "3.3",
              "sampleRepo": "https://github.com/openshift/django-ex.git"
            },
            "from": {
              "Kind": "DockerImage",
              "Name": "registry.access.redhat.com/openshift3/python-33-rhel7:latest"
            }
          },
          {
            "name": "2.7",
            "annotations": {
              "description": "Build and run Python 2.7 applications",
              "iconClass": "icon-python",
              "tags": "builder,python",
              "supports":"python:2.7,python",
              "version": "2.7",
              "sampleRepo": "https://github.com/openshift/django-ex.git"
            },
            "from": {
              "Kind": "DockerImage",
              "Name": "registry.access.redhat.com/rhscl/python-27-rhel7:latest"
            }
          },
          {
            "name": "3.4",
            "annotations": {
              "description": "Build and run Python 3.4 applications",
              "iconClass": "icon-python",
              "tags": "builder,python",
              "supports":"python:3.4,python",
              "version": "3.4",
              "sampleRepo": "https://github.com/openshift/django-ex.git"
            },
            "from": {
              "Kind": "DockerImage",
              "Name": "registry.access.redhat.com/rhscl/python-34-rhel7:latest"
            }
          }
        ]
      }
    },
    {
      "kind": "ImageStream",
      "apiVersion": "v1",
      "metadata": {
        "name": "mysql",
        "creationTimestamp": null
      },
      "spec": {
        "tags": [
          {
            "name": "latest",
            "from": {
              "Kind": "ImageStreamTag",
              "Name": "5.5"
            }
          },
          {
            "name": "5.5",
            "from": {
              "Kind": "DockerImage",
              "Name": "registry.access.redhat.com/openshift3/mysql-55-rhel7:latest"
            }
          },
          {
            "name": "5.6",
            "from": {
              "Kind": "DockerImage",
              "Name": "registry.access.redhat.com/rhscl/mysql-56-rhel7:latest"
            }
          }
        ]
      }
    },
    {
      "kind": "ImageStream",
      "apiVersion": "v1",
      "metadata": {
        "name": "postgresql",
        "creationTimestamp": null
      },
      "spec": {
        "tags": [
          {
            "name": "latest",
            "from": {
              "Kind": "ImageStreamTag",
              "Name": "9.2"
            }
          },
          {
            "name": "9.2",
            "from": {
              "Kind": "DockerImage",
              "Name": "registry.access.redhat.com/openshift3/postgresql-92-rhel7:latest"
            }
          },
          {
            "name": "9.4",
            "from": {
              "Kind": "DockerImage",
              "Name": "registry.access.redhat.com/rhscl/postgresql-94-rhel7:latest"
            }
          }
        ]
      }
    },
    {
      "kind": "ImageStream",
      "apiVersion": "v1",
      "metadata": {
        "name": "mongodb",
        "creationTimestamp": null
      },
      "spec": {
        "tags": [
          {
            "name": "latest",
            "from": {
              "Kind": "ImageStreamTag",
              "Name": "2.4"
            }
          },
          {
            "name": "2.4",
            "from": {
              "Kind": "DockerImage",
              "Name": "registry.access.redhat.com/openshift3/mongodb-24-rhel7:latest"
            }
          },
          {
            "name": "2.6",
            "from": {
              "Kind": "DockerImage",
              "Name": "registry.access.redhat.com/rhscl/mongodb-26-rhel7:latest"
            }
          }
        ]
      }
    },
    {
      "kind": "ImageStream",
      "apiVersion": "v1",
      "metadata": {
        "name": "jenkins",
        "creationTimestamp": null
      },
      "spec": {
        "tags": [
          {
            "name": "latest",
            "from": {
              "Kind": "ImageStreamTag",
              "Name": "1"
            }
          },
          {
            "name": "1",
            "from": {
              "Kind": "DockerImage",
              "Name": "registry.access.redhat.com/openshift3/jenkins-1-rhel7:latest"
            }
          }
        ]
      }
    }
  ]
}

3.

Actual results:
# oc get is -n openshift
NAME                                  DOCKER REPO                                                                  TAGS               UPDATED
jboss-amq-62                          registry.access.redhat.com/jboss-amq-6/amq62-openshift                       1.1,1.1-2,latest   9 hours ago
jboss-eap64-openshift                 registry.access.redhat.com/jboss-eap-6/eap64-openshift                       1.1-2,latest,1.1   9 hours ago
jboss-webserver30-tomcat7-openshift   registry.access.redhat.com/jboss-webserver-3/webserver30-tomcat7-openshift   1.1,1.1-2,latest   9 hours ago
jboss-webserver30-tomcat8-openshift   registry.access.redhat.com/jboss-webserver-3/webserver30-tomcat8-openshift   1.1-3,latest,1.1   9 hours ago
jenkins                               172.30.105.146:5000/openshift/jenkins                                        1,latest           9 hours ago
mongodb                               172.30.105.146:5000/openshift/mongodb                                        2.4,latest         9 hours ago
mysql                                 172.30.105.146:5000/openshift/mysql                                          5.5,latest         9 hours ago
nodejs                                172.30.105.146:5000/openshift/nodejs                                         0.10,latest        9 hours ago
perl                                  172.30.105.146:5000/openshift/perl                                           latest,5.16        9 hours ago
php                                   172.30.105.146:5000/openshift/php                                            5.5,latest         9 hours ago
postgresql                            172.30.105.146:5000/openshift/postgresql                                     9.2,latest         9 hours ago
python                                172.30.105.146:5000/openshift/python                                         latest,3.3         9 hours ago
ruby                                  172.30.105.146:5000/openshift/ruby                                           2.0,latest         9 hour


IS is referencing 172.30.105.146:5000, which is internal docker-registry service IP.

Try to pull those images, but not found.
# docker pull 172.30.105.146:5000/openshift/php 
Using default tag: latest
Trying to pull repository 172.30.105.146:5000/openshift/php ... not found
Error: image openshift/php:latest not found



Expected results:
The correct message is showing.


Additional info:
Comment 1 Maciej Szulik 2015-11-17 10:36:09 EST
This is not a bug, for now it's intended behavior. We're discussing the output of oc describe and oc get regarding image streams in https://github.com/openshift/origin/issues/5927.
Comment 2 Johnny Liu 2015-11-17 20:59:25 EST
So the output will be changed in the coming release, here I suggest add "UpcomingRelease" keyword, and keep this bug opened status to track this issue.
Comment 3 Maciej Szulik 2016-04-27 09:16:16 EDT
*** Bug 1330720 has been marked as a duplicate of this bug. ***
Comment 4 Ryan Howe 2016-05-20 14:46:33 EDT
After a fresh 3.1 install

# oc get is 

Will still display the wrong docker repo for the rhel7 images streams added by the installer. 

This is misleading. 

/usr/share/openshift/examples/image-streams/image-streams-rhel7.json

While jboss image streams are correctly configured. 

/usr/share/openshift/examples/xpaas-streams/jboss-image-streams.json 



What is missing is dockerImageRepository being defined, since it is not defined it gets defaulted to the internal registry. 

"spec": {
                "dockerImageRepository": "registry.access.redhat.com/XXXXX"

Either this needs to be documented as to why we want to show the internal registry as the docker repo  or corrected by adding dockerImageRepository equal to the registry from which the image is being pulled.
Comment 5 Michal Fojtik 2016-08-04 07:18:08 EDT
bump?
Comment 6 Maciej Szulik 2016-08-05 06:20:33 EDT
On my radar.
Comment 7 Maciej Szulik 2016-08-05 15:04:34 EDT
Fixed in https://github.com/openshift/origin/pull/10239
Comment 8 openshift-github-bot 2016-08-10 07:56:18 EDT
Commit pushed to master at https://github.com/openshift/origin

https://github.com/openshift/origin/commit/7df259d03a0bcbd75e9cc51f7699c8b25f3431e1
Bug 1281735 - remove the internal docker registry information from
imagestream
Comment 9 openshift-github-bot 2016-08-10 16:35:08 EDT
Commit pushed to master at https://github.com/openshift/origin

https://github.com/openshift/origin/commit/b222c23aeeeba1753cbc186185de482d8e7a0f33
Revert "Bug 1281735 - remove the internal docker registry information from imagestream"
Comment 10 Maciej Szulik 2016-08-10 18:40:11 EDT
The change was reverted in https://github.com/openshift/origin/pull/10343 assigning back to myself.
Comment 11 Maciej Szulik 2016-10-31 10:57:52 EDT
Based on this comment [1] we can't remove that now, it'll be fixed in one of the future versions. 


[1] https://github.com/openshift/origin/pull/10239#issuecomment-238995216
Comment 12 Maciej Szulik 2017-01-03 06:59:17 EST
I'm closing this in favor of https://trello.com/c/lsWunRTg/858-need-to-advertise-a-public-pull-spec-for-image since this will be addressing this problem.
Comment 13 Maciej Szulik 2017-01-03 07:06:51 EST
Actually, with pullthrough logic [1] which is on by default, users are able to pull from the internal address already, so moving to on-qa.

[1] https://github.com/openshift/origin/blob/a5eaf994f6e86e28bff529de7da66a708f029699/pkg/dockerregistry/server/repositorymiddleware.go#L177
Comment 14 ge liu 2017-01-04 03:48:57 EST
Verified in env:

[root@openshift-129 ~]# openshift version
openshift v3.4.0.38
kubernetes v1.4.0+776c994
etcd 3.1.0-rc.0


[root@openshift-137 ~]# docker login -u geliu -p Rxy-aZj-Im-lGBbqCaA-YNm712g4IhDRQt-d05ik5L8 -e geliu@redhat.com 172.xxx.7.5:5000
Flag --email has been deprecated, will be removed in 1.13.
Login Succeeded

[root@dhcp-137-141 ~]# oc get is
NAME              DOCKER REPO                              TAGS      UPDATED
hello-openshift   172.xxx.7.5:5000/lgproj/hello-openshift   latest    2 hours ago

[root@openshift-137 ~]# docker pull  172.xxx.7.5:5000/lgproj/hello-openshift
Using default tag: latest
Trying to pull repository 172.xxx.7.5:5000/lgproj/hello-openshift ... 
sha256:5d2a5bf63f70ac2f963044e1961ca43c79352f57eebd6318b82c1d696310f3e4: Pulling from 172.30.7.5:5000/lgproj/hello-openshift
5e96ef59afb4: Pull complete 
Digest: sha256:5d2a5bf63f70ac2f963044e1961ca43c79352f57eebd6318b82c1d696310f3e4
Status: Downloaded newer image for 172.xxx.7.5:5000/lgproj/hello-openshift:latest
[root@openshift-137 ~]#
Comment 16 errata-xmlrpc 2017-04-12 15:04:47 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:0884

Note You need to log in before you can comment on or make changes to this bug.