Bugzilla (bugzilla.redhat.com) will be under maintenance for infrastructure upgrades and will not be unavailable on July 31st between 12:30 AM - 05:30 AM UTC. We appreciate your understanding and patience. You can follow status.redhat.com for details.
Bug 1652532 - docker_image_availability" fails everytime on openshift 3.11 [NEEDINFO]
Summary: docker_image_availability" fails everytime on openshift 3.11
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Installer
Version: 3.11.0
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: 3.11.z
Assignee: Scott Dodson
QA Contact: Johnny Liu
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-11-22 10:26 UTC by Jaspreet Kaur
Modified: 2018-11-29 21:07 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-11-29 21:06:22 UTC
Target Upstream Version:
sdodson: needinfo? (jkaur)

Attachments (Terms of Use)
Inventory file (2.54 KB, text/plain)
2018-11-22 10:26 UTC, Jaspreet Kaur 		no flags Details
Add an attachment (proposed patch, testcase, etc.)

Description Jaspreet Kaur 2018-11-22 10:26:44 UTC 
Created attachment 1507909 [details]
Inventory file

Description of problem: Installation fails with below error

  2. Hosts:    host1, host2, host3
     Play:     OpenShift Health Checks
     Task:     Run health checks (install) - EL
     Message:  One or more checks failed
     Details:  check "docker_image_availability":
               One or more required container images are not available:
                   registry.redhat.io/openshift3/ose-control-plane:v3.11,
                   registry.redhat.io/openshift3/ose-deployer:v3.11,
                   registry.redhat.io/openshift3/ose-docker-registry:v3.11,
                   registry.redhat.io/openshift3/ose-haproxy-router:v3.11,
                   registry.redhat.io/openshift3/ose-pod:v3.11,
                   registry.redhat.io/openshift3/registry-console:v3.11,
                   registry.redhat.io/rhel7/etcd:3.2.22
               Checked with: skopeo inspect [--tls-verify=false] [--creds=<user>:<pass>] docker://<registry>/<image>




Version-Release number of the following components:

# rpm -q openshift-ansible
openshift-ansible-3.11.43-1.git.0.fa69a02.el7.noarch

# rpm -q ansible
ansible-2.6.7-1.el7ae.noarch

[quicklab@master-2 ~]$ ansible --version
ansible 2.6.7
  config file = /home/quicklab/ansible.cfg
  configured module search path = [u'/home/quicklab/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/site-packages/ansible
  executable location = /usr/bin/ansible
  python version = 2.7.5 (default, Sep 12 2018, 05:31:16) [GCC 4.8.5 20150623 (Red Hat 4.8.5-36)]

How reproducible:

Steps to Reproduce:
1. Installed a new cluster with ocp 3.11
2.
3.

Actual results: Installation fails on above check

Expected results: shouldn't have failed on above check

Additional info:
Please attach logs from ansible-playbook with the -vvv flag
Comment 2 Scott Dodson 2018-11-26 21:31:46 UTC 
Can you please gather complete logs at -vvv level? That will show us all the images it's trying to inspect and the output from those commands.
Comment 3 J. Alexander Jacocks 2018-11-27 20:53:15 UTC 
I'm having the same issue, and I attached the logs, below.
Comment 4 J. Alexander Jacocks 2018-11-27 21:05:21 UTC 
And never mind, I figured out the problem, based on the error:

CHECK [memory_availability : master.jajtest] **********************
fatal: [master.jajtest]: FAILED! => {
    "changed": true,
    "checks": {
        "disk_availability": {},
        "docker_image_availability": {
            "changed": true,
            "failed": true,
            "failures": [
                [
                    "OpenShiftCheckException",
                    "One or more required container images are not available:\n    registry.redhat.io/openshift3/ose-control-plane:v3.11,\n    registry.redhat.io/openshift3/ose-deployer:v3.11,\n    registry.redhat.io/openshift3/ose-docker-registry:v3.11,\n    registry.redhat.io/openshift3/ose-haproxy-router:v3.11,\n    registry.redhat.io/openshift3/ose-pod:v3.11,\n    registry.redhat.io/openshift3/registry-console:v3.11,\n    registry.redhat.io/rhel7/etcd:3.2.22\nChecked with: skopeo inspect [--tls-verify=false] [--creds=<user>:<pass>] docker://<registry>/<image>\n"
                ]
            ],

And from the inventory file:

$ grep oreg inventory_ose 
oreg_auth_user=${INSERT_VALUE_HERE}
oreg_auth_password=${INSERT_VALUE_HERE}

The issue is the new need for a Red Hat login and password to install OSE:

https://access.redhat.com/solutions/3661111

And:

https://docs.openshift.com/container-platform/3.11/release_notes/ocp_3_11_release_notes.html#ocp-311-registry-auth-credentials-required
Comment 5 Michael Gugino 2018-11-29 21:07:59 UTC 
This should fail early if no oreg_auth_* variables are defined, which did go through QE.  Most likely inventory problem (eg, disable sanity checks).
Note You need to log in before you can comment on or make changes to this bug.