Bug 1654257 - Director deployed OpenShift deployment fails when using default containers-prepare-parameter.yaml
Summary: Director deployed OpenShift deployment fails when using default containers-pr...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-heat-templates
Version: 14.0 (Rocky)
Hardware: Unspecified
OS: Unspecified
urgent
urgent
Target Milestone: rc
: 14.0 (Rocky)
Assignee: Martin André
QA Contact: Marius Cornea
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-11-28 10:38 UTC by Martin André
Modified: 2019-01-11 11:55 UTC (History)
6 users (show)

Fixed In Version: openstack-tripleo-heat-templates-9.0.1-0.20181013060898.el7ost
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-01-11 11:55:06 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Launchpad 1805611 0 None None None 2018-11-28 10:51:52 UTC
OpenStack gerrit 620557 0 None None None 2018-11-28 10:53:45 UTC
Red Hat Product Errata RHEA-2019:0045 0 None None None 2019-01-11 11:55:15 UTC

Description Martin André 2018-11-28 10:38:18 UTC
Description of problem:


Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1. Generate a default containers-prepare-parameter.yaml file with `openstack tripleo container image prepare default --local-push-destination --output-env-file containers-prepare-parameter.yaml`
2. Deploy with CNS enabled, using the generated prepare file:
   $ openstack overcloud deploy --stack openshift --templates \
        -r /home/stack/openshift_roles_data.yaml \
        -n /home/stack/network_data_openshift.yaml \
        -e /usr/share/openstack-tripleo-heat-templates/environments/network-environment.yaml \
        -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml \
        -e /usr/share/openstack-tripleo-heat-templates/environments/openshift.yaml \
        -e /usr/share/openstack-tripleo-heat-templates/environments/openshift-cns.yaml \
        -e /home/stack/openshift_env.yaml \
        -e /home/stack/containers-prepare-parameter.yaml

3. Wait for deployment to finish

Actual results:

openshift-ansible fails with:

TASK [openshift_storage_glusterfs : Wait for GlusterFS pods] *******************
FAILED - RETRYING: Wait for GlusterFS pods (30 retries left).
FAILED - RETRYING: Wait for GlusterFS pods (29 retries left).
FAILED - RETRYING: Wait for GlusterFS pods (28 retries left).
FAILED - RETRYING: Wait for GlusterFS pods (27 retries left).
FAILED - RETRYING: Wait for GlusterFS pods (26 retries left).
[snip]
{"waiting": {"message": "Back-off pulling image \"172.16.0.1:8787/rhgs3/rhgs-server-rhel7\"", "reason": "ImagePullBackOff"}}
[snip]

INSTALLER STATUS ***************************************************************
Initialization              : Complete (0:02:09)
Health Check                : Complete (0:00:21)
Node Bootstrap Preparation  : Complete (0:05:24)
etcd Install                : Complete (0:01:46)
Master Install              : Complete (0:08:45)
Master Additional Install   : Complete (0:02:05)
Node Join                   : Complete (0:00:51)
GlusterFS Install           : In Progress (0:06:08)

Nodes try to download an image with 'latest' tag:

Events:
  Type     Reason   Age                  From                         Message
  ----     ------   ----                 ----                         -------
  Normal   Pulling  1h (x168 over 14h)   kubelet, openshift-worker-1  pulling image "172.16.0.1:8787/rhgs3/rhgs-server-rhel7"
  Normal   BackOff  7m (x3834 over 14h)  kubelet, openshift-worker-1  Back-off pulling image "172.16.0.1:8787/rhgs3/rhgs-server-rhel7"
  Warning  Failed   2m (x3855 over 14h)  kubelet, openshift-worker-1  Error: ImagePullBackOff


Expected results:
Deployment succeeds without errors.

Additional info:

This is caused by the "tag_from_label: '{version}-{release}'" line in the generated containers-prepare-parameter.yaml file. The gluster images are pulled with the 'latest' tag and uploaded to the local registry with a versioned tag, eg '3.11.0-6'.

Openshift-ansible commit https://github.com/openshift/openshift-ansible/commit/0be4b2565beb92c064917627863401af7dfb73d3 deprecated the use of openshift_storage_glusterfs_*_version variables to specify the image tags. They now need to be passed directly into the openshift_storage_glusterfs_*_image variables.

As a workaround we can remove the "tag_from_label" line from containers-prepare-parameter.yaml.

Comment 1 Martin André 2018-11-28 10:53:46 UTC
Submitted upstream patch at https://review.openstack.org/#/c/620557/

Comment 18 Martin André 2019-01-10 10:15:09 UTC
No doc text required.

Comment 19 errata-xmlrpc 2019-01-11 11:55:06 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2019:0045


Note You need to log in before you can comment on or make changes to this bug.