Bug 1609963 - 'openstack undercloud install' fails during 'Check for unhealthy containers after step 5' : openstack-heat-api-cfn and openstack-heat-api are unhealthy
Summary: 'openstack undercloud install' fails during 'Check for unhealthy containers a...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-heat-templates
Version: 14.0 (Rocky)
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: beta
: 14.0 (Rocky)
Assignee: Emilien Macchi
QA Contact: Gurenko Alex
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-07-30 21:53 UTC by Alexander Chuzhoy
Modified: 2019-01-11 11:51 UTC (History)
9 users (show)

Fixed In Version: openstack-tripleo-heat-templates-9.0.0-0.20180919080945.0rc1.0rc1.el7ost
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-01-11 11:51:11 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
OpenStack gerrit 587006 0 None MERGED Revert "Fix deploy health checks" 2020-05-02 02:36:54 UTC
Red Hat Product Errata RHEA-2019:0045 0 None None None 2019-01-11 11:51:19 UTC

Description Alexander Chuzhoy 2018-07-30 21:53:40 UTC
'openstack undercloud install' fails during 'Check for unhealthy containers after step 5' : openstack-heat-api-cfn and openstack-heat-api are unhealthy

Environment:
heat-cfntools-1.2.6-5.el7.noarch
python-heat-agent-docker-cmd-1.6.1-0.20180709100740.fdd6a5f.el7ost.noarch
ansible-role-redhat-subscription-1.0.1-1.el7ost.noarch
openstack-heat-agents-1.6.1-0.20180709100740.fdd6a5f.el7ost.noarch
openstack-heat-common-12.0.0-0.20180604085325.7d878a8.el7ost.noarch
python-tripleoclient-heat-installer-10.3.1-0.20180716131225.300dfb3.el7ost.noarch
python-heat-agent-json-file-1.6.1-0.20180709100740.fdd6a5f.el7ost.noarch
python-heat-agent-apply-config-1.6.1-0.20180709100740.fdd6a5f.el7ost.noarch
ansible-pacemaker-1.0.4-0.20180220234310.0e4d7c0.el7ost.noarch
ansible-role-tripleo-modify-image-0.0.1-0.20180718224444.c2ef865.el7ost.noarch
python-heat-agent-ansible-1.6.1-0.20180709100740.fdd6a5f.el7ost.noarch
openstack-heat-api-12.0.0-0.20180604085325.7d878a8.el7ost.noarch
python2-heatclient-1.16.0-0.20180619123220.f095a0f.el7ost.noarch
python-heat-agent-1.6.1-0.20180709100740.fdd6a5f.el7ost.noarch
python-heat-agent-puppet-1.6.1-0.20180709100740.fdd6a5f.el7ost.noarch
ansible-role-container-registry-0.0.1-0.20180718101745.9fcedac.el7ost.noarch
instack-undercloud-9.1.1-0.20180716115151.bc82d48.el7ost.noarch
openstack-heat-monolith-12.0.0-0.20180604085325.7d878a8.el7ost.noarch
puppet-heat-13.1.1-0.20180719053944.8ef3646.el7ost.noarch
python-heat-agent-hiera-1.6.1-0.20180709100740.fdd6a5f.el7ost.noarch
ansible-2.5.4-1.el7ae.noarch
ansible-tripleo-ipsec-8.1.1-0.20180405121919.325d233.el7ost.noarch
openstack-tripleo-heat-templates-9.0.0-0.20180720154239.959e1d7.el7ost.noarch
openstack-heat-engine-12.0.0-0.20180604085325.7d878a8.el7ost.noarch


Steps to reproduce:
Edit undercloud.conf to include:
[stack@undercloud75 ~]$ cat undercloud.conf 
[DEFAULT]
# Network interface on the Undercloud that will be handling the PXE
# boots and DHCP for Overcloud instances. (string value)
local_interface = eth1
local_ip = 192.168.0.1/24
#undercloud_public_host = 192.168.0.2
#undercloud_admin_host = 192.168.0.3
generate_service_certificate = false
container_images_file=/home/stack/containers-prepare-parameter.yaml
undercloud_ntp_servers=clock.redhat.com
docker_insecure_registries=docker-registry.engineering.redhat.com
# BZ#1608508
undercloud_enable_selinux=False
[ctlplane-subnet]
local_subnet = ctlplane-subnet
cidr = 192.168.0.0/24
dhcp_start = 192.168.0.5
dhcp_end = 192.168.0.24
gateway = 192.168.0.1
inspection_iprange = 192.168.0.100,192.168.0.120
masquerade = true

Note the attempt to use nonSSL undercloud.

Run 'openstack undercloud install'.

Encountered https://bugzilla.redhat.com/show_bug.cgi?id=1609896

Re-ran the command 'openstack undercloud install'

Result:
}

TASK [Check for unhealthy containers after step 5] ****************************************************************************************************************************************************************
ok: [undercloud75]

TASK [Debug output for task which failed: Check for unhealthy containers after step 5] ****************************************************************************************************************************
fatal: [undercloud75]: FAILED! => {
    "failed_when_result": true,
    "outputs.stdout_lines|default([])|union(outputs.stderr_lines|default([]))": [
        "0a29b766d34c        192.168.24.1:8787/rhosp14/openstack-heat-api-cfn:2018-07-26.6   \"kolla_start\"       2 minutes ago       Up 2 minutes (unhealthy)                       heat_api_cfn",
        "a0ea4705e67d        192.168.24.1:8787/rhosp14/openstack-heat-api:2018-07-26.6       \"kolla_start\"       2 minutes ago       Up 2 minutes (unhealthy)                       heat_api"
    ]   
}   

NO MORE HOSTS LEFT ************************************************************************************************************************************************************************************************

PLAY RECAP ********************************************************************************************************************************************************************************************************
undercloud75               : ok=216  changed=58   unreachable=0    failed=1

Not cleaning working directory /home/stack/tripleo-heat-installer-templates
Not cleaning ansible directory /home/stack/undercloud-ansible-Cm4Mv9
Install artifact is located at /home/stack/undercloud-install-20180730214110.tar.bzip2

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Deployment Failed!

ERROR: Heat log files: /var/log/heat-launcher/undercloud_deploy-yfWWeD



The issue reproduces.

Comment 2 Emilien Macchi 2018-07-30 22:02:55 UTC

*** This bug has been marked as a duplicate of bug 1608903 ***

Comment 3 Marius Cornea 2018-07-31 15:49:30 UTC
Per our discussion on IRC I'm reopening this bug to keep track of the health checks validation removal in https://review.openstack.org/#/c/587006/

Comment 4 Alex Schultz 2018-07-31 18:35:55 UTC
We've reverted this logic upstream, the real issue being that the container health checks were changed but we've seen intermittent problems with container health during the deploy which we'll be working towards understanding resolving in the next release.

Comment 9 Artem Hrechanychenko 2019-01-08 14:15:37 UTC
VERIFIED

openstack-tripleo-heat-templates-9.0.1-0.20181013060907.el7ost.noarch.rpm

Comment 11 errata-xmlrpc 2019-01-11 11:51:11 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2019:0045


Note You need to log in before you can comment on or make changes to this bug.