Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1624081

Summary: Openshift deployment via Director fails when re-running the overcloud deploy command used for initial deployment
Product: Red Hat OpenStack Reporter: Marius Cornea <mcornea>
Component: openstack-tripleo-heat-templatesAssignee: John Trowbridge <jtrowbri>
Status: CLOSED ERRATA QA Contact: Gurenko Alex <agurenko>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 14.0 (Rocky)CC: dbecker, jtrowbri, ltomasbo, m.andre, mburns, mcornea, morazi, racedoro
Target Milestone: rcKeywords: Triaged
Target Release: 14.0 (Rocky)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: openstack-tripleo-heat-templates-9.0.1-0.20181013060882.el7ost Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-01-11 11:51:51 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
ansible.log none

Description Marius Cornea 2018-08-30 21:21:54 UTC
Description of problem:
Openshift deployment via Director fails when re-running the overcloud deploy command with:

TASK [generate openshift playbook] *********************************************
Thursday 30 August 2018  15:59:27 -0400 (0:00:00.050)       0:02:30.779 ******* 
fatal: [undercloud]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'openshift_upgrade' is undefined\n\nThe error appears to have been in '/var/lib/mistral/openshift/external_deploy_steps_tasks.yaml': line 124, column 5, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n      openshift_upgrade_playbook_path: /usr/share/ansible/openshift-ansible/playbooks/byo/openshift-cluster/upgrades/v3_9/upgrade.yml\n  - copy:\n    ^ here\n"}


Version-Release number of selected component (if applicable):
openstack-tripleo-heat-templates-9.0.0-0.20180818200902.cb08cb1.el7ost.noarch

How reproducible:
100%

Steps to Reproduce:
1. Deploy Openshift with 1 master + 1 worker:
openstack overcloud deploy \
--stack openshift \
--templates \
-r /home/stack/openshift_roles_data.yaml \
-e /usr/share/openstack-tripleo-heat-templates/environments/openshift.yaml \
-e /home/stack/openshift_env.yaml \
-e /home/stack/containers-default-parameters.yaml

2. Make sure deployment is successful

3. Re-run the same overcloud deploy command

Actual results:
Fails.

Expected results:
Doesn't fail. Deployment should be idempotent.

Additional info:
Same failure occurs in case of a configuration change.

Comment 1 Marius Cornea 2018-08-30 21:24:50 UTC
Created attachment 1479913 [details]
ansible.log

Comment 2 John Trowbridge 2018-09-26 13:31:36 UTC
I have seen this as well. It is because we have different behaviour for a stack update[1]. Would it be possible to test with the fix to new node detection?[2]

[1] https://github.com/openstack/tripleo-heat-templates/blob/master/extraconfig/services/openshift-master.yaml#L341-L355
[2] https://review.openstack.org/600012

Comment 3 John Trowbridge 2018-09-27 16:09:59 UTC
I was able to reproduce this upstream even with the patch in my comment above (600012). I made and upstream bug[1] and patch[2] for it.

[1] https://bugs.launchpad.net/tripleo/+bug/1794824 
[2] https://review.openstack.org/605796

Comment 4 Marius Cornea 2018-09-28 22:08:17 UTC
(In reply to John Trowbridge from comment #3)
> I was able to reproduce this upstream even with the patch in my comment
> above (600012). I made and upstream bug[1] and patch[2] for it.
> 
> [1] https://bugs.launchpad.net/tripleo/+bug/1794824 
> [2] https://review.openstack.org/605796

I tried applying but next it fails on this task:

TASK [generate openshift playbook] *********************************************
Friday 28 September 2018  18:01:40 -0400 (0:00:00.052)       0:03:47.760 ****** 
fatal: [undercloud]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'new_masters' is undefined\n\nThe error appears to have been in '/var/lib/mistral/openshift/external_deploy_steps_tasks.yaml': line 158, column 5, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n      openshift_worker_scaleup_playbook_path: /usr/share/ansible/openshift-ansible/playbooks/openshift-node/scaleup.yml\n  - copy:\n    ^ here\n"}

Comment 5 John Trowbridge 2018-10-08 18:50:26 UTC
I was able to get past this second issue with Martin's fix for the update tasks:

https://review.openstack.org/608658

With that patch and the previous one I posted, I am able to rerun the deploy command as expected.

Comment 6 Martin André 2018-10-23 16:40:27 UTC
This is still missing https://review.openstack.org/#/c/605796/ from upstream.

Comment 25 Martin André 2019-01-10 10:26:45 UTC
No doc text required.

Comment 26 errata-xmlrpc 2019-01-11 11:51:51 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2019:0045