Bug 1547149

Summary: [OSP13]Cannot replace controller - Error validating environment for plan overcloud - result': u'Failure caused by error in tasks: set_get_tempurl\n\n set_get_tempurl [task_ex_id=cad23b19-e799-4f40-a389-a7354d57005a]
Product: Red Hat OpenStack Reporter: Artem Hrechanychenko <ahrechan>
Component: openstack-tripleo-heat-templatesAssignee: Emilien Macchi <emacchi>
Status: CLOSED WORKSFORME QA Contact: Gurenko Alex <agurenko>
Severity: medium Docs Contact:
Priority: low    
Version: 13.0 (Queens)CC: ahrechan, aschultz, dpeacock, jcoufal, mbayer, mburns, michele, ohochman, rhel-osp-director-maint, sbaker, wznoinsk, yrabl, zgreenbe
Target Milestone: rcKeywords: Reopened
Target Release: 13.0 (Queens)   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-05-02 13:13:07 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Artem Hrechanychenko 2018-02-20 15:43:10 UTC 
Description of problem:
http://pastebin.test.redhat.com/557407

https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/12/html-single/director_installation_and_usage/#sect-Replacing_Controller_Nodes

For OSP13 , step with run overcloud_deploy command with -e /home/stack/remove-controller.yaml



Version-Release number of selected component (if applicable):
openstack-swift-object-2.15.2-0.20180118220641.449d83f.el7ost.noarch
openstack-neutron-ml2-12.0.0-0.20180123043113.d32ad6e.el7ost.noarch
openstack-heat-common-10.0.0-0.20180123134729.85a3e98.el7ost.noarch
openstack-ironic-conductor-10.0.1-0.20180123173432.2c630d9.el7ost.noarch
puppet-openstacklib-12.2.0-0.20180119083603.12107fd.el7ost.noarch
openstack-tripleo-common-8.3.1-0.20180123050219.el7ost.noarch
openstack-mistral-api-6.0.0-0.20180122153726.ae7950e.el7ost.noarch
openstack-neutron-12.0.0-0.20180123043113.d32ad6e.el7ost.noarch
openstack-nova-api-17.0.0-0.20180123163703.27eadbc.el7ost.noarch
openstack-nova-conductor-17.0.0-0.20180123163703.27eadbc.el7ost.noarch
openstack-tripleo-common-containers-8.3.1-0.20180123050219.el7ost.noarch
openstack-nova-common-17.0.0-0.20180123163703.27eadbc.el7ost.noarch
openstack-neutron-common-12.0.0-0.20180123043113.d32ad6e.el7ost.noarch
puppet-openstack_extras-12.2.0-0.20180119072029.b090b97.el7ost.noarch
openstack-tripleo-image-elements-8.0.0-0.20180117094122.02d0985.el7ost.noarch
openstack-swift-account-2.15.2-0.20180118220641.449d83f.el7ost.noarch
openstack-keystone-13.0.0-0.20180123044720.9cd5f19.el7ost.noarch
openstack-heat-api-cfn-10.0.0-0.20180123134729.85a3e98.el7ost.noarch
openstack-ironic-common-10.0.1-0.20180123173432.2c630d9.el7ost.noarch
openstack-ironic-staging-drivers-0.8.1-0.20180118014800.ae60c0b.el7ost.noarch
openstack-mistral-executor-6.0.0-0.20180122153726.ae7950e.el7ost.noarch
openstack-nova-scheduler-17.0.0-0.20180123163703.27eadbc.el7ost.noarch
python2-openstackclient-3.13.0-0.20180109064941.8c5f755.el7ost.noarch
openstack-tripleo-puppet-elements-8.0.0-0.20180117092204.120eca8.el7ost.noarch
openstack-tripleo-heat-templates-8.0.0-0.20180122224017.el7ost.noarch
openstack-nova-compute-17.0.0-0.20180123163703.27eadbc.el7ost.noarch
openstack-heat-engine-10.0.0-0.20180123134729.85a3e98.el7ost.noarch
openstack-ironic-api-10.0.1-0.20180123173432.2c630d9.el7ost.noarch
openstack-ironic-inspector-7.0.1-0.20180122225049.0cf3316.el7ost.noarch
openstack-mistral-common-6.0.0-0.20180122153726.ae7950e.el7ost.noarch
openstack-tripleo-ui-8.1.1-0.20180122135122.aef02d8.el7ost.noarch
openstack-swift-proxy-2.15.2-0.20180118220641.449d83f.el7ost.noarch
openstack-tempest-17.2.1-0.20180123175633.el7ost.noarch
openstack-tripleo-validations-8.1.1-0.20180119231917.2ff3c79.el7ost.noarch
openstack-zaqar-6.0.0-0.20180123030653.1feeb36.el7ost.noarch
openstack-nova-placement-api-17.0.0-0.20180123163703.27eadbc.el7ost.noarch
openstack-selinux-0.8.13-0.20180108164838.c6158ce.el7ost.noarch
openstack-glance-16.0.0-0.20180123074513.466c104.el7ost.noarch
python-openstackclient-lang-3.13.0-0.20180109064941.8c5f755.el7ost.noarch
openstack-swift-container-2.15.2-0.20180118220641.449d83f.el7ost.noarch
python2-openstacksdk-0.9.19-0.20171105041723.bd60aa4.el7ost.noarch
openstack-neutron-openvswitch-12.0.0-0.20180123043113.d32ad6e.el7ost.noarch
openstack-heat-api-10.0.0-0.20180123134729.85a3e98.el7ost.noarch
openstack-puppet-modules-11.0.0-0.20171011152327.71ad01c.el7ost.noarch
openstack-mistral-engine-6.0.0-0.20180122153726.ae7950e.el7ost.noarch


How reproducible:
Always

Steps to Reproduce:
1. Deploy OSP13 with 3 controllers + 1 compute + 1 empty ironic node
2. Try to replace controlling using official guide from osp12

Actual results:
Error validating environment for plan overcloud: ERROR: Internal Error after running overcloud_deploy

Expected results:
Pass

Additional info:
Comment 6 Emilien Macchi 2018-03-28 17:59:04 UTC 
*** Bug 1561614 has been marked as a duplicate of this bug. ***
Comment 7 Artem Hrechanychenko 2018-03-29 14:22:42 UTC 
Didn't reproduced last time

Will re-open in case of reproducing
Comment 8 Yogev Rabl 2018-04-30 18:46:51 UTC 
I have managed to reproduce this bug multiple times
Comment 10 Waldemar Znoinski 2018-04-30 18:49:59 UTC 
Yogev what puddle you've tried this with?
CI doesn't see this problem since at least a month already. Please retest with 2018-04-24.1 puddle if possible.
Comment 11 Omri Hochman 2018-04-30 20:42:13 UTC 
(In reply to Yogev Rabl from comment #8)
> I have managed to reproduce this bug multiple times

Yogev - this issue is for replace controller ,  according Artem test this scenario should work with latest bits.

Switch to PIDONE
Comment 12 Michele Baldessari 2018-05-02 12:12:00 UTC 
Artem, last email from you was that with all the right packages/patches it works. correct?
Comment 13 Yogev Rabl 2018-05-02 12:36:59 UTC 
After debugging the environment it seems that the output provided by the CLI client is misleading. 
A misconfiguration in the network config files was the real problem and it was notified in the heat-engine.log. 

The real bug here is in the tripleo client that provides a generic error without any information to the operator that runs the deployment.

I would set back to DF:DFG as a tripleo output error with a low priority
Comment 14 Michele Baldessari 2018-05-02 13:13:07 UTC 
Feel free to open a BZ with the specific error/issue you are seeing. It is impossible to understand what the exact issue here is and in any case it would be a different bug.