After successfully running the following to a working overcloud: openstack overcloud deploy --use-tripleo-heat-templates Running the above command again should just do a stack-update with no changes, however the update eventually fails with the following: [cloud-user@undercloud ~]$ heat resource-list -n2 overcloud | grep "OS::Heat::StructuredDeployment " | grep UPDATE_FAILED | 0 | 462c6dbc-73d2-41ef-b8a2-5f57a255d368 | OS::Heat::StructuredDeployment | UPDATE_FAILED | 2015-07-07T00:23:39Z | ControllerOvercloudServicesDeployment_Step4 | [cloud-user@undercloud ~]$ heat deployment-output-show --format raw 462c6dbc-73d2-41ef-b8a2-5f57a255d368 deploy_stderr Warning: Scope(Class[Keystone]): Execution of db_sync does not depend on $enabled anymore. Please use sync_db instead. Error: Could not find data item keystone_signing_certificate in any Hiera data file and no default supplied at /var/lib/heat-config/heat-config-puppet/4918074e-d62c-44f3-a0cb-b9540f7e3a28.pp:426 on node overcloud-controller-0.localdomain Error: Could not find data item keystone_signing_certificate in any Hiera data file and no default supplied at /var/lib/heat-config/heat-config-puppet/4918074e-d62c-44f3-a0cb-b9540f7e3a28.pp:426 on node overcloud-controller-0.localdomain
We probably need to take that flag out for GA.
I've just confirmed the exact same error message for occurs for both --use-tripleo-heat-templates and --plan overcloud stacks
I think the problem with --use-tripleo-heat-templates can be found here: https://github.com/rdo-management/python-rdomanager-oscplugin/blob/master/rdomanager_oscplugin/v1/overcloud_deploy.py#L417-L422 The utils.create_environment_file re-creates the environment file on each deploy, and we only add the Keystone cert to it on a Heat CREATE. So the fix would be to update utils.create_environment_file to only create the file if it doesn't exist.
Because the env file generated by utils.create_environment_file stores mainly scale parameters I think this file should be updated on update operation too (even if it already exists), but the existing keystone params should be kept, not overwritten.
(In reply to Steve Baker from comment #4) > I've just confirmed the exact same error message for occurs for both > --use-tripleo-heat-templates and --plan overcloud stacks I tested it and tuskar repeated deploy worked. Have you destroyed heat stack overcloud after --use-tht and before running tuskar deploy?
This bug should be fixed with https://bugzilla.redhat.com/show_bug.cgi?id=1240123
*** Bug 1240123 has been marked as a duplicate of this bug. ***
A partner and I tried the fix from the above bug and tried manually applying the config change as described by Graeme in the duplicate bug: https://bugzilla.redhat.com/show_bug.cgi?id=1240123#c6 We too see from `heat stack-show overcloud` that the SigningCertificate IS being passed from hiera.datafiles (which is what bz 1240123 fixes). However, puppet still cannot access it and we get the same error above Is the fix incomplete? Let me know what I can provide to help. John
(In reply to John Fulton from comment #11) > A partner and I tried the fix from the above bug and tried manually applying > the > config change as described by Graeme in the duplicate bug: > > https://bugzilla.redhat.com/show_bug.cgi?id=1240123#c6 > > We too see from `heat stack-show overcloud` that the SigningCertificate IS > being passed from hiera.datafiles (which is what bz 1240123 fixes). However, > puppet still cannot access it and we get the same error above Is the fix > incomplete? Let me know what I can provide to help. > > John Fix should work, but I am not sure about SigningCertificate etc. Better ask patch author Graeme..
Hi, All my patch does is ensure that the variables originally given to heat on first deploy are provided to subsequent redeploys. John are you able to give me some more background about the problem you are experiencing with the partner? Is the first overcloud deploy working successfully, and subsequent deployments failing? Are you able to give us a copy of heat stack-show overcloud Somewhere private, which should contain all the keys and certs that are being generated? Also, are you seeing the values for these parameters being written to disk in heira under /etc/puppet/hieradata/ on the controller nodes in question? Regards, Graeme
> Is the first overcloud deploy working successfully, and subsequent > deployments failing? No, they are all failing and the partner has yet to successfully deploy an overcloud on bare metal using OSP-Director Beta 2. - He tried a first deployment using the images that he had built - We then attributed this to needing to you Red Hat provided images - He then removed those images from glance, imported ours (case 01470774) - He then attempted to redeploy using our images and had this issue [stack@manager ~]$ heat stack-list +----+------------+--------------+---------------+ | id | stack_name | stack_status | creation_time | +----+------------+--------------+---------------+ +----+------------+--------------+---------------+ [stack@manager ~]$ openstack baremetal introspection bulk start Waiting for discovery to finish Discovery for UUID 829da90a-a0af-4036-90cc-bfb183c90760 finished successfully. Discovery for UUID 9ad871c5-9a1b-4bb6-86da-def2af416633 finished successfully. Discovery for UUID e4906227-388a-407c-afa3-0c631ede5c95 finished successfully. [stack@manager ~]$ openstack management plan list +--------------------------------------+-----------+-------------+------------------------------------------------------------------+ | uuid | name | description | roles | +--------------------------------------+-----------+-------------+------------------------------------------------------------------+ | 297198d2-39ac-438a-8e07-e44d6924080a | overcloud | None | Compute, Ceph-Storage, Cinder-Storage, Controller, Swift-Storage | +--------------------------------------+-----------+-------------+------------------------------------------------------------------+ <--- snip ---> /tmp/tmpcP1zUP/hieradata/common.yaml /tmp/tmpcP1zUP/puppet/manifests/ringbuilder.pp /tmp/tmpcP1zUP/extraconfig/post_deploy/default.yaml /tmp/tmpcP1zUP/net-config-noop.yaml /tmp/tmpcP1zUP/puppet/all-nodes-config.yaml /tmp/tmpcP1zUP/firstboot/userdata_default.yaml ERROR: openstack Heat Stack create failed. [stack@manager ~]$ heat stack-list +--------------------------------------+------------+---------------+----------------------+ | id | stack_name | stack_status | creation_time | +--------------------------------------+------------+---------------+----------------------+ | ede7daa2-e5da-46c4-9823-1ed8cca9fe6e | overcloud | CREATE_FAILED | 2015-07-07T21:21:58Z | +--------------------------------------+------------+---------------+----------------------+ [stack@manager ~]$
(In reply to John Fulton from comment #16) > > Is the first overcloud deploy working successfully, and subsequent > > deployments failing? > > No, they are all failing and the partner has yet to successfully > deploy an overcloud on bare metal using OSP-Director Beta 2. > > - He tried a first deployment using the images that he had built > - We then attributed this to needing to you Red Hat provided images > - He then removed those images from glance, imported ours (case 01470774) > - He then attempted to redeploy using our images and had this issue > > [stack@manager ~]$ heat stack-list > +----+------------+--------------+---------------+ > | id | stack_name | stack_status | creation_time | > +----+------------+--------------+---------------+ > +----+------------+--------------+---------------+ > [stack@manager ~]$ openstack baremetal introspection bulk start > Waiting for discovery to finish > Discovery for UUID 829da90a-a0af-4036-90cc-bfb183c90760 finished > successfully. > Discovery for UUID 9ad871c5-9a1b-4bb6-86da-def2af416633 finished > successfully. > Discovery for UUID e4906227-388a-407c-afa3-0c631ede5c95 finished > successfully. > [stack@manager ~]$ openstack management plan list > +--------------------------------------+-----------+-------------+----------- > -------------------------------------------------------+ > | uuid | name | description | roles > | > +--------------------------------------+-----------+-------------+----------- > -------------------------------------------------------+ > | 297198d2-39ac-438a-8e07-e44d6924080a | overcloud | None | Compute, > Ceph-Storage, Cinder-Storage, Controller, Swift-Storage | > +--------------------------------------+-----------+-------------+----------- > -------------------------------------------------------+ > > <--- snip ---> > /tmp/tmpcP1zUP/hieradata/common.yaml > /tmp/tmpcP1zUP/puppet/manifests/ringbuilder.pp > /tmp/tmpcP1zUP/extraconfig/post_deploy/default.yaml > /tmp/tmpcP1zUP/net-config-noop.yaml > /tmp/tmpcP1zUP/puppet/all-nodes-config.yaml > /tmp/tmpcP1zUP/firstboot/userdata_default.yaml > > ERROR: openstack Heat Stack create failed. > > [stack@manager ~]$ heat stack-list > +--------------------------------------+------------+---------------+-------- > --------------+ > | id | stack_name | stack_status | > creation_time | > +--------------------------------------+------------+---------------+-------- > --------------+ > | ede7daa2-e5da-46c4-9823-1ed8cca9fe6e | overcloud | CREATE_FAILED | > 2015-07-07T21:21:58Z | > +--------------------------------------+------------+---------------+-------- > --------------+ > > [stack@manager ~]$ I would say this might have to be a separate bug lodged as this isn't related to doing overcloud deploy twice. In fact, the bug fix I did only does anything if an existing overcloud exists in heat stack-list when you run overcloud deploy. Would probably be the best way to get more exposure and someone to properly work through the troubleshooting for the different occurrence here (initial deploy failing with keystone parameters not being passed through)
Adding bug documentation (not sure if needed).
Verified: Environment: instack-undercloud-2.1.2-21.el7ost.noarch Deploying the overcloud several times results in "Overcloud Deployed".
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2015:1549
I agreed with the comment by Graeme Gillies on 2015-07-12 20:34:57 EDT. BZ closed as per errata.