Bugzilla will be upgraded to version 5.0. The upgrade date is tentatively scheduled for 2 December 2018, pending final testing and feedback.
First Last Prev Next    This bug is not in your last search results.
Bug 1340453 - Heat stack on overcloud fails with: Resource CREATE failed: Error: Cannot get stack domain user token, no stack domain id configured, please fix your heat.conf
Heat stack on overcloud fails with: Resource CREATE failed: Error: Cannot get...
Status: CLOSED ERRATA
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-heat-templates (Show other bugs)
9.0 (Mitaka)
Unspecified Unspecified
urgent Severity urgent
: ga
: 9.0 (Mitaka)
Assigned To: Gaël Chamoulaud
Marius Cornea
: Triaged
Depends On:
Blocks:
  Show dependency tree / graph
 
Reported: 2016-05-27 08:46 EDT by Marius Cornea
Modified: 2016-10-24 05:32 EDT (History)
20 users (show)

See Also:
Fixed In Version: openstack-tripleo-heat-templates-2.0.0-17.el7ost
Doc Type: If docs needed, set a value
Doc Text:
In previous releases of Heat, domain resources were created before /etc/heat/heat.conf was configured on the overcloud. However, domain resources depended on settings from that file; as such, these resources were not created correctly, preventing users from creating heat stacks. Users had to manually restart the Pacemaker Heat Engine resource to work around the problem. This release corrects the sequence of steps for deploying the Heat service, thereby fixing the problem.
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-08-11 07:31:26 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
heat.conf (1.75 KB, text/plain)
2016-05-27 08:46 EDT, Marius Cornea 		no flags Details
Add an attachment (proposed patch, testcase, etc.)


External Trackers
Tracker ID Priority Status Summary Last Updated
Launchpad 1599232 None None None 2016-07-19 09:14 EDT
OpenStack gerrit 331652 None None None 2016-06-20 08:46 EDT
OpenStack gerrit 342032 None None None 2016-07-19 09:15 EDT
Red Hat Product Errata RHEA-2016:1599 normal SHIPPED_LIVE Red Hat OpenStack Platform 9 director Release Candidate Advisory 2016-08-11 11:25:37 EDT

  None (edit)
Description Marius Cornea 2016-05-27 08:46:23 EDT 
Created attachment 1162457 [details]
heat.conf

Description of problem:
Heat stack creation on overcloud fails with:

Heat stack on overcloud fails with: Resource CREATE failed: Error: Cannot get stack domain user token, no stack domain id configured, please fix your heat.conf 

Version-Release number of selected component (if applicable):
openstack-tripleo-heat-templates-2.0.0-8.el7ost.noarch

How reproducible:
100%

Steps to Reproduce:
1. Deploy overcloud 
2. Launch a heat stack on overcloud

Actual results:
Heat stack creation fails.

Expected results:
Heat stack creation succeeds.

Additional info:

Attaching the heat.conf on one of the controllers. 

It looks that there is no heat_stack Keystone domain created:

[root@overcloud-controller-0 heat]# mysql -e 'use keystone;select * from domain;'
+--------------------------+--------------------------+---------+-------+
| id                       | name                     | enabled | extra |
+--------------------------+--------------------------+---------+-------+
| <<keystone.domain.root>> | <<keystone.domain.root>> |       0 | {}    |
+--------------------------+--------------------------+---------+-------+
Comment 2 Marius Cornea 2016-05-27 09:09:27 EDT 
The heat_stack domain and user are set up:

[stack@undercloud ~]$ openstack --os-auth-url http://172.16.18.25:5000 --os-username admin --os-project-name admin --os-password AsxhfvUqmeTVVp33YTXCYPVjF --os-identity-api-version 3 domain list
+----------------------------------+------------+---------+--------------------+
| ID                               | Name       | Enabled | Description        |
+----------------------------------+------------+---------+--------------------+
| 8b38cc65098d405a8ffe6a2662ae634a | heat_stack | True    |                    |
| default                          | Default    | True    | The default domain |
+----------------------------------+------------+---------+--------------------+

[stack@undercloud ~]$ openstack --os-auth-url http://172.16.18.25:5000 --os-username admin --os-project-name admin --os-password AsxhfvUqmeTVVp33YTXCYPVjF --os-identity-api-version 3 user list
+----------------------------------+----------------------------------------+
| ID                               | Name                                   |
+----------------------------------+----------------------------------------+
| 4ae78026b6084014ad1d3429b1a48b2e | aodh                                   |
| 4d50cf7d684743afa9406efe0739b58c | neutron                                |
| 5e67934b18cc43ea8ebfbc85ea02f982 | glance                                 |
| 605e371c6f1f402c9f8baf558248eaed | sahara                                 |
| 61a30ca59daa45a3b7578f0849946649 | cinder                                 |
| 75876d2a3a6d432095870bbc43586210 | vm04-vm04-e7sxmlyfjdn4-wh-ch2vctxpcyho |
| 790ac57025104891ae2f7987d6faeff2 | admin                                  |
| 991bd3484ea14a5d9f499b7f2f1a7314 | heat                                   |
| adce358666c74531ae4b2a05ea72a194 | ceilometer                             |
| d760c4aed7324e3fa6d6d746d28d7046 | heat_stack_domain_admin                |
| dbf66c766b6a4133aac5a38fd942b7ed | gnocchi                                |
| e20607846ceb4f3c9b8573da5b7d6c84 | nova                                   |
| f084df27799a49efa6d6c2696275b6aa | cinderv2                               |
| f9e209747e6b4e7b9c18bb68eeb3a0f3 | swift                                  |
+----------------------------------+----------------------------------------+
Comment 3 Thomas Hervé 2016-05-27 10:45:22 EDT 
Looking the logs, it seems that heat is started and the configuration is changed afterwards. Indeed, restarting heat-engine on all the controllers fix the issue.

It's most likely a problem in puppet-heat doing the keystone domain configuration too late, or not forcing a heat restart once done.
Comment 4 Marius Cornea 2016-06-02 05:36:57 EDT 
OK, so it seems I was able to reproduce this on 8 as well. Indeed after running pcs resource restart openstack-heat-engine-clone on the overcloud controller I was able to create a stack.
Comment 5 Kevin Jones 2016-06-02 21:17:52 EDT 
I reproduced this as well on OSP 8 deployment (w nuage integration) in a lab environment.

Running this as suggested on my controller also resolved the issue:
pcs resource restart openstack-heat-engine-clone
Comment 7 Mike Orazi 2016-06-27 16:30:10 EDT 
We moved this out of post -> assigned as I bel8ieve reproducing has become an issue.  Gael or Marius, can you update with the present status please?
Comment 10 Emilien Macchi 2016-06-30 11:06:18 EDT 
I have a question for Heat folks:

what needs to happen *before* starting Heat:

1) configure heat.conf (already done in puppet-heat, see[1]) ?
2) create Keystone domain, user and user role for heat_admin ?

[1] https://github.com/openstack/puppet-heat/blob/master/manifests/deps.pp#L17

In both case, it seems we have an orchestration issue.
Gaël is working on https://review.openstack.org/#/c/331652/ but I don't agree with this change, it's not the right way to fix the problem. We need to use puppet orchestration to solve it, that's why I need to know what needs to happen before starting Heat.
Comment 11 Thomas Hervé 2016-06-30 13:14:27 EDT 
Heat needs stack_user_domain_id or stack_user_domain_name config options set in the heat.conf before starting. The domain itself doesn't need to exist until a stack is actually created, but the value needs to be there. Considering the exception that we got, it would seem it's not when heat-engine starts.
Comment 16 Jaromir Coufal 2016-07-13 18:00:21 EDT 
Does it fail in all cases? If not, what are the cases, what is the severity?
Comment 17 Jaromir Coufal 2016-07-13 18:01:11 EDT 
Do we have workaround?
Comment 18 Marius Cornea 2016-07-14 03:39:19 EDT 
(In reply to Jaromir Coufal from comment #17)
> Do we have workaround?

Yes, the workaround is easy: run 'pcs resource restart openstack-heat-engine-clone' after deployment is done.

(In reply to Jaromir Coufal from comment #16)
> Does it fail in all cases? If not, what are the cases, what is the severity?

No, it doesn't fail in all cases, only with particular templates, i.e. from my testing stack that creates vm, network, volume, attaches volume, writes a file system and mount it inside and signals back once it's done.
Comment 19 Luigi Toscano 2016-07-21 09:07:55 EDT 
Just a note to say that this blocks Sahara, which uses Heat as the engine to spawn Big Data clusters.
Comment 22 errata-xmlrpc 2016-08-11 07:31:26 EDT 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHEA-2016-1599.html
Note You need to log in before you can comment on or make changes to this bug.
First Last Prev Next    This bug is not in your last search results.