Bug 1269002 - instack-undercloud: overcloud HA deployment fails - the rabbitmq doesn't run on the controllers.
instack-undercloud: overcloud HA deployment fails - the rabbitmq doesn't run ...
Status: CLOSED CURRENTRELEASE
Product: RDO
Classification: Community
Component: rdo-manager (Show other bugs)
unspecified
x86_64 Linux
high Severity unspecified
: GA
: Liberty
Assigned To: Giulio Fidente
yeylon@redhat.com
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2015-10-05 18:08 EDT by Alexander Chuzhoy
Modified: 2016-04-18 02:51 EDT (History)
6 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-04-01 08:14:37 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
/var/log/messages from compute (252.73 KB, application/x-gzip)
2015-10-05 18:13 EDT, Alexander Chuzhoy
no flags Details

  None (edit)
Description Alexander Chuzhoy 2015-10-05 18:08:17 EDT
instack-undercloud: overcloud HA deployment fails - the rabbitmq doesn't run on the controllers.


Step to reproduce:
Attempt to deploy overcloud with "openstack overcloud deploy --templates --control-scale 3 --compute-scale 1 --timeout 90 --ntp-server x.x.x.x"

Result:
The overcloud deployment fails:
[stack@instack ~]$ heat resource-list -n 5 overcloud|grep FAIL
| ComputeNodesPostDeployment                | 46fec2f4-4542-471c-bbba-f3503297af35          | OS::TripleO::ComputePostDeployment                | CREATE_FAILED      | 2015-10-05T19:48:19 | overcloud                                                                              |
| ControllerBootstrapNodeDeployment         | 646505ac-12fc-4c6d-b8a0-82aefcb81117          | OS::Heat::StructuredDeployments                   | CREATE_FAILED      | 2015-10-05T19:48:19 | overcloud                                                                              |
| ControllerClusterDeployment               | 51ab7384-a669-4d7f-80be-2e706380a2c4          | OS::Heat::StructuredDeployments                   | CREATE_FAILED      | 2015-10-05T19:48:19 | overcloud                                                                              |
| ControllerSwiftDeployment                 | 7022a48b-7a82-43ca-83b9-322229d27ac2          | OS::Heat::StructuredDeployments                   | CREATE_FAILED      | 2015-10-05T19:48:19 | overcloud                                                                              |
| VipDeployment                             | 32299bf3-a1b5-4432-8cfc-99cd49446dfe          | OS::Heat::StructuredDeployments                   | CREATE_FAILED      | 2015-10-05T19:48:19 | overcloud                                                                              |
| ComputePuppetDeployment                   | 9f5d2a90-c8cc-467f-ade1-9d00a93cc1a3          | OS::Heat::StructuredDeployments                   | CREATE_FAILED      | 2015-10-05T20:00:03 | overcloud-ComputeNodesPostDeployment-himyngfw7azo    







[stack@instack ~]$ heat resource-show overcloud  ComputeNodesPostDeployment
+------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Property               | Value                                                                                                                                                            |
+------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| attributes             | {}                                                                                                                                                               |
| creation_time          | 2015-10-05T19:48:19                                                                                                                                              |
| description            |                                                                                                                                                                  |
| links                  | http://192.0.2.1:8004/v1/b4752199bdb2492faaee42746a11254d/stacks/overcloud/1c9c8374-ce36-47d8-a8de-d1d77690bdc2/resources/ComputeNodesPostDeployment (self)      |
|                        | http://192.0.2.1:8004/v1/b4752199bdb2492faaee42746a11254d/stacks/overcloud/1c9c8374-ce36-47d8-a8de-d1d77690bdc2 (stack)                                          |
|                        | http://192.0.2.1:8004/v1/b4752199bdb2492faaee42746a11254d/stacks/overcloud-ComputeNodesPostDeployment-himyngfw7azo/46fec2f4-4542-471c-bbba-f3503297af35 (nested) |
| logical_resource_id    | ComputeNodesPostDeployment                                                                                                                                       |
| physical_resource_id   | 46fec2f4-4542-471c-bbba-f3503297af35                                                                                                                             |
| required_by            |                                                                                                                                                                  |
| resource_name          | ComputeNodesPostDeployment                                                                                                                                       |
| resource_status        | CREATE_FAILED                                                                                                                                                    |
| resource_status_reason | CREATE aborted                                                                                                                                                   |
| resource_type          | OS::TripleO::ComputePostDeployment                                                                                                                               |
| updated_time           | 2015-10-05T19:48:19                                                                                                                                              |
+------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+


Repeating messages on the compute:
ct  5 22:01:07 localhost neutron-openvswitch-agent: 2015-10-05 22:01:07.948 5684 ERROR oslo.messaging._drivers.impl_rabbit [req-f4b95faa-02c1-4600-9504-80aed60d4f07 - - - - -] AMQP server on 192.0.2.19:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 1 seconds.
Oct  5 22:01:07 localhost neutron-openvswitch-agent: 2015-10-05 22:01:07.949 5684 ERROR oslo.messaging._drivers.impl_rabbit [-] AMQP server on 192.0.2.19:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 2 seconds.



the rabbitmq doesn't run on the controllers.
Comment 1 Alexander Chuzhoy 2015-10-05 18:13 EDT
Created attachment 1080083 [details]
/var/log/messages from compute
Comment 2 Alexander Chuzhoy 2015-10-05 18:14:00 EDT
Environment:
instack-0.0.8-dev4.el7.centos.noarch
instack-undercloud-2.1.3-dev219.el7.centos.noarch
Comment 3 Alexander Chuzhoy 2015-10-05 18:15:15 EDT
Used Centos 7.1 and followed the procedure here:
http://docs.openstack.org/developer/tripleo-docs/basic_deployment/basic_deployment_cli.html
Comment 4 Mike Orazi 2015-10-13 17:27:40 EDT
This may be fixed by some combination of:  https://bugzilla.redhat.com/show_bug.cgi?id=1270964 and https://bugzilla.redhat.com/show_bug.cgi?id=1270325

Can we confirm whether the fixes for those 2 issues makes this go away?
Comment 5 Giulio Fidente 2015-10-14 04:16:34 EDT
Alex, I was unable to reproduce this; can you post which version of the openstack-tripleo-heat-templates you have installed on the undercloud and which version of openstack-puppet-modules you have installed on one of the overcloud controllers?
Comment 6 Alexander Chuzhoy 2015-10-14 10:39:26 EDT
I re-deployed the setup (newer version).
I'm unable to reproduce this so far.

Note You need to log in before you can comment on or make changes to this bug.