Bug 1358345 - Add a base constraint so gnocchi services should start after galera
Summary: Add a base constraint so gnocchi services should start after galera
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-heat-templates
Version: 9.0 (Mitaka)
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ga
: 9.0 (Mitaka)
Assignee: Pradeep Kilambi
QA Contact: Yurii Prokulevych
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-07-20 14:42 UTC by Pradeep Kilambi
Modified: 2016-08-11 11:36 UTC (History)
6 users (show)

Fixed In Version: openstack-tripleo-heat-templates-2.0.0-23.el7ost
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-08-11 11:36:44 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHEA-2016:1599 normal SHIPPED_LIVE Red Hat OpenStack Platform 9 director Release Candidate Advisory 2016-08-11 15:25:37 UTC
OpenStack gerrit 344823 None None None 2016-07-20 14:42:38 UTC

Description Pradeep Kilambi 2016-07-20 14:42:39 UTC
When we restart the cluster sometime statsd comes up before galera causing connection issues and fails. Add a constraint to order so statsd starts after openstack-core-clone. 

Relevant traceback from yurii:

2016-07-20 11:57:54.399 6045 CRITICAL gnocchi [-] StorageError: Unable to start coordinator: Error while reading from socket: ('Connection closed by server.',)
2016-07-20 11:57:54.399 6045 ERROR gnocchi Traceback (most recent call last):
2016-07-20 11:57:54.399 6045 ERROR gnocchi   File "/usr/bin/gnocchi-statsd", line 10, in <module>
2016-07-20 11:57:54.399 6045 ERROR gnocchi     sys.exit(statsd())
2016-07-20 11:57:54.399 6045 ERROR gnocchi   File "/usr/lib/python2.7/site-packages/gnocchi/cli.py", line 75, in statsd
2016-07-20 11:57:54.399 6045 ERROR gnocchi     statsd_service.start()
2016-07-20 11:57:54.399 6045 ERROR gnocchi   File "/usr/lib/python2.7/site-packages/gnocchi/statsd.py", line 174, in start
2016-07-20 11:57:54.399 6045 ERROR gnocchi     stats = Stats(conf)
2016-07-20 11:57:54.399 6045 ERROR gnocchi   File "/usr/lib/python2.7/site-packages/gnocchi/statsd.py", line 38, in __init__
2016-07-20 11:57:54.399 6045 ERROR gnocchi     self.storage = storage.get_driver(self.conf)
2016-07-20 11:57:54.399 6045 ERROR gnocchi   File "/usr/lib/python2.7/site-packages/gnocchi/storage/__init__.py", line 155, in get_driver
2016-07-20 11:57:54.399 6045 ERROR gnocchi     return get_driver_class(conf)(conf.storage)
2016-07-20 11:57:54.399 6045 ERROR gnocchi   File "/usr/lib/python2.7/site-packages/gnocchi/storage/ceph.py", line 61, in __init__
2016-07-20 11:57:54.399 6045 ERROR gnocchi     super(CephStorage, self).__init__(conf)
2016-07-20 11:57:54.399 6045 ERROR gnocchi   File "/usr/lib/python2.7/site-packages/gnocchi/storage/_carbonara.py", line 61, in __init__
2016-07-20 11:57:54.399 6045 ERROR gnocchi     raise storage.StorageError("Unable to start coordinator: %s" % e)
2016-07-20 11:57:54.399 6045 ERROR gnocchi StorageError: Unable to start coordinator: Error while reading from socket: ('Connection closed by server.',)
2016-07-20 11:57:54.399 6045 ERROR gnocchi

Comment 4 Yurii Prokulevych 2016-07-29 12:01:24 UTC
New constraint added :
----------------------
pcs constraint order | grep gnocchi
  start openstack-core-clone then start openstack-gnocchi-metricd-clone (kind:Mandatory)
  start openstack-gnocchi-metricd-clone then start openstack-gnocchi-statsd-clone (kind:Mandatory)

Verified with openstack-tripleo-heat-templates-2.0.0-24.el7ost.noarch

Deploy command:
openstack  overcloud deploy --libvirt-type qemu \
    --ntp-server clock.redhat.com --templates \
    --control-scale 3 --compute-scale 1 --ceph-storage-scale 1 \
    -e storage-environment.yaml 2>&1

cat storage-environment.yaml
parameter_defaults:
  CinderEnableIscsiBackend: false
  CinderEnableRbdBackend: true
  NovaEnableRbdBackend: false
  GlanceBackend: rbd
  GnocchiBackend: file

Comment 6 errata-xmlrpc 2016-08-11 11:36:44 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHEA-2016-1599.html


Note You need to log in before you can comment on or make changes to this bug.