Bug 1365880 - Gnocchi statsd gets configured only on first controller in HA environment
Summary: Gnocchi statsd gets configured only on first controller in HA environment
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: puppet-tripleo
Version: 10.0 (Newton)
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: rc
: 10.0 (Newton)
Assignee: RHOS Maint
QA Contact: Pradeep Kilambi
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-08-10 12:00 UTC by Marius Cornea
Modified: 2016-12-14 15:50 UTC (History)
11 users (show)

Fixed In Version: puppet-tripleo-5.2.0-1.el7ost
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-12-14 15:50:57 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Launchpad 1604708 0 None None None 2016-08-10 12:00:52 UTC
OpenStack gerrit 345000 0 None None None 2016-08-10 14:40:12 UTC
Red Hat Product Errata RHEA-2016:2948 0 normal SHIPPED_LIVE Red Hat OpenStack Platform 10 enhancement update 2016-12-14 19:55:27 UTC

Description Marius Cornea 2016-08-10 12:00:53 UTC
Description of problem:
Gnocchi statsd gets configured only on first controller in HA environment and it fails to start on the rest of 2 controllers:

Deploy command:

source ~/stackrc
export THT='/home/stack/tripleo-heat-templates'
openstack overcloud deploy --templates $THT \
-e $THT/environments/network-isolation.yaml \
-e $THT/environments/network-management.yaml \
-e ~/templates/network-environment.yaml \
-e $THT/environments/hyperconverged-ceph.yaml \
-e $THT/environments/puppet-ceph-devel.yaml \
-e $THT/environments/puppet-pacemaker.yaml \
-e ~/templates/disk-layout.yaml \
--control-scale 3 \
--control-flavor controller \
--compute-scale 3 \
--compute-flavor compute \
--ntp-server clock.redhat.com \
--libvirt-type qemu

on overcloud-controller-0:

[root@overcloud-controller-0 ~]# grep -v ^# /etc/gnocchi/gnocchi.conf | grep -v ^$
[DEFAULT]
log_dir = /var/log/gnocchi
[api]
max_limit = 1000
host=10.0.0.11
workers=4
port=8041
[archive_policy]
[cors]
[cors.subdomain]
[database]
[indexer]
url = mysql+pymysql://gnocchi:qjwaGYdYh9gtPfBTsW7QC2pgn.0.12/gnocchi
[metricd]
[oslo_middleware]
[oslo_policy]
policy_file = /etc/gnocchi/policy.json
[statsd]
resource_id = 0a8b55df-f90f-491c-8cb9-7cdecec6fc26
user_id = 27c0d3f8-e7ee-42f0-8317-72237d1c5ae3
project_id = 6c38cd8d-099a-4cb2-aecf-17be688e8616
archive_policy_name = low
flush_delay = 10
[storage]
driver = ceph
ceph_pool = metrics
ceph_username = openstack
ceph_keyring = /etc/ceph/ceph.client.openstack.keyring
ceph_conffile = /etc/ceph/ceph.conf
[keystone_authtoken]
auth_uri=http://10.0.0.12:5000/v2.0
identity_uri=http://192.168.0.15:35357
admin_user=gnocchi
admin_password=qjwaGYdYh9gtPfBTsW7QC2pgn
admin_tenant_name=service

On controller-1:

[root@overcloud-controller-1 ~]# grep -v ^# /etc/gnocchi/gnocchi.conf | grep -v ^$
[DEFAULT]
log_dir = /var/log/gnocchi
[api]
max_limit = 1000
host=10.0.0.10
workers=4
port=8041
[archive_policy]
[cors]
[cors.subdomain]
[database]
[indexer]
url = mysql+pymysql://gnocchi:qjwaGYdYh9gtPfBTsW7QC2pgn.0.12/gnocchi
[metricd]
[oslo_middleware]
[oslo_policy]
policy_file = /etc/gnocchi/policy.json
[statsd]
[storage]
driver = ceph
ceph_pool = metrics
ceph_username = openstack
ceph_keyring = /etc/ceph/ceph.client.openstack.keyring
ceph_conffile = /etc/ceph/ceph.conf
[keystone_authtoken]
auth_uri=http://10.0.0.12:5000/v2.0
identity_uri=http://192.168.0.15:35357
admin_user=gnocchi
admin_password=qjwaGYdYh9gtPfBTsW7QC2pgn
admin_tenant_name=service

The service fails to start on controller-1 and controller-2:

CRITICAL gnocchi [-] RequiredOptError: value required for option: statsd.resource_id
ERROR gnocchi Traceback (most recent call last):
ERROR gnocchi File "/usr/bin/gnocchi-statsd", line 10, in <module>
ERROR gnocchi sys.exit(statsd())
ERROR gnocchi File "/usr/lib/python2.7/site-packages/gnocchi/cli.py", line 70, in statsd
ERROR gnocchi statsd_service.start()
ERROR gnocchi File "/usr/lib/python2.7/site-packages/gnocchi/statsd.py", line 172, in start
ERROR gnocchi raise cfg.RequiredOptError(field, cfg.OptGroup("statsd"))
ERROR gnocchi RequiredOptError: value required for option: statsd.resource_id
ERROR gnocchi

Version-Release number of selected component (if applicable):
openstack-tripleo-heat-templates-5.0.0-0.20160725225924.77f6c40.1.el7ost.noarch

How reproducible:
100$

Steps to Reproduce:
1. Deploy overcloud with pacemaker
2. Check gnocchi-statsd status

Actual results:
[root@overcloud-controller-1 heat-admin]# pcs status | grep -A2 gnocchi-statsd
 Clone Set: openstack-gnocchi-statsd-clone [openstack-gnocchi-statsd]
     Started: [ overcloud-controller-0 ]
     Stopped: [ overcloud-controller-1 overcloud-controller-2 ]
--
* openstack-gnocchi-statsd_start_0 on overcloud-controller-1 'not running' (7): call=97, status=complete, exitreason='none',
    last-rc-change='Wed Aug 10 11:50:39 2016', queued=0ms, exec=2182ms
* openstack-gnocchi-statsd_start_0 on overcloud-controller-2 'not running' (7): call=100, status=complete, exitreason='none',
    last-rc-change='Wed Aug 10 11:50:39 2016', queued=0ms, exec=2180ms


Expected results:
Gnocchi statsd is started on all controllers.

Additional info:

Comment 7 errata-xmlrpc 2016-12-14 15:50:57 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHEA-2016-2948.html


Note You need to log in before you can comment on or make changes to this bug.