Bug 1414271 - Extend configuration for Gnocchi at install/setup time through Director. [NEEDINFO]
Summary: Extend configuration for Gnocchi at install/setup time through Director.
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-heat-templates
Version: 10.0 (Newton)
Hardware: Unspecified
OS: Unspecified
medium
low
Target Milestone: ga
: 12.0 (Pike)
Assignee: Pradeep Kilambi
QA Contact: Sasha Smolyak
URL:
Whiteboard: FailedQA
Depends On:
Blocks: 1470821
TreeView+ depends on / blocked
 
Reported: 2017-01-18 08:46 UTC by Irina Petrova
Modified: 2020-07-16 09:07 UTC (History)
10 users (show)

Fixed In Version: openstack-tripleo-heat-templates-7.0.3-11.el7ost puppet-gnocchi-11.3.0-1.el7ost
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-12-13 21:00:46 UTC
Target Upstream Version:
ssmolyak: needinfo? (pkilambi)


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
OpenStack gerrit 511233 0 None MERGED Add processing delay param for metricd 2020-07-15 07:13:54 UTC
OpenStack gerrit 511236 0 None MERGED Set metric procssing delay for metricd 2020-07-15 07:13:54 UTC
OpenStack gerrit 515683 0 None MERGED Add processing delay param for metricd 2020-07-15 07:13:54 UTC
OpenStack gerrit 515684 0 None MERGED Set metric procssing delay for metricd 2020-07-15 07:13:54 UTC
Red Hat Product Errata RHEA-2017:3462 0 normal SHIPPED_LIVE Red Hat OpenStack Platform 12.0 Enhancement Advisory 2018-02-16 01:43:25 UTC

Description Irina Petrova 2017-01-18 08:46:49 UTC
Description of problem:

Based on some performance tests:

   ~~~
   The installer will configure much less metricd processing workers than
   you will likely require for large scale with the defaults.
   You may need to tune metric_processing_delay at the expense of system
   resources to obtain your aggregated measures closer to realtime.
   Due to the higher system usage (cpu usage) on the machines hosting
   metricd, I would advise moving Gnocchi API and metricd to a separate
   machine. 
   Ceph needs pgs tuned for metrics pool (In my case I used another yaml
   in tripleo to bump the number of PGs to 512 (36 total OSDs)).

   ...
   In order to get the 1,000 instances all on the low archival-policy
   setting with default metrics collected I had to adjust two settings to
   gnocchi.conf                     <----------------------
   workers = 48                     <----------------------
   metric_processing_delay = 30     <----------------------
   ~~~

We recommended to a customer to tweak the number of workers and metric_processing_delay. The results were promising but as we don't see any metric_processing_delay parameter in the gnocchi puppet manifests [1], we would like to request them added.

If there's a better way to configure/update those via TripleO, please move the BZ to the appropriate BZ Component and or advise on it.

The goal is simple:
Configure gnocchi.conf at install/setup time through the Director.


[1] https://github.com/openstack/puppet-gnocchi/tree/stable/newton/manifests

Comment 2 Pradeep Kilambi 2017-05-02 14:09:49 UTC
The pupet patches have landed. I'll check the builds and move to modified.

Comment 5 Sasha Smolyak 2017-07-13 12:24:22 UTC
What about changeable number of workers? Did we drop that feature?

Comment 6 Pradeep Kilambi 2017-07-13 14:02:55 UTC
The workers support is already in place:

https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/services/gnocchi-metricd.yaml#L32-L35

hence not in the patch listed.

Comment 12 Jon Schlueter 2017-11-21 21:25:44 UTC
openstack-tripleo-heat-templates-7.0.3-11.el7ost

Comment 16 Sasha Smolyak 2017-12-13 12:12:11 UTC
There are still remainings of the same code in /usr/share/openstack-tripleo-heat-templates/puppet/services/gnocchi-base.yaml:

  MetricProcessingDelay:
    default: 30
    description: Delay between processing metrics.

        gnocchi::storage::metric_processing_delay: {get_param: MetricProcessingDelay}

Don't we need to clean it from there?

Comment 19 errata-xmlrpc 2017-12-13 21:00:46 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:3462


Note You need to log in before you can comment on or make changes to this bug.