Bug 1970913 - [RHCS Containers] Setting environment variable TCMALLOC_MAX_TOTAL_THREAD_CACHE_BYTES for all the ceph daemons
Summary: [RHCS Containers] Setting environment variable TCMALLOC_MAX_TOTAL_THREAD_CACH...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Ceph Storage
Classification: Red Hat Storage
Component: Ceph-Ansible
Version: 4.2
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
: 4.2z3
Assignee: Dimitri Savineau
QA Contact: Ameena Suhani S H
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-06-11 12:57 UTC by Vikhyat Umrao
Modified: 2021-09-27 18:26 UTC (History)
12 users (show)

Fixed In Version: ceph-ansible-4.0.61-1.el8cp, ceph-ansible-4.0.61-1.el7cp
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-09-27 18:26:24 UTC
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github ceph ceph-ansible pull 6640 0 None open container: set tcmalloc value by default 2021-06-29 17:28:10 UTC
Red Hat Issue Tracker RHCEPH-1895 0 None None None 2021-09-27 15:20:17 UTC
Red Hat Product Errata RHBA-2021:3670 0 None None None 2021-09-27 18:26:47 UTC

Description Vikhyat Umrao 2021-06-11 12:57:26 UTC
Description of problem:

[RHCS Containers] Setting environment variable TCMALLOC_MAX_TOTAL_THREAD_CACHE_BYTES for all the ceph daemons

I have verified it is set properly in rpm-based installations maybe because of /etc/sysconfig/ceph has this variable set in rpm-based installation.

Coming to containers we have to set the following options:

1. for osds: In all.yml 
ceph_tcmalloc_max_total_thread_cache: 134217728

2. For mons: In mons.yml
ceph_mon_docker_extra_env: "-e TCMALLOC_MAX_TOTAL_THREAD_CACHE_BYTES={{ ceph_tcmalloc_max_total_thread_cache }}"


3. For mgrs: In mgrs.yml
ceph_mgr_docker_extra_env: "-e TCMALLOC_MAX_TOTAL_THREAD_CACHE_BYTES={{ ceph_tcmalloc_max_total_thread_cache }}"


4. For rgws: In rgws.yml 
ceph_rgw_docker_extra_env: "-e TCMALLOC_MAX_TOTAL_THREAD_CACHE_BYTES={{ ceph_tcmalloc_max_total_thread_cache }}"

We need a fix where this is all set by default in RHCS 4.x containers clusters and we do not have to set it manually. The default value for this variable will be 134217728 (128MB).

For RHCS 5.0 we are handling this here - https://bugzilla.redhat.com/show_bug.cgi?id=1970597

Version-Release number of selected component (if applicable):
RHCS 4.2

Comment 1 Vikhyat Umrao 2021-06-11 13:57:30 UTC
(In reply to Vikhyat Umrao from comment #0)
> Description of problem:
> 
> [RHCS Containers] Setting environment variable
> TCMALLOC_MAX_TOTAL_THREAD_CACHE_BYTES for all the ceph daemons
> 
> I have verified it is set properly in rpm-based installations maybe because
> of /etc/sysconfig/ceph has this variable set in rpm-based installation.
> 
> Coming to containers we have to set the following options:
> 
> 1. for osds: In all.yml 
> ceph_tcmalloc_max_total_thread_cache: 134217728
> 
> 2. For mons: In mons.yml
> ceph_mon_docker_extra_env: "-e TCMALLOC_MAX_TOTAL_THREAD_CACHE_BYTES={{
> ceph_tcmalloc_max_total_thread_cache }}"
> 
> 
> 3. For mgrs: In mgrs.yml
> ceph_mgr_docker_extra_env: "-e TCMALLOC_MAX_TOTAL_THREAD_CACHE_BYTES={{
> ceph_tcmalloc_max_total_thread_cache }}"
> 
> 
> 4. For rgws: In rgws.yml 
> ceph_rgw_docker_extra_env: "-e TCMALLOC_MAX_TOTAL_THREAD_CACHE_BYTES={{
> ceph_tcmalloc_max_total_thread_cache }}"
> 
> We need a fix where this is all set by default in RHCS 4.x containers
> clusters and we do not have to set it manually. The default value for this
> variable will be 134217728 (128MB).
> 
> For RHCS 5.0 we are handling this here -
> https://bugzilla.redhat.com/show_bug.cgi?id=1970597
> 
> Version-Release number of selected component (if applicable):
> RHCS 4.2

I forgot to mention MDS we also need for MDS :)

Comment 13 errata-xmlrpc 2021-09-27 18:26:24 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Red Hat Ceph Storage 4.2 Bug Fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:3670


Note You need to log in before you can comment on or make changes to this bug.