Bug 1790551 - Etcd storage requirements
Summary: Etcd storage requirements
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Documentation
Version: 4.2.z
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: 4.2.z
Assignee: Tami Love
QA Contact: Mike Fiedler
Latha S
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-01-13 15:35 UTC by Keith Fryklund
Modified: 2024-01-06 04:27 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-06-01 12:48:26 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 1868922 0 high CLOSED [Docs] Document debugging cluster performance issues with Prometheus 2024-03-25 16:18:00 UTC
Red Hat Knowledge Base (Solution) 5060021 0 None None None 2020-10-07 08:43:19 UTC

Internal Links: 1868922

Description Keith Fryklund 2020-01-13 15:35:39 UTC
Document URL: 
https://docs.openshift.com/container-platform/4.2/scalability_and_performance/optimizing-storage.html

Section Number and Name: 
4.2/scalability_and_performance/optimizing-storage

Describe the issue: 
I have a production OCP4 env on RHOS that I'm planning, and I want to make sure my etcd database has the necessary storage configuration to sustain a large cluster. In the documentation linked above, it says, "the lowest consistent latency storage technology is preferable", which is kind of ambiguous.  In my case, I've tried using ceph and it ended up not being fast enough so we're now going with local ssd.  

Suggestions for improvement:  
Requesting that we add etcd storage reference architecture, suggested etcd storage solutions, and/or storage performance requirements/tests we can run to evaluate our etcd storage solution.  

Additional information:

Comment 4 Mike Fiedler 2020-11-17 14:27:19 UTC
etcd speed (SSD/NVME, no networked storage, no spinning disk), size and monitoring recommendations have been improved in 4.6.   We recommend monitoring

etcd_server_quota_backend_bytes which is the current quota limit 
etcd_mvcc_db_total_size_in_use_in_bytes which indicates the actual database usage after a history compaction
etcd_debugging_mvcc_db_total_size_in_bytes

See https://docs.openshift.com/container-platform/4.6/scalability_and_performance/recommended-host-practices.html#recommended-etcd-practices_

Comment 6 Mike Fiedler 2020-11-17 20:06:04 UTC
+1 to suggestion by @oarribas.  Thanks.

Comment 13 Tami Love 2022-03-23 16:21:39 UTC
@kfryklun @oarribas Review this PR that provides a link to etcd storage recommendations. Thanks.

https://github.com/openshift/openshift-docs/pull/43626

Comment 16 Tami Love 2022-04-27 19:18:25 UTC
@oarribas @mifiedle I left a review request for the BZ in https://github.com/openshift/openshift-docs/pull/43626. Thank you!

Comment 21 Mike Fiedler 2022-05-26 00:31:07 UTC
Approved in PR, waiting for it to merge

Comment 22 Tami Love 2022-05-26 02:15:35 UTC
An additional change was made to the content, per the peer review, and the review request is out for Dev and QE verification. The PR link is here for more information: https://github.com/openshift/openshift-docs/pull/43626

Comment 23 Tami Love 2022-06-01 12:48:26 UTC
PR merged.

Comment 24 Tami Love 2022-06-01 12:52:56 UTC
PR merged.

Comment 25 Red Hat Bugzilla 2024-01-06 04:27:32 UTC
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 120 days


Note You need to log in before you can comment on or make changes to this bug.