Bug 1790551

Summary:	Etcd storage requirements
Product:	OpenShift Container Platform	Reporter:	Keith Fryklund <kfryklun>
Component:	Documentation	Assignee:	Tami Love <tlove>
Status:	CLOSED NOTABUG	QA Contact:	Mike Fiedler <mifiedle>
Severity:	medium	Docs Contact:	Latha S <lmurthy>
Priority:	medium
Version:	4.2.z	CC:	ahoffer, aos-bugs, erich, jmelvin, lmurthy, mifiedle, oarribas, tlove
Target Milestone:	---	Keywords:	Reopened
Target Release:	4.2.z
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2022-06-01 12:48:26 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Keith Fryklund 2020-01-13 15:35:39 UTC

Document URL: 
https://docs.openshift.com/container-platform/4.2/scalability_and_performance/optimizing-storage.html

Section Number and Name: 
4.2/scalability_and_performance/optimizing-storage

Describe the issue: 
I have a production OCP4 env on RHOS that I'm planning, and I want to make sure my etcd database has the necessary storage configuration to sustain a large cluster. In the documentation linked above, it says, "the lowest consistent latency storage technology is preferable", which is kind of ambiguous.  In my case, I've tried using ceph and it ended up not being fast enough so we're now going with local ssd.  

Suggestions for improvement:  
Requesting that we add etcd storage reference architecture, suggested etcd storage solutions, and/or storage performance requirements/tests we can run to evaluate our etcd storage solution.  

Additional information:

Comment 4 Mike Fiedler 2020-11-17 14:27:19 UTC

etcd speed (SSD/NVME, no networked storage, no spinning disk), size and monitoring recommendations have been improved in 4.6.   We recommend monitoring

etcd_server_quota_backend_bytes which is the current quota limit 
etcd_mvcc_db_total_size_in_use_in_bytes which indicates the actual database usage after a history compaction
etcd_debugging_mvcc_db_total_size_in_bytes

See https://docs.openshift.com/container-platform/4.6/scalability_and_performance/recommended-host-practices.html#recommended-etcd-practices_

Comment 5 oarribas 2020-11-17 15:15:47 UTC

Maybe a link from [1] to [2] would be helpful to easily find the additional information regarding the disks recommendations for etcd.


[1] https://docs.openshift.com/container-platform/4.6/scalability_and_performance/optimizing-storage.html#other-specific-application-storage-recommendations
[2] https://docs.openshift.com/container-platform/4.6/scalability_and_performance/recommended-host-practices.html#recommended-etcd-practices_

Comment 6 Mike Fiedler 2020-11-17 20:06:04 UTC

+1 to suggestion by @oarribas.  Thanks.

Comment 13 Tami Love 2022-03-23 16:21:39 UTC

@kfryklun @oarribas Review this PR that provides a link to etcd storage recommendations. Thanks.

https://github.com/openshift/openshift-docs/pull/43626

Comment 16 Tami Love 2022-04-27 19:18:25 UTC

@oarribas @mifiedle I left a review request for the BZ in https://github.com/openshift/openshift-docs/pull/43626. Thank you!

Comment 21 Mike Fiedler 2022-05-26 00:31:07 UTC

Approved in PR, waiting for it to merge

Comment 22 Tami Love 2022-05-26 02:15:35 UTC

An additional change was made to the content, per the peer review, and the review request is out for Dev and QE verification. The PR link is here for more information: https://github.com/openshift/openshift-docs/pull/43626

Comment 23 Tami Love 2022-06-01 12:48:26 UTC

PR merged.

Comment 24 Tami Love 2022-06-01 12:52:56 UTC

PR merged.

Comment 25 Red Hat Bugzilla 2024-01-06 04:27:32 UTC

The needinfo request[s] on this closed bug have been removed as they have been unresolved for 120 days