Bug 1275994

Summary:	Incorrect recommendation on pg_num and pgp_num settings in installation guide
Product:	[Red Hat Storage] Red Hat Ceph Storage	Reporter:	Taco Scargo <taco>
Component:	Documentation	Assignee:	John Wilkins <jowilkin>
Status:	CLOSED CURRENTRELEASE	QA Contact:	ceph-qe-bugs <ceph-qe-bugs>
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	1.3.0	CC:	asriram, hnallurv, hyelloji, kdreyer, ngoswami, tganguly, vumrao
Target Milestone:	rc	Keywords:	Documentation, ZStream
Target Release:	1.3.2
Hardware:	All
OS:	All
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2016-03-01 08:21:54 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Taco Scargo 2015-10-28 10:38:01 UTC

Description of problem:
In the RHCS 1.3 installation guide it is recommended to set osd_pool_default_pg_num and osd_pool_default_pgp_num to a too high number, causing too many PGs

Version-Release number of selected component (if applicable):
1.3

How reproducible:
always

Steps to Reproduce:
1. install RHCS using the steps described in the manual
2. configure ceph.conf as described in chapter 7 including osd_pool_default_pg_num and osd_pool_default_pgp_num. 
3. install rgw and start (which will create all rgw pools automatically using the values defined in step 3

Actual results:
too many pgs (e.g. 21000+ for 50 OSD cluster)

Expected results:
approx. 2000 pgs for a 50 OSD cluster

Additional info:

According to Tyler Brekke the osd_pool_default_pg_num and osd_pool_default_pg_num are never set up, and definitely not with these kind of high numbers. Recommendation is to use lower numbers and create all rgw pools manually.

Comment 2 John Wilkins 2015-11-02 22:47:23 UTC

The comments were only present in the 1.3 branch. These were removed as evidenced here:

https://gitlab.cee.redhat.com/jowilkin/red-hat-ceph-storage-installation-guide-rhel/commit/279bbf1a6a4a33f2d88598847a1adf4aa8f6fb7b

We have a downstream version of PGCALC now, and the maximum pg value is sometimes hit when people set up an RGW instance. I've modified the appropriate sections to point to the PG calculator page, and also provided guidance on the max pgs per osd warning there too. 

https://gitlab.cee.redhat.com/jowilkin/red-hat-ceph-storage-object-gateway-rhel/commit/5d8d08e5f3592b361ababde1b9bf847d16525797

https://gitlab.cee.redhat.com/jowilkin/red-hat-ceph-storage-object-gateway-rhel/commit/42701c81e92d06ab879ececc233b6e47d97ca761

Comment 3 Hemanth Kumar 2016-01-14 09:41:50 UTC

Verified the cephpgc link in the Document.. Looks good..

I would like to see the same procedure/pgcalc link in Ubuntu Install Guide also..

Comment 4 John Wilkins 2016-02-09 21:04:13 UTC

See https://gitlab.cee.redhat.com/red-hat-ceph-storage-documentation/doc-Red_Hat_Ceph_Storage_1.3-Installation_Guide_for_Ubuntu/commit/be9f34f239d7149c8755ec7cc2d1678b45070a40

Comment 5 Tanay Ganguly 2016-02-11 06:40:40 UTC

Marking it Verified.

Added the pgc calculator link in Ubuntu installation as well.