Bug 2179330

Summary: [cee/sd][Cephadm][RGW] RGW parameters set under ceph_conf_overrides does not get applied to the cluster after upgrading from RHCS 4.3 to 5.3
Product: [Red Hat Storage] Red Hat Ceph Storage Reporter: Milind <milverma>
Component: CephadmAssignee: Adam King <adking>
Status: CLOSED WONTFIX QA Contact: Aditya Ramteke <aramteke>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 5.3CC: adking, bkunal, ceph-eng-bugs, cephqe-warriors, gabrioux, gmeno, jmulligan, ksachdev, mbenjamin, msaini, racpatel, saraut, sostapov, tonay, vereddy
Target Milestone: ---Flags: adking: needinfo? (milverma)
Target Release: 5.3z5   
Hardware: Unspecified   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-08-16 15:12:37 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Milind 2023-03-17 11:53:01 UTC 
Description of problem:

Deployed a containerized RHCS 4.3 cluster with rgw parameters in all.yml under ceph_conf_overrides.
Since upgrading to RHCS 5.3, none of the rgw parameters listed in ceph_conf_overrides have been applied to the RGW configuration.

ceph_conf_overrides in all.yml
------------------------------

ceph_conf_overrides:
  global:
    mon_allow_pool_delete: true
  client.rgw.node3.rgw0:
    rgw_dns_name: node3-rgw0
    rgw_override_bucket_index_max_shards: 16
    rgw_bucket_default_quota_max_objects: 1638400
  client.rgw.node3.rgw1:
    rgw_dns_name: node3-rgw1
    rgw_override_bucket_index_max_shards: 16
    rgw_bucket_default_quota_max_objects: 1638400
  client.rgw.node2.rgw0:
    rgw_dns_name: node3-rgw0
    rgw_override_bucket_index_max_shards: 16
    rgw_bucket_default_quota_max_objects: 1638400
  client.rgw.node2.rgw1:
    rgw_dns_name: node2-rgw1
    rgw_override_bucket_index_max_shards: 16
    rgw_bucket_default_quota_max_objects: 1638400

I have attached the following outputs to the BZ:
ceph_config_dump.txt (After 5.3 upgrade)
rgw_configuration_applied.txt (After 5.3 upgrade)

Version-Release number of selected component (if applicable):
RHCS 5.3

How reproducible:
Always

Steps to Reproduce:
1. Create a containerized RHCS 4.3 cluster with config overrides for RGW
2. Upgrade the cluster to RHCS 5.3
3. Check if the config_overrides parameters actually applied

Actual results:
None of the config overrides parameters are applied t the actual configuration

Expected results:
The config_overrides parameter should apply
Comment 21 Scott Ostapovicz 2023-06-14 16:03:42 UTC 
Missed the 5.3 z4 deadline.  Moving from z4 to z5.