Bug 2213680

Summary: RHOSP deployment : The CephFSID environment value does not match the stack configuration value
Product: Red Hat OpenStack Reporter: XinhuaLi <xili>
Component: openstack-tripleoAssignee: OSP Team <rhos-maint>
Status: CLOSED NOTABUG QA Contact: Joe H. Rahme <jhakimra>
Severity: high Docs Contact:
Priority: unspecified    
Version: 16.2 (Train)CC: eharney, fpantano, johfulto, mburns, tkajinam
Target Milestone: ---Keywords: Triaged
Target Release: ---Flags: fpantano: needinfo? (xili)
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-07-05 13:22:51 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description XinhuaLi 2023-06-09 01:52:24 UTC
Description of problem:

RHOSP 16.2.5 + External Ceph 5.3 
During RHOSP 16.2.5 deployment, we tried to integrate with external ceph and facing " The CephFSID environment value does not match the stack  configuration value error."


Version-Release number of selected component (if applicable):
RHOSP 16.2.5 
Ceph 5.3


How reproducible:
1. Deploy RHOSP 16.2.5
2. Deployment external Ceph 5.3
3. Integrate RHOSP 16.2.5 with external Ceph 5.3
4. Update yaml (such as manila-cephfsnative-config.yaml) and deploy
or add new "cephfs" and deploy  

" The CephFSID environment value does not match the stack  configuration value error." shown.

It seems that yaml cannot be modified after integrating RHOSP 16.2.5 with external Ceph 5.3. 

Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:
We check the code related to " The CephFSID environment value does not match the stack  configuration value error."
~~~
  env_ceph_fsid = environment.get('parameter_defaults',
                                    {}).get('CephClusterFSID', False)
    stack_ceph_fsid = stack.environment().get('parameter_defaults',
                                              {}).get('CephClusterFSID', False)

    if bool(env_ceph_fsid) and env_ceph_fsid != stack_ceph_fsid:
        raise exceptions.InvalidConfiguration('The CephFSID environment value '
                                              ' ({}) does not match the stack '
                                              ' configuration value ({}).'
                                              ' Ensure the CephClusterFSID '
                                              ' param is properly configured '
                                              ' in the storage environment '
                                              ' files.'
                                              .format(env_ceph_fsid,
                                                      stack_ceph_fsid))
~~~

I am thinking that if CephFSID will be generated each time with new one during each integrating. 
Please help to check

Regards
Sam

Comment 2 John Fulton 2023-07-05 12:53:07 UTC
(In reply to XinhuaLi from comment #0)
> How reproducible:
> 1. Deploy RHOSP 16.2.5
> 2. Deployment external Ceph 5.3
> 3. Integrate RHOSP 16.2.5 with external Ceph 5.3

This is not a supported pattern. It should be.

1. Deploy Ceph
2. Deploy OpenStack with Ceph by following the doc [A]

It is not a supported feature to add external Ceph to an overcloud after it has been deployed the first time via a stack update. We do not test that pattern.


[A] https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/16.2/html-single/integrating_an_overcloud_with_an_existing_red_hat_ceph_storage_cluster/index

Comment 4 John Fulton 2023-07-05 13:22:51 UTC
This is not a supported pattern as per #2. We have a workaround if this is necessary though it requires SE. If you need to pursue the path, then we can discuss further in the SE.