Bug 2261950

1. Proposed title of this feature request

Add supportability to disable in-transit encryption. 

2. Who is the customer behind the request?

This RFE was raised during the ODF 4.14 Hackathon.

3. What is the nature and description of the request? 

Please, we'd like to request supportability to disable in-transit encryption. As described in [1], this is not currently supported. It requires the recreation of the storage cluster. 

4. Why does the customer need this? (List the business requirements here)

It's possible that after enabling in-transit encryption, a customer might be unable to use it because of the performance impact observed for 128/4096 block sizes. Refer to [2] for further information. Or, a customer might need to disable it for other business reasons. 

There should be a way to turn off this feature without reinstalling the cluster. 

5. How would the customer like to achieve this? (List the functional requirements here)

I tested these steps in my lab, and they worked for me:

- Patched the storage cluster and disabled encryption:

		$ oc patch storagecluster ocs-external-storagecluster -n openshift-storage --type json --patch  '[{ "op": "replace", "path": "/spec/network/connections/encryption", "value": {"enabled": false} }]'
		storagecluster.ocs.openshift.io/ocs-external-storagecluster patched
		
- Make sure the storagecluster gets ready:

		$ oc get storagecluster -n openshift-storage
		NAME                      	AGE   PHASE   EXTERNAL   CREATED AT         	VERSION
		ocs-external-storagecluster   18h   Ready   true   2024-01-08T16:33:42Z   4.14.2

- Make sure the `rook-ceph-csi-config` CM sets the mon ports back to 6789:

		oc get cm rook-ceph-csi-config -o json -n openshift-storage | jq -r '.data'
		{
		  "csi-cluster-config-json": "[{\"clusterID\":\"openshift-storage\",\"monitors\":[\"10.0.93.166:6789\",\"10.0.93.53:6789\",\"10.0.93.164:6789\",\"10.0.91.79:6789\",\"10.0.93.41:6789\"],\"namespace\":\"openshift-storage\"}]"
		}

- Remount Ceph RBD / FS PVs.  

  I manually cordoned the nodes to trigger the remount using port `6789`. This is something that also needs better documentation. Is this the only way to ensure the file systems / RBD images get remounted correctly, or is there any better procedure?
 
		$ oc get pods -o wide
		NAME                      	READY   STATUS RESTARTS   AGE  IP   NODE    NOMINATED NODE   READINESS GATES
		simple-app-6797bc74b4-fppkk   1/1 Running   0     3m50s   10.128.0.220   master-0.nravinaocp.lab.upshift.rdu2.redhat.com   <none>  <none>
		
		$ oc adm cordon  master-0.nravinaocp.lab.upshift.rdu2.redhat.com
		node/master-0.nravinaocp.lab.upshift.rdu2.redhat.com cordoned
		
		$ oc delete pod simple-app-6797bc74b4-fppkk
		pod "simple-app-6797bc74b4-fppkk" deleted
		
		$ oc adm uncordon master-0.nravinaocp.lab.upshift.rdu2.redhat.com
		node/master-0.nravinaocp.lab.upshift.rdu2.redhat.com uncordoned
		
		$ oc get pods -o wide
		NAME                      	READY   STATUS RESTARTS   AGE   IP         NODE       NOMINATED NODE   READINESS GATES
		simple-app-6797bc74b4-dc9gc   1/1 	   Running   0     46s   10.130.1.212   master-2.nravinaocp.lab.upshift.rdu2.redhat.com   <none>       <none>
		
		$ oc exec simple-app-6797bc74b4-dc9gc -- mount | grep mnt
		10.0.93.166:6789,10.0.93.53:6789,10.0.93.164:6789,10.0.91.79:6789,10.0.93.41:6789:/volumes/csi/csi-vol-05e577bd-11be-4e5c-82b0-c3d2398b5e11/36bfbd9a-2e67-415b-a2eb-7be623b9e0e7 on /mnt type ceph (rw,relatime,seclabel,name=csi-cephfs-node,secret=<hidden>,fsid=00000000-0000-0000-0000-000000000000,acl,mds_namespace=my-filesystem)


6. For each functional requirement listed, specify how Red Hat and the customer can test to confirm the requirement is successfully implemented.

Following a documented procedure to disable in-transit encryption that doesn't require ODF reinstallation clearly states how to cleanly remount the file systems to switch between ports 3300 and 6789. 

7. Is there already an existing RFE upstream or in Red Hat Bugzilla?

No

8. Does the customer have any specific timeline dependencies and which release would they like to target (i.e. RHEL5, RHEL6)?

No.

9. Is the sales team involved in this request and do they have any additional input?

No

10. List any affected packages or components.

OCS operator

11. Would the customer be able to assist in testing this functionality if implemented?

Yes

[1] https://hackmd.io/@Yh4a4hAATcW2BNYBJVSx4w/BkukxFUsj#Case-4-Disabling-in-transit-encryption
[2] https://bugzilla.redhat.com/show_bug.cgi?id=2215628