Bug 1872858

Describe the issue:
-----------------------
Due to following bug, even if cleanup policy is set on the storagecluster (more details in Additional Info), sometimes no cleanup job is triggered:

 Bug 1870637 - OCS 4.5 Uninstall: Cleanup policy for long running OCS 4.5 cluster is not working


Hence, user is expected to follow manual process of removing the leftovers.

a) Delete the content of /var/lib/rook 
b) Wipefs the disk for LSO deployments

So, we need to add a Note after Step 6 to re-direct to a Troubleshooting Guide(TG) where we can add a section to manually remove the leftovers( process would be similar to 4.4 doc steps mentioned here:

A) Add a Note after this step in Uninstall guide to go to troubleshooting section
 "Step 6 Ensure that the resulting cleanup jobs are completed and all the nodes are cleaned up." 

B) Add the steps to manually cleanup the leftover resources

Probable content for Troubleshooting guide:
------------------------------------------------
#1. Clean up the storage operator artifacts on each node.


#2. if the cluster has local volume objects, 

List and note the devices to be cleaned up.

$ oc get localvolume -n local-storage $LV -o jsonpath='{ .spec.storageClassDevices[*].devicePaths[*] }'


#3. Wipe the disks for each of the local volumes listed in step 4 so that they can be reused. (Not sure if we can do it before 


>> Things to be discussed:

1. Step "8. Delete the local volume created during the deployment and for each of the local volumes " in 4.4 docs comes before WipeFS step 9. But if we re-direct to TG after Step 6 in OCS 4.5 docs, then we can't ask them to run #1 , go back to main guide to do Step #8 and come back to do step #3 : WipeFS the disk.

2. We have removed following sub-step from Current Step 8 as we used to think we do not need this list. We may have to add it back for safety.

List and note the devices to be cleaned up.

$ oc get localvolume -n local-storage $LV -o jsonpath='{ .spec.storageClassDevices[*].devicePaths[*] }'



Document URL: 

Add a Note to re-direct in Uninstall Guide- https://access.redhat.com/documentation/en-us/red_hat_openshift_container_storage/4.5/html-single/deploying_openshift_container_storage/index?lb_target=preview#assembly_uninstalling-openshift-container-storage_aws-vmware

Add a new section in Troubleshooting guide

https://access.redhat.com/documentation/en-us/red_hat_openshift_container_storage/4.5/html-single/troubleshooting_openshift_container_storage/index?lb_target=preview#troubleshooting-and-deleting-remaining-resources-during-uninstall_rhocs



Additional information:
----------------------------

Setting the cleanup policy is expected to do the following

$ oc label -n openshift-storage storagecluster --all cleanup.ocs.openshift.io=yes-really-destroy-data 

Output:

storagecluster.ocs.openshift.io/ocs-storagecluster labeled

The above command performs the following when the StorageCluster is deleted:
>>Clean up the cluster namespace on the dataDirHostPath.
>>Delete all the ceph monitor directories on the dataDirHostPath. 
>> Clean up the devices on each node.