Bug 1975536

Summary: [Service Catalog] Remove stale cruft installed by CVO in earlier releases
Product: OpenShift Container Platform Reporter: Jack Ottofaro <jack.ottofaro>
Component: Service CatalogAssignee: Jesus M. Rodriguez <jesusr>
Status: CLOSED WONTFIX QA Contact: Fan Jia <jfan>
Severity: low Docs Contact:
Priority: low    
Version: 4.9CC: aos-bugs, mfojtik, sttts, xxia, yanyang
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 1975533 Environment:
Last Closed: 2022-02-17 01:01:10 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1975533    
Bug Blocks:    
Attachments:
Description Flags
Spreadsheet containing leaked resources none

Description Jack Ottofaro 2021-06-23 21:18:20 UTC 
Created attachment 1793632 [details]
Spreadsheet containing leaked resources

+++ This bug was initially created as a clone of Bug #1975533 +++

This "stale cruft" is created as a result of the following scenario. Release A had manifest M that lead the CVO to reconcile resource R. But then the component maintainers decided they didn't need R any longer, so they dropped manifest M in release B. The new CVO will no longer reconcile R, but clusters updating from A to B will still have resource R in-cluster, as an unmaintained orphan.

Now that https://issues.redhat.com/browse/OTA-222 has been implemented teams can go back through and create deletion manifests for these leaked resources.

The attachment delete-candidates.csv contains a list of leaked resources as compared to a freshly installed 4.9 cluster. Use this list to find your component's resources and use the manifest delete annotation (https://github.com/openshift/cluster-version-operator/pull/438) to remove them.

Note also that in the case of a cluster-scoped resource it may not need to be removed but simply be modified to remove namespace.
Comment 1 Fan Jia 2021-06-24 09:28:48 UTC 
Service catalog is removed from OCP since 4.5. I think this bug could be closed.
Comment 2 Yang Yang 2021-07-06 07:17:49 UTC 
(In reply to Fan Jia from comment #1)
> Service catalog is removed from OCP since 4.5. I think this bug could be
> closed.

CVO now implemented the way to remove the no longer used resources to avoid resources leaking after cluster upgrades. Please refer to https://github.com/openshift/cluster-version-operator/blob/master/docs/dev/object-deletion.md. To make it worked in service catalog, you need to reinsert those removed yaml but add the delete annotaion.
Comment 4 Jesus M. Rodriguez 2022-02-17 01:01:10 UTC 
To clean up the servicecatalog remover jobs you can do the following:

# remove the clusterrolebindings
oc delete clusterrolebinding system:openshift:operator:openshift-service-catalog-apiserver-remover
oc delete clusterrolebinding system:openshift:operator:openshift-service-catalog-controller-manager-remover

# remove the service accounts
oc delete serviceaccount openshift-service-catalog-apiserver-remover -n openshift-service-catalog-removed
oc delete serviceaccount openshift-service-catalog-controller-manager-remover -n openshift-service-catalog-removed

# remove the namespace
oc delete namespace openshift-service-catalog-removed

The CVO feature to cleanup these resources was done after servicecatalog was removed from the product. We are no longer maintaining these operators or jobs. 

I'm closing this bug as wontfix since there is a work around.