Bug 1864283

First: This may not be the right routing, please confirm/reroute.

Description of problem:

I recently counted the number of objects in a freshly-installed v4 OSD cluster (CCS [1] and non-CCS [2]). Before any customization, customer workloads, etc., it was in the tens of thousands. The majority of these objects were packagemanifests, which seem to be created at a rate of one per operator per namespace. The vast majority of these objects will not be used in a real cluster. The concern is how this scales and contributes to the load on etcd. Today the number of operators is in the 200s-300s (depending on configuration), but this will only grow. And the number of namespaces just out of the box is in the 70s *before* the customer gets in there. I'm told some customer clusters use thousands of namespaces. As an example, we break the million-object mark with 3000 namespaces and 350-ish operators.

Version-Release number of selected component (if applicable): Unsure. Anywhere we're creating packagemanifests, I imagine.


How reproducible: 100%


Steps to Reproduce:
1. Create a cluster
2. Count packagemanifests

for ns in `oc get namespaces`; do oc get packagemanifests -n $ns | wc l; done

kind of thing.

Actual results: See linked spreadsheets


Expected results: I dunno. Maybe this is acceptable as is. But maybe we find a way to create these objects only as needed.


Additional info:

[1] https://docs.google.com/spreadsheets/d/1B8TtoeCFKzpvwF6jEIwS5t1xPaKG37GzxOn4UQ-5xro/edit#gid=1770916906
[2] https://docs.google.com/spreadsheets/d/18v2oRPXN2dLgfi2aDfsyFVRE_3HTeOdK8JPrnR2dnXg/edit#gid=1770916906