Description of problem:
Kube API Server Operator managed observed config retains the old set of latency config arg val pairs when switching from a latency profile to an empty profile. (Empty latency profile is attainable when nodes.config.openshift.io/cluster object has empty spec, i.e. `spec: {}`)

Version-Release number of selected component (if applicable): 4.11


How reproducible: Always


Steps to Reproduce:
1. `oc edit nodes.config.openshift.io`, change `workerLatencyProfile` to `MediumUpdateAverageReaction`
2. wait for sometime to allow the rollout to happen in `oc get KubeAPIServer -o yaml` under the `observedConfig` section
3. `observedConfig` will contain default-not-ready-toleration-seconds and default-unreachable-toleration-seconds with a value of 60.
4. `oc edit nodes.config.openshift.io`, change profile back to empty, i.e. by setting `spec: {}`

Actual results:
5. seems like `observedConfig` will still retain default-not-ready-toleration-seconds and default-unreachable-toleration-seconds with a value of 60 which is Medium profile, though medium profile was already unset

Expected results:
`observedConfig` shouldn't have default-not-ready-toleration-seconds and default-unreachable-toleration-seconds arg values when empty profile is re-activated on the cluster.

Additional info: This problem was also seen in KCMO (cluster-kube-controller-manager-operator) but due to the latest bump in library-go for KCMO, that problem is resolved now. For KASO also, library-go bump should help resolve the problem.

[1] Enhancement proposal: https://github.com/openshift/enhancements/blob/master/enhancements/worker-latency-profile/worker-latency-profile.md#workerlatencyprofile
[2] Problem description: https://github.com/openshift/library-go/pull/1372/#discussion_r901719238
[3] KCMO PR that already fixed the problem for KCM: https://github.com/openshift/cluster-kube-controller-manager-operator/pull/629