Created attachment 1746906 [details]
manila-operator-log

Description of problem:
Upgrade failed from OCP4.6 to 4.7.0-0.nightly-2021-01-12-203716 due to cluster operator storage degraded. 
$ occluster 
OpenStack
NAME      VERSION                             AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.6.0-0.nightly-2021-01-12-112514   True        True          85m     Unable to apply 4.7.0-0.nightly-2021-01-12-203716: the cluster operator storage is degraded

When checking the co status, we see manila sync error 
status:
  conditions:
  - lastTransitionTime: "2021-01-13T04:02:32Z"
    message: 'ManilaCSIDriverOperatorCRDegraded: ManilaControllerDegraded: StorageClass.storage.k8s.io "csi-manila-ceph" is invalid: parameters: Forbidden: updates to parameters are forbidden.'
    reason: ManilaCSIDriverOperatorCR_ManilaController_SyncError

From the manila operator log(with Debug devel)
I0113 04:27:32.413504       1 manila.go:88] Manila sync started
I0113 04:27:32.414033       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-cluster-csi-drivers", Name:"manila-csi-driver-operator", UID:"eba82efa-e37c-4707-9813-e3e7d10d44ac", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'StorageClassUpdateFailed' Failed to update StorageClass.storage.k8s.io/csi-manila-ceph: StorageClass.storage.k8s.io "csi-manila-ceph" is invalid: parameters: Forbidden: updates to parameters are forbidden.
I0113 04:27:32.549082       1 manila.go:133] Syncing storage class for shareType type ceph
I0113 04:27:32.606454       1 request.go:581] Throttling request took 57.290385ms, request: GET:https://172.30.0.1:443/apis/storage.k8s.io/v1/storageclasses/csi-manila-ceph
I0113 04:27:32.606518       1 request.go:581] Throttling request took 393.919879ms, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-manila-csi-driver/serviceaccounts/manila-csi-driver-controller-sa
I0113 04:27:32.618247       1 storage.go:51] StorageClass "csi-manila-ceph" changes: {"parameters":{"appendShareMetadata":"{\"openshiftClusterID\": \"\"}"}}
I0113 04:27:32.806450       1 request.go:581] Throttling request took 440.304705ms, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-manila-csi-driver/secrets/csi-manila-secrets
I0113 04:27:32.806492       1 request.go:581] Throttling request took 188.13889ms, request: PUT:https://172.30.0.1:443/apis/storage.k8s.io/v1/storageclasses/csi-manila-ceph
I0113 04:27:32.814273       1 manila.go:124] Manila sync finished
I0113 04:27:32.814465       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-cluster-csi-drivers", Name:"manila-csi-driver-operator", UID:"eba82efa-e37c-4707-9813-e3e7d10d44ac", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'StorageClassUpdateFailed' Failed to update StorageClass.storage.k8s.io/csi-manila-ceph: StorageClass.storage.k8s.io "csi-manila-ceph" is invalid: parameters: Forbidden: updates to parameters are forbidden.
E0113 04:27:32.814504       1 base_controller.go:250] "ManilaController" controller failed to sync "key", err: StorageClass.storage.k8s.io "csi-manila-ceph" is invalid: parameters: Forbidden: updates to parameters are forbidden.
I0113 04:27:33.006439       1 request.go:581] Throttling request took 592.800242ms, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-config/configmaps/cloud-provider-config


Version-Release number of selected component (if applicable):
4.7.0-0.nightly-2021-01-12-203716

How reproducible:
2/2

Steps to Reproduce:
Upgrade OCP4.6 to OCP4.7 on OSP 

Actual results:


Expected results:

Master Log:

Node Log (of failed PODs):

PV Dump:

PVC Dump:

StorageClass Dump (if StorageClass used by PV/PVC):

Additional info: