Description of problem (please be detailed as possible and provide log snippests): The sidecar logs for ODF 4.12 cluster with CSI_SIDECAR_LOG_LEVEL : 5 is not as much as for ODF 4.11 clusters. And for ODF 4.12 cluster there are no entry for the sidecar actions like creating a pvc, resizing pvc when CSI_SIDECAR_LOG_LEVEL: "5" is set however the logs triggered for CSI_SIDECAR_LOG_LEVEL: "1". For ODF 4.12 cluster---- [ammahapa@ammahapa ~]$ oc get cm rook-ceph-operator-config -o yaml apiVersion: v1 data: CSI_ENABLE_METADATA: "true" CSI_SIDECAR_LOG_LEVEL: "5" kind: ConfigMap metadata: creationTimestamp: "2022-09-20T08:00:17Z" name: rook-ceph-operator-config namespace: openshift-storage resourceVersion: "17446064" uid: 27620482-cc7e-4117-9a26-41a902faa17c For ODF 4.11 cluster---- [ammahapa@ammahapa ~]$ oc get cm rook-ceph-operator-config -o yaml apiVersion: v1 data: CSI_ENABLE_CSIADDONS: "true" CSI_ENABLE_METADATA: "true" CSI_LOG_LEVEL: "5" CSI_PLUGIN_TOLERATIONS: |2- - key: node.ocs.openshift.io/storage operator: Equal value: "true" effect: NoSchedule CSI_PROVISIONER_TOLERATIONS: |2- - key: node.ocs.openshift.io/storage operator: Equal value: "true" effect: NoSchedule kind: ConfigMap metadata: creationTimestamp: "2022-09-22T10:40:32Z" name: rook-ceph-operator-config namespace: openshift-storage resourceVersion: "1912448" uid: c404fd07-7749-424a-915d-f8c162f8689a Version of all relevant components (if applicable): For ODF 4.12 cluster: OCP --- 4.12.0-0.nightly-2022-09-19-200722 ODF --- 4.12.0-59 For ODF 4.11 cluster: OCP --- 4.11.0-0.nightly-2022-09-20-234850 ODF --- 4.11.2-1 Does this issue impact your ability to continue to work with the product (please explain in detail what is the user impact)? Its not blocking the feature validation. Is there any workaround available to the best of your knowledge? No Rate from 1 - 5 the complexity of the scenario you performed that caused this bug (1 - very simple, 5 - very complex)? 4 Can this issue reproducible? Yes Can this issue reproduce from the UI? Yes If this is a regression, please provide more details to justify this: NA Steps to Reproduce: 1. Update CSI_SIDECAR_LOG_LEVEL: "5" in rook-ceph-operator-config configmap for ODF 4.12 cluster and for ODF 4.11 default value is there, CSI_LOG_LEVEL: "5" 2. Check the sidecar logs for ODF 4.12 cluster and ODF 4.11 cluster and compare 3. Provision a PVC for ODF 4.12 cluster and check sidecar log for csi-provisioner log stream. Actual results: The sidecar logs for ODF 4.12 cluster with CSI_SIDECAR_LOG_LEVEL : 5 is not as much as for ODF 4.11 clusters with default log level set, CSI_LOG_LEVEL: "5". And for ODF 4.12 cluster sidecar logs did not trigger for sidecar actions like creating a pvc, resizing pvc when CSI_SIDECAR_LOG_LEVEL: "5" is set. Expected results: The sidecar logs for ODF 4.12 cluster with CSI_SIDECAR_LOG_LEVEL : 5 should be as much as for ODF 4.11 clusters with default log level set, CSI_LOG_LEVEL: "5". And for ODF 4.12 cluster sidecar logs should trigger for sidecar actions like creating a pvc, resizing pvc when CSI_SIDECAR_LOG_LEVEL: "5" is set. Additional info:
I have accessed the cluster and can confirm the feature is working as expected! I have created a PVC ( humble-test3) , Cloned, Snapshotted, Resized ..etc on this cluster at log level set to "1" for SIDECAR. I can see the 'required' logs popping up !
The number of lines of logs on 4.11 & 4.12 is becoming so different due to something else here. Basically, if you see the logs in 4.11 the message "leader election.go:278] successfully renewed lease openshift-storage/openshift-storage-rbd-csi-ceph-com" gets printed every 5 seconds. but the same is done in 4.12 every 26 seconds. Basically, these are logs of leader election & the difference in the interval is because in 4.12 has upstream k8s change to leader election method. So given enough time, there will be a huge difference in the number of lines of logs due to this.
Hi Amrita, Do we still need this?