Bug 2130841 - Sidecar logs for ODF 4.12 cluster with CSI_SIDECAR_LOG_LEVEL : 5 is not as much as for ODF 4.11 clusters. [NEEDINFO]
Summary: Sidecar logs for ODF 4.12 cluster with CSI_SIDECAR_LOG_LEVEL : 5 is not as mu...
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Red Hat OpenShift Data Foundation
Classification: Red Hat Storage
Component: ocs-operator
Version: 4.12
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: ---
Assignee: Humble Chirammal
QA Contact: Martin Bukatovic
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2022-09-29 06:52 UTC by Amrita Mahapatra
Modified: 2023-08-09 17:00 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-10-19 03:30:04 UTC
Embargoed:
mparida: needinfo? (ammahapa)


Attachments (Terms of Use)

Description Amrita Mahapatra 2022-09-29 06:52:21 UTC
Description of problem (please be detailed as possible and provide log
snippests): The sidecar logs for ODF 4.12 cluster with CSI_SIDECAR_LOG_LEVEL : 5 is not as much as for ODF 4.11 clusters.
And for ODF 4.12 cluster there are no entry for the sidecar actions like creating a pvc, resizing pvc when CSI_SIDECAR_LOG_LEVEL: "5" is set however the logs triggered for CSI_SIDECAR_LOG_LEVEL: "1".

For ODF 4.12 cluster----

[ammahapa@ammahapa ~]$ oc get cm rook-ceph-operator-config -o yaml
apiVersion: v1
data:
  CSI_ENABLE_METADATA: "true"
  CSI_SIDECAR_LOG_LEVEL: "5"
kind: ConfigMap
metadata:
  creationTimestamp: "2022-09-20T08:00:17Z"
  name: rook-ceph-operator-config
  namespace: openshift-storage
  resourceVersion: "17446064"
  uid: 27620482-cc7e-4117-9a26-41a902faa17c


For ODF 4.11 cluster----

[ammahapa@ammahapa ~]$ oc get cm rook-ceph-operator-config -o yaml
apiVersion: v1
data:
  CSI_ENABLE_CSIADDONS: "true"
  CSI_ENABLE_METADATA: "true"
  CSI_LOG_LEVEL: "5"
  CSI_PLUGIN_TOLERATIONS: |2-

    - key: node.ocs.openshift.io/storage
      operator: Equal
      value: "true"
      effect: NoSchedule
  CSI_PROVISIONER_TOLERATIONS: |2-

    - key: node.ocs.openshift.io/storage
      operator: Equal
      value: "true"
      effect: NoSchedule
kind: ConfigMap
metadata:
  creationTimestamp: "2022-09-22T10:40:32Z"
  name: rook-ceph-operator-config
  namespace: openshift-storage
  resourceVersion: "1912448"
  uid: c404fd07-7749-424a-915d-f8c162f8689a

Version of all relevant components (if applicable):
For ODF 4.12 cluster:
OCP --- 4.12.0-0.nightly-2022-09-19-200722
ODF --- 4.12.0-59

For ODF 4.11 cluster:
OCP --- 4.11.0-0.nightly-2022-09-20-234850
ODF --- 4.11.2-1

Does this issue impact your ability to continue to work with the product
(please explain in detail what is the user impact)? Its not blocking the feature validation.


Is there any workaround available to the best of your knowledge? No


Rate from 1 - 5 the complexity of the scenario you performed that caused this
bug (1 - very simple, 5 - very complex)? 4


Can this issue reproducible? Yes


Can this issue reproduce from the UI? Yes


If this is a regression, please provide more details to justify this: NA


Steps to Reproduce:
1. Update CSI_SIDECAR_LOG_LEVEL: "5" in rook-ceph-operator-config configmap for ODF 4.12 cluster and for ODF 4.11 default value is there, CSI_LOG_LEVEL: "5"

2. Check the sidecar logs for ODF 4.12 cluster and ODF 4.11 cluster and compare
3. Provision a PVC for ODF 4.12 cluster and check sidecar log for csi-provisioner log stream.



Actual results: The sidecar logs for ODF 4.12 cluster with CSI_SIDECAR_LOG_LEVEL : 5 is not as much as for ODF 4.11 clusters with default log level set, CSI_LOG_LEVEL: "5". 
And for ODF 4.12 cluster sidecar logs did not trigger for sidecar actions like creating a pvc, resizing pvc when CSI_SIDECAR_LOG_LEVEL: "5" is set.

Expected results: The sidecar logs for ODF 4.12 cluster with CSI_SIDECAR_LOG_LEVEL : 5 should be as much as for ODF 4.11 clusters with default log level set, CSI_LOG_LEVEL: "5". 
And for ODF 4.12 cluster sidecar logs should trigger for sidecar actions like creating a pvc, resizing pvc when CSI_SIDECAR_LOG_LEVEL: "5" is set.


Additional info:

Comment 6 Humble Chirammal 2022-09-30 06:50:22 UTC
I have accessed the cluster and can confirm the feature is working as expected!

I have created a PVC ( humble-test3) , Cloned, Snapshotted, Resized ..etc on this cluster at log level set to "1" for SIDECAR. I can see the 'required' logs popping up !

Comment 11 Malay Kumar parida 2022-09-30 08:31:30 UTC
The number of lines of logs on 4.11 & 4.12 is becoming so different due to something else here. 
Basically, if you see the logs in 4.11 the message 
"leader election.go:278] successfully renewed lease openshift-storage/openshift-storage-rbd-csi-ceph-com"
gets printed every 5 seconds.

but the same is done in 4.12 every 26 seconds.

Basically, these are logs of leader election & the difference in the interval is because 
in 4.12 has upstream k8s change to leader election method.

So given enough time, there will be a huge difference in the number of lines of logs due to this.

Comment 12 Malay Kumar parida 2022-10-11 15:05:40 UTC
Hi Amrita, Do we still need this?


Note You need to log in before you can comment on or make changes to this bug.