Description of problem: At present, eventhough storage vendor CSI drivers can emit metrics which list down volume stats, the kubernetes/openshift CSI driver is incapable of capturing the metrics and log it. The metrics support is added in kubernetes recently via below PR. https://github.com/kubernetes/kubernetes/pull/76188 With this, below volume stats can be captured from a CSI driver: kubelet_volume_stats_available_bytes{namespace="default",persistentvolumeclaim="csi-pvc"} 0 kubelet_volume_stats_capacity_bytes{namespace="default",persistentvolumeclaim="csi-pvc"} 10 kubelet_volume_stats_inodes{namespace="default",persistentvolumeclaim="csi-pvc"} 0 kubelet_volume_stats_inodes_free{namespace="default",persistentvolumeclaim="csi-pvc"} 0 kubelet_volume_stats_inodes_used{namespace="default",persistentvolumeclaim="csi-pvc"} 0 kubelet_volume_stats_used_bytes{namespace="default",persistentvolumeclaim="csi-pvc"} 0 This is a request to make sure this patch or functionality is available in OCP 4.2 builds. Version-Release number of selected component (if applicable): OCP 4.x How reproducible: Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Master Log: Node Log (of failed PODs): PV Dump: PVC Dump: StorageClass Dump (if StorageClass used by PV/PVC): Additional info:
Humble - since you wrote the PR in upstream, can you open a backport request against openshift/origin ? I think we should be fine with merging it.
(In reply to Hemant Kumar from comment #2) > Humble - since you wrote the PR in upstream, can you open a backport request > against openshift/origin ? I think we should be fine with merging it. Sure Hemanth. Let me file the backport PR. Thanks
(In reply to Humble Chirammal from comment #3) > (In reply to Hemant Kumar from comment #2) > > Humble - since you wrote the PR in upstream, can you open a backport request > > against openshift/origin ? I think we should be fine with merging it. > > Sure Hemanth. Let me file the backport PR. Thanks I have filed below PR for the same. Please review. Thanks! https://github.com/openshift/origin/pull/22933
(In reply to Humble Chirammal from comment #4) > (In reply to Humble Chirammal from comment #3) > > (In reply to Hemant Kumar from comment #2) > > > Humble - since you wrote the PR in upstream, can you open a backport request > > > against openshift/origin ? I think we should be fine with merging it. > > > > Sure Hemanth. Let me file the backport PR. Thanks > > I have filed below PR for the same. Please review. Thanks! > > https://github.com/openshift/origin/pull/22933 [Status update] The above PR is reviewed & merged ( Thanks Hemanth) in Openshift Origin, hopefully it will be in downstream build soon.
Update the status according to above https://bugzilla.redhat.com/show_bug.cgi?id=1714184#c7
Passed when Using ceph rbd driver. kubelet_volume_stats_available_bytes{endpoint="https-metrics",instance="10.0.165.131:10250",job="kubelet",namespace="openshift-storage",node="ip-10-0-165-131.us-east-2.compute.internal",persistentvolumeclaim="rbd-pvc",service="kubelet"} 1029275648 kubelet_volume_stats_capacity_bytes{endpoint="https-metrics",instance="10.0.165.131:10250",job="kubelet",namespace="openshift-storage",node="ip-10-0-165-131.us-east-2.compute.internal",persistentvolumeclaim="rbd-pvc",service="kubelet"} 1063256064 kubelet_volume_stats_inodes{endpoint="https-metrics",instance="10.0.165.131:10250",job="kubelet",namespace="openshift-storage",node="ip-10-0-165-131.us-east-2.compute.internal",persistentvolumeclaim="rbd-pvc",service="kubelet"} 524288 kubelet_volume_stats_inodes_free{endpoint="https-metrics",instance="10.0.165.131:10250",job="kubelet",namespace="openshift-storage",node="ip-10-0-165-131.us-east-2.compute.internal",persistentvolumeclaim="rbd-pvc",service="kubelet"} 524285 kubelet_volume_stats_inodes_used{endpoint="https-metrics",instance="10.0.165.131:10250",job="kubelet",namespace="openshift-storage",node="ip-10-0-165-131.us-east-2.compute.internal",persistentvolumeclaim="rbd-pvc",service="kubelet"} 3 kubelet_volume_stats_used_bytes{endpoint="https-metrics",instance="10.0.165.131:10250",job="kubelet",namespace="openshift-storage",node="ip-10-0-165-131.us-east-2.compute.internal",persistentvolumeclaim="rbd-pvc",service="kubelet"} 33980416
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:2922