Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1714184

Summary:	Include Metrics Support for CSI drivers .
Product:	OpenShift Container Platform	Reporter:	Humble Chirammal <hchiramm>
Component:	Storage	Assignee:	Humble Chirammal <hchiramm>
Status:	CLOSED ERRATA	QA Contact:	Chao Yang <chaoyang>
Severity:	high	Docs Contact:
Priority:	high
Version:	4.2.0	CC:	aos-bugs, aos-storage-staff, bbennett, hekumar, mifiedle
Target Milestone:	---
Target Release:	4.2.0
Hardware:	x86_64
OS:	Linux
URL:	https://github.com/openshift/origin/pull/22933
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2019-10-16 06:29:21 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Humble Chirammal 2019-05-27 10:41:27 UTC

Description of problem:

At present, eventhough storage vendor CSI drivers can emit metrics which list down volume stats, the kubernetes/openshift CSI driver is incapable of capturing the metrics and log it.  The metrics support is added in kubernetes recently via below PR.

https://github.com/kubernetes/kubernetes/pull/76188

With this, below volume stats can be captured from a CSI driver:

kubelet_volume_stats_available_bytes{namespace="default",persistentvolumeclaim="csi-pvc"} 0
kubelet_volume_stats_capacity_bytes{namespace="default",persistentvolumeclaim="csi-pvc"} 10
kubelet_volume_stats_inodes{namespace="default",persistentvolumeclaim="csi-pvc"} 0
kubelet_volume_stats_inodes_free{namespace="default",persistentvolumeclaim="csi-pvc"} 0
kubelet_volume_stats_inodes_used{namespace="default",persistentvolumeclaim="csi-pvc"} 0
kubelet_volume_stats_used_bytes{namespace="default",persistentvolumeclaim="csi-pvc"} 0

This is a request to make sure this patch or functionality is available in OCP 4.2 builds.

Version-Release number of selected component (if applicable):

OCP 4.x

How reproducible:

Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:

Master Log:

Node Log (of failed PODs):

PV Dump:

PVC Dump:

StorageClass Dump (if StorageClass used by PV/PVC):

Additional info:

Comment 2 Hemant Kumar 2019-05-28 14:21:11 UTC

Humble - since you wrote the PR in upstream, can you open a backport request against openshift/origin ? I think we should be fine with merging it.

Comment 3 Humble Chirammal 2019-05-28 14:27:06 UTC

(In reply to Hemant Kumar from comment #2)
> Humble - since you wrote the PR in upstream, can you open a backport request
> against openshift/origin ? I think we should be fine with merging it.

Sure Hemanth. Let me file the backport PR. Thanks

Comment 4 Humble Chirammal 2019-05-29 10:12:55 UTC

(In reply to Humble Chirammal from comment #3)
> (In reply to Hemant Kumar from comment #2)
> > Humble - since you wrote the PR in upstream, can you open a backport request
> > against openshift/origin ? I think we should be fine with merging it.
> 
> Sure Hemanth. Let me file the backport PR. Thanks

I have filed below PR for the same. Please review. Thanks!

https://github.com/openshift/origin/pull/22933

Comment 5 Humble Chirammal 2019-05-31 06:22:46 UTC

(In reply to Humble Chirammal from comment #4)
> (In reply to Humble Chirammal from comment #3)
> > (In reply to Hemant Kumar from comment #2)
> > > Humble - since you wrote the PR in upstream, can you open a backport request
> > > against openshift/origin ? I think we should be fine with merging it.
> > 
> > Sure Hemanth. Let me file the backport PR. Thanks
> 
> I have filed below PR for the same. Please review. Thanks!
> 
> https://github.com/openshift/origin/pull/22933

[Status update]

The above PR is reviewed & merged ( Thanks Hemanth) in Openshift Origin, hopefully it will be 
in downstream build soon.

Comment 8 Chao Yang 2019-08-15 09:03:22 UTC

Update the status according to above https://bugzilla.redhat.com/show_bug.cgi?id=1714184#c7

Comment 11 Chao Yang 2019-08-22 08:13:14 UTC

Passed when Using ceph rbd driver.

kubelet_volume_stats_available_bytes{endpoint="https-metrics",instance="10.0.165.131:10250",job="kubelet",namespace="openshift-storage",node="ip-10-0-165-131.us-east-2.compute.internal",persistentvolumeclaim="rbd-pvc",service="kubelet"}  1029275648

kubelet_volume_stats_capacity_bytes{endpoint="https-metrics",instance="10.0.165.131:10250",job="kubelet",namespace="openshift-storage",node="ip-10-0-165-131.us-east-2.compute.internal",persistentvolumeclaim="rbd-pvc",service="kubelet"}  1063256064

kubelet_volume_stats_inodes{endpoint="https-metrics",instance="10.0.165.131:10250",job="kubelet",namespace="openshift-storage",node="ip-10-0-165-131.us-east-2.compute.internal",persistentvolumeclaim="rbd-pvc",service="kubelet"}  524288

kubelet_volume_stats_inodes_free{endpoint="https-metrics",instance="10.0.165.131:10250",job="kubelet",namespace="openshift-storage",node="ip-10-0-165-131.us-east-2.compute.internal",persistentvolumeclaim="rbd-pvc",service="kubelet"}  524285

kubelet_volume_stats_inodes_used{endpoint="https-metrics",instance="10.0.165.131:10250",job="kubelet",namespace="openshift-storage",node="ip-10-0-165-131.us-east-2.compute.internal",persistentvolumeclaim="rbd-pvc",service="kubelet"}  3

kubelet_volume_stats_used_bytes{endpoint="https-metrics",instance="10.0.165.131:10250",job="kubelet",namespace="openshift-storage",node="ip-10-0-165-131.us-east-2.compute.internal",persistentvolumeclaim="rbd-pvc",service="kubelet"} 33980416

Comment 13 errata-xmlrpc 2019-10-16 06:29:21 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:2922