Bug 2051881 - [vSphere CSI driver Operator] RWX volumes counts metrics `vsphere_rwx_volumes_total` not valid
Summary: [vSphere CSI driver Operator] RWX volumes counts metrics `vsphere_rwx_volumes...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Storage
Version: 4.10
Hardware: All
OS: Linux
medium
high
Target Milestone: ---
: 4.11.0
Assignee: Hemant Kumar
QA Contact: Penghao Wang
URL:
Whiteboard:
Depends On:
Blocks: 2052609
TreeView+ depends on / blocked
 
Reported: 2022-02-08 09:44 UTC by Penghao Wang
Modified: 2022-08-10 10:48 UTC (History)
1 user (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 2052609 (view as bug list)
Environment:
Last Closed: 2022-08-10 10:47:58 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift vsphere-problem-detector pull 73 0 None open Bug 2051881: Dont forget to register metrics before emitting 2022-02-08 19:22:59 UTC
Red Hat Product Errata RHSA-2022:5069 0 None None None 2022-08-10 10:48:20 UTC

Description Penghao Wang 2022-02-08 09:44:29 UTC
Description of problem:
[vSphere CSI driver Operator] RWX volumes counts metrics `vsphere_rwx_volumes_total` not valid

Version-Release number of selected component (if applicable):
$ oc get clusterversion
NAME      VERSION                              AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.10.0-0.nightly-2022-02-06-194614   True        False         6h47m   Cluster version is 4.10.0-0.nightly-2022-02-06-194614

How reproducible: 
  Always

Steps to Reproduce:

1. Install a vSphere cluster (vCenter has vSAN file share config).
2. Create 2 RWX pvcs with thin-csi storageclass and 2 daemonsets consume the pvcs. 
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: pvc-rwx
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 1Gi
  storageClassName: thin-csi
  volumeMode: Filesystem
---
kind: DaemonSet
metadata:
  name: myds
  namespace: default
spec:
  selector:
    matchLabels:
      app: myds
  template:
    metadata:
      labels:
        app: myds
    spec:
      containers:
        - name: mycontainer
          image: quay.io/pewang/storagetest:base
          ports:
            - containerPort: 8080
          volumeMounts:
          - mountPath: /mnt/storage
            name: data
      volumes:
        - name: data
          persistentVolumeClaim:
            claimName: pvc-rwx

4. Wait for the daemonsets become ready.
5. Check the RWX volumes counts metrics `vsphere_rwx_volumes_total`:
$ token=`oc sa get-token prometheus-k8s -n openshift-monitoring`
$ oc exec -n openshift-monitoring prometheus-k8s-0 -c prometheus -i -- curl -k -H "Authorization: Bearer $token" https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/query?query=vsphere_rwx_volumes_total
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    63    0    63    0     0   3705      0 --:--:-- --:--:-- --:--:--  3705{"status":"success","data":{"resultType":"vector","result":[]}}

$ oc -n openshift-monitoring exec -c prometheus prometheus-k8s-0 -- curl -k -H "Authorization: Bearer $token" 'https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/label/__name__/values' | jq | grep -i "vsphere"
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 76670    0 76670    0     0  2773k      0 --:--:-- --:--:-- --:--:-- 2773k
    "cloudprovider_vsphere_vcenter_versions",
    "cluster:vsphere_esxi_version_total:sum",
    "cluster:vsphere_node_hw_version_total:sum",
    "cluster:vsphere_vcenter_info:sum",
    "vsphere_cluster_check_errors",
    "vsphere_cluster_check_total",
    "vsphere_datastore_total",
    "vsphere_esxi_version_total",
    "vsphere_node_check_errors",
    "vsphere_node_check_total",
    "vsphere_node_hw_version_total",
    "vsphere_sync_errors",
    "vsphere_vcenter_info",



Actual results:
In step 5: vsphere_rwx_volumes_total could be query and accure value is "2"


Expected results:
In step 5: vsphere_rwx_volumes_total metrics not found

Comment 7 errata-xmlrpc 2022-08-10 10:47:58 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: OpenShift Container Platform 4.11.0 bug fix and security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:5069


Note You need to log in before you can comment on or make changes to this bug.