tested with PR, could see PV/PVC labels
# oc -n openshift-monitoring get deployment kube-state-metrics -o yaml | grep metric-labels-allowlist
        - --metric-labels-allowlist=pods=[*],nodes=[*],namespaces=[*],persistentvolumes=[*],persistentvolumeclaims=[*]

# oc -n openshift-monitoring exec -c kube-state-metrics $(oc -n openshift-monitoring get po | grep kube-state-metrics | awk '{print $1}') -- kube-state-metrics --version
version.Version{GitCommit:"94716629", BuildDate:"2021-05-05T12:44:19Z", Release:"v2.0.0", GoVersion:"go1.16.1", Compiler:"gc", Platform:"linux/amd64"}

# oc get pv pvc-de3ffcfc-daa0-4cdf-9c45-257844cd9979 -o jsonpath="{.metadata.labels"}
{"topology.kubernetes.io/region":"us-central1","topology.kubernetes.io/zone":"us-central1-a"}

search with kube_persistentvolume_labels, could see the PV labels
# token=`oc sa get-token prometheus-k8s -n openshift-monitoring`
# oc -n openshift-monitoring exec -c prometheus prometheus-k8s-0 -- curl -k -H "Authorization: Bearer $token" 'https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/query?query=kube_persistentvolume_labels' | jq
...
{
  "status": "success",
  "data": {
    "resultType": "vector",
    "result": [
      {
        "metric": {
          "__name__": "kube_persistentvolume_labels",
          "container": "kube-rbac-proxy-main",
          "endpoint": "https-main",
          "job": "kube-state-metrics",
          "label_topology_kubernetes_io_region": "us-central1",
          "label_topology_kubernetes_io_zone": "us-central1-a",
          "namespace": "openshift-monitoring",
          "persistentvolume": "pvc-de3ffcfc-daa0-4cdf-9c45-257844cd9979",
          "service": "kube-state-metrics"
        },
        "value": [
          1636357638.705,
          "1"
        ]
      },
...

pvc labels
# oc -n openshift-user-workload-monitoring get pvc thanosruler-thanos-ruler-user-workload-0 -o jsonpath="{.metadata.labels"}
{"app":"thanos-ruler","app.kubernetes.io/instance":"user-workload","app.kubernetes.io/managed-by":"prometheus-operator","app.kubernetes.io/name":"thanos-ruler","thanos-ruler":"user-workload"}

search with kube_persistentvolumeclaim_labels, could see the PVC labels
# oc -n openshift-monitoring exec -c prometheus prometheus-k8s-0 -- curl -k -H "Authorization: Bearer $token" 'https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/query?query=kube_persistentvolumeclaim_labels' | jq
...
      {
        "metric": {
          "__name__": "kube_persistentvolumeclaim_labels",
          "container": "kube-rbac-proxy-main",
          "endpoint": "https-main",
          "job": "kube-state-metrics",
          "label_app": "thanos-ruler",
          "label_app_kubernetes_io_instance": "user-workload",
          "label_app_kubernetes_io_managed_by": "prometheus-operator",
          "label_app_kubernetes_io_name": "thanos-ruler",
          "label_thanos_ruler": "user-workload",
          "namespace": "openshift-user-workload-monitoring",
          "persistentvolumeclaim": "thanosruler-thanos-ruler-user-workload-0",
          "service": "kube-state-metrics"
        },
        "value": [
          1636357881.414,
          "1"
        ]
      },
...

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.8.20 bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:4574