tested with the PR, issue is fixed
for example, namespace ns1 has namespace labels
# oc get ns ns1 -o jsonpath="{.metadata.labels"}
{"deploy.method":"user","kubernetes.io/metadata.name":"ns1","network.openshift.io/policy-group":"ns1"}

seach with "kube_namespace_labels{namespace="ns1"}", we can see the namespace's labels 
# token=`oc sa get-token prometheus-k8s -n openshift-monitoring`
# oc -n openshift-monitoring exec -c prometheus prometheus-k8s-0 -- curl -k -H "Authorization: Bearer $token" 'https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/query?query=kube_namespace_labels%7Bnamespace%3D%22ns1%22%7D' | jq
{
  "status": "success",
  "data": {
    "resultType": "vector",
    "result": [
      {
        "metric": {
          "__name__": "kube_namespace_labels",
          "container": "kube-rbac-proxy-main",
          "endpoint": "https-main",
          "job": "kube-state-metrics",
          "label_deploy_method": "user",
          "label_kubernetes_io_metadata_name": "ns1",
          "label_network_openshift_io_policy_group": "ns1",
          "namespace": "ns1",
          "service": "kube-state-metrics"
        },
        "value": [
          1634549085.821,
          "1"
        ]
      }
    ]
  }
}

# oc -n openshift-monitoring get deploy kube-state-metrics -oyaml | grep metric-labels-allowlis
        - --metric-labels-allowlist=pods=[*],node=[*],namespaces=[*]

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.8.17 bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:3927