Bug 1840820 - Additional PVCs created in openshift-monitoring namespace after upgrading from 4.3 to 4.4.4
Summary: Additional PVCs created in openshift-monitoring namespace after upgrading fro...
Keywords:
Status: CLOSED DUPLICATE of bug 1833427
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Monitoring
Version: 4.4
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
: ---
Assignee: Pawel Krupa
QA Contact: Junqi Zhao
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-05-27 16:36 UTC by Aditya Deshpande
Modified: 2023-10-06 20:17 UTC (History)
9 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-05-28 09:52:26 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Knowledge Base (Solution) 5112351 0 None None None 2020-05-27 17:35:43 UTC

Description Aditya Deshpande 2020-05-27 16:36:23 UTC
Description of problem:

After upgrading from OCP 4.3.21 to 4.4.4, there are additional PVCs getting created for prometheus and alertmanager though there were PVCs present before upgrade.

Before upgrade: 
~~~
# oc get pvc
NAME                                 STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
prom-prometheus-k8s-0                Bound    pvc-af243bba-8281-4617-965b-a9cf43adcf41   1Gi        RWO            gp2            4h59m
prom-prometheus-k8s-1                Bound    pvc-679cf5a9-83d7-4927-b15f-fbfecf824720   1Gi        RWO            gp2            4h59m

# oc get cm cluster-monitoring-config -o yaml
apiVersion: v1
data:
  config.yaml: |
    prometheusK8s:
      volumeClaimTemplate:
        metadata:
          name: prom          <==== PVC-prefix-name
        spec:
          storageClassName: gp2
          resources:
            requests:
              storage: 1Gi
kind: ConfigMap
~~~

Here, it was already PVCs were present with PVC-prefix-name: "prom"

But after  upgrade to 4.4.4, it is observed that additional PVC with default PVC-prefix-name got created and new PVCs are getting used by the respective pods.

After upgrade:
~~~
# oc get pvc
NAME                                 STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
prom-prometheus-k8s-0                Bound    pvc-af243bba-8281-4617-965b-a9cf43adcf41   1Gi        RWO            gp2            4h59m
prom-prometheus-k8s-1                Bound    pvc-679cf5a9-83d7-4927-b15f-fbfecf824720   1Gi        RWO            gp2            4h59m
prometheus-k8s-db-prometheus-k8s-0   Bound    pvc-ef25921b-1a59-4145-ad9f-14904f0feb97   1Gi        RWO            gp2            109m
prometheus-k8s-db-prometheus-k8s-1   Bound    pvc-6e8ea9fd-eb97-4a3a-a1ef-32319b615fa1   1Gi        RWO            gp2            109m

# oc describe pod prometheus-k8s-0
Volumes:
  prometheus-k8s-db:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  prometheus-k8s-db-prometheus-k8s-0
    ReadOnly:   false
~~~

Version-Release number of selected component (if applicable):
# oc get clusterversion
NAME      VERSION   AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.4.4     True        False         69m     Cluster version is 4.4.4

How reproducible:
Always

Steps to Reproduce:
1. deploy OCP v4.3.21 with customized PVC-prefix-name for prometheus and alertmanager pods with persistent storage.
2. upgrade to 4.4.4 using stable channel
3. check the pvc inside openshift-monitoring namespace after upgrade

Actual results:
Additional PVCs with default pvc-prefix-name are getting created though there are already present PVCs which are created before upgrade. Existing PVCs are not being used by the new prometheus and alertmanager pods.

Expected results:
Prometheus and alertmanager pods should use the existing PVCs which are created before upgrade.
Also, there should not be any new creation of PVCs with default pvc-prefix-name as there are PVCs present already.

Additional info:
The cluster-monitoring-config configmpa inside the openshift-monitoring namespace is not changed during upgrade process.

Comment 2 Pawel Krupa 2020-05-28 09:52:26 UTC

*** This bug has been marked as a duplicate of bug 1833427 ***


Note You need to log in before you can comment on or make changes to this bug.