Bug 1889681 - Co-existing community Prometheus cause data lost because of the names of CRDs
Summary: Co-existing community Prometheus cause data lost because of the names of CRDs
Keywords:
Status: CLOSED DUPLICATE of bug 1881258
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: OLM
Version: 4.5
Hardware: All
OS: Unspecified
unspecified
high
Target Milestone: ---
: ---
Assignee: Evan Cordell
QA Contact: Jian Zhang
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-10-20 11:17 UTC by Naoto Sano
Modified: 2020-10-20 12:07 UTC (History)
9 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-10-20 12:07:48 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Naoto Sano 2020-10-20 11:17:02 UTC
Description of problem:

The data of Prometheus and Alertmanager get lost if
 - CMO's persistence is configured using prefix as described in doc
 - install community Prometheus Oprator


Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.Install old OCP (4.3, 4.4 or 4.5.x) - it installs CMO at the same time
2.Configure persistence of CMO's Prometheus and Alertmanager using prefix (in the ConfigMap 'cluster-monitoring-config'
3.Install community Prometheus Operator (into another namespace like 'monitoring'
4.Upgrade OCP to newer (4.4, 4.5.y)


Actual results:
Data lost


Expected results:
Data is kept


Additional info:
The root cause is CMO and community PO are both using the same (fully qualified) name for CRDs; *.monitoring.coreos.com

Step #2 creates PVCs with prefix, but step #3 produces another set of PVCs without prefix because CRDs like 'prometheuses.monitoring.coreos.com' are overwritten by community PO.

Step #4 reverts CRDs and PVCs with prefix are used again, but from users view point, recent metrics are not shown in Grafana.

Comment 1 Naoto Sano 2020-10-20 11:48:32 UTC
I think CMO should use different CRD names like *.monitoring.openshift.io from community PO's name like *.monitoring.coreos.com.

Comment 2 Naoto Sano 2020-10-20 11:56:54 UTC
This is found in a IBM's Cloud Pak customer. There is a IBM's support case#.

Comment 3 Simon Pasquier 2020-10-20 12:07:48 UTC
This looks like a duplicate of bug 1881258.

*** This bug has been marked as a duplicate of bug 1881258 ***


Note You need to log in before you can comment on or make changes to this bug.