Bug 2111952

Summary:	PagerDuty alerting is not working
Product:	[Red Hat Storage] Red Hat OpenShift Data Foundation	Reporter:	Filip Balák <fbalak>
Component:	odf-managed-service	Assignee:	Ohad <omitrani>
Status:	CLOSED CURRENTRELEASE	QA Contact:	Filip Balák <fbalak>
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	4.11	CC:	aeyal, dbindra, lgangava, ocs-bugs, odf-bz-bot
Target Milestone:	---	Keywords:	AutomationBlocker, Regression
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	2.0.6	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2022-11-02 05:20:24 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Filip Balák 2022-07-28 13:57:55 UTC

Description of problem:
PagerDuty alerts are not triggered and internal prometheus alerting site is not available when trying to access with: oc port-forward svc/prometheus-operated 9090 -n openshift-storage

Version-Release number of selected component (if applicable):
ocs-osd-deployer.v2.0.4

How reproducible:
2/2

Steps to Reproduce:
1. Set PagerDuty integration into ocs-provider-qe-deadmanssnitch
2. Shut down one osd
3. Wait for alerts
4. oc port-forward svc/prometheus-operated 9090 -n openshift-storage
5. visit http://localhost:9090/alerts

Actual results:
Alerting is not working and list of alerts in http://localhost:9090/alerts is not accessible.

Expected results:
Alerting is working.

Additional info:

Comment 1 Filip Balák 2022-07-28 14:41:57 UTC

As discussed in https://chat.google.com/room/AAAASHA9vWs/5OakFiBxlH0:
This was caused by an issue in ocm (missing ocs-provider-qe-prom-remote-write secret): https://app.slack.com/client/T027F3GAJ/C01L46M0FQC/thread/C01L46M0FQC-1658934690.637499
Freshly deployed clusters are working.

Comment 4 Leela Venkaiah Gangavarapu 2022-08-29 06:58:46 UTC

@fbalak,

- based on https://bugzilla.redhat.com/show_bug.cgi?id=2111952#c1 can this bug be closed?

thanks,
leela.

Comment 5 Dhruv Bindra 2022-09-05 11:18:31 UTC

Moving to ON_QA as it was fixed and works on fresh deployment

Comment 6 Filip Balák 2022-09-19 10:11:21 UTC

Original problem is resolved. --> VERIFIED

Tested with:
ocs-osd-deployer.v2.0.6