Bug 2230358

Summary: No sendgrid emails are sent when PVC is utilised even more than 75% or 80%
Product: [Red Hat Storage] Red Hat OpenShift Data Foundation Reporter: suchita <sgatfane>
Component: odf-managed-serviceAssignee: Ohad <omitrani>
Status: NEW --- QA Contact: Neha Berry <nberry>
Severity: high Docs Contact:
Priority: unspecified    
Version: 4.10CC: odf-bz-bot
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description suchita 2023-08-09 11:19:14 UTC
Description of problem:
Ususally PVC utilizaion alert get raised when it is utilised more than 75% abd 80%. This sendgrid alerts are missing now 

Version-Release number of selected component (if applicable):


How reproducible:
3/3

Steps to Reproduce:
1.Deploy RHODFMS cluster with ROSA4.11.45 (and upgraded to ROSA 4.12.16)
2. Create PVC and Run IO for utilisation of more than 75% or 80% 

3.

Actual results:
No alert were raised by sendgrid / no mail for alert

Expected results:
"Persistent Volume Usage is Nearly Full " SendGrid alert should be raised

Additional info:

$ oc get secret | grep smtp
ocs-consumer-smtp                                    Opaque                                5      2d4h

$ oc get csv
NAME                                      DISPLAY                       VERSION           REPLACES                                  PHASE
mcg-operator.v4.10.14                     NooBaa Operator               4.10.14           mcg-operator.v4.10.13                     Succeeded
observability-operator.v0.0.25            Observability Operator        0.0.25            observability-operator.v0.0.25-rc         Succeeded
ocs-operator.v4.10.9                      OpenShift Container Storage   4.10.9            ocs-operator.v4.10.8                      Succeeded
ocs-osd-deployer.v2.0.13                  OCS OSD Deployer              2.0.13            ocs-osd-deployer.v2.0.12                  Succeeded
odf-csi-addons-operator.v4.10.9           CSI Addons                    4.10.9            odf-csi-addons-operator.v4.10.8           Succeeded
odf-operator.v4.10.9                      OpenShift Data Foundation     4.10.9            odf-operator.v4.10.8                      Succeeded
ose-prometheus-operator.4.10.0            Prometheus Operator           4.10.0            ose-prometheus-operator.4.8.0             Succeeded
route-monitor-operator.v0.1.547-2ea77e3   Route Monitor Operator        0.1.547-2ea77e3   route-monitor-operator.v0.1.545-0a44ea3   Succeeded

$ oc get pods
NAME                                               READY   STATUS    RESTARTS       AGE
addon-ocs-consumer-catalog-n5dgm                   1/1     Running   1              4h15m
alertmanager-managed-ocs-alertmanager-0            2/2     Running   2              117m
csi-addons-controller-manager-5b854d9667-98gnn     2/2     Running   3 (17m ago)    4h19m
csi-cephfsplugin-bhqw9                             2/2     Running   2              125m
csi-cephfsplugin-f729b                             2/2     Running   1 (23m ago)    112m
csi-cephfsplugin-provisioner-8bdfdf6cb-h72hn       5/5     Running   5              117m
csi-cephfsplugin-provisioner-8bdfdf6cb-nmq4z       5/5     Running   14 (17m ago)   4h19m
csi-cephfsplugin-svv9z                             2/2     Running   2              4h26m
csi-rbdplugin-2fx6j                                3/3     Running   3              4h26m
csi-rbdplugin-m6bnb                                3/3     Running   3              125m
csi-rbdplugin-provisioner-645c8bb55c-6qlff         6/6     Running   6              117m
csi-rbdplugin-provisioner-645c8bb55c-lmrb5         6/6     Running   10 (17m ago)   4h19m
csi-rbdplugin-z7hxl                                3/3     Running   0              112m
ocs-metrics-exporter-7448fc69b6-t99nx              1/1     Running   1              4h19m
ocs-operator-6689f778bd-kkgtf                      1/1     Running   6 (14m ago)    130m
ocs-osd-aws-data-gather-5ccf8445bb-5kj4k           1/1     Running   1              117m
ocs-osd-controller-manager-75cbdbcc4d-p2srz        3/3     Running   7 (14m ago)    4h19m
odf-console-6d6b484c46-mgl6g                       1/1     Running   1              117m
odf-operator-controller-manager-79789b8f5f-p86hr   2/2     Running   3 (17m ago)    4h19m
prometheus-managed-ocs-prometheus-0                3/3     Running   4              4h15m
prometheus-operator-c74f5f6c9-fl7sj                1/1     Running   1              117m
redhat-operators-8ttj5                             1/1     Running   1              117m
rook-ceph-operator-77775c8f64-2l6cm                1/1     Running   1              117m
rook-ceph-tools-5547cb5857-sqbcp                   1/1     Running   0              7m31s

$ oc get clusterversion
NAME      VERSION   AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.12.26   True        False         44h     Cluster version is 4.12.26