Bug 1888614

Summary: [External] Unreachable monitoring-endpoint used during deployment causes ocs-operator to crash
Product: [Red Hat Storage] Red Hat OpenShift Container Storage Reporter: Rachael <rgeorge>
Component: ocs-operatorAssignee: Anmol Sachan <asachan>
Status: CLOSED ERRATA QA Contact: Rachael <rgeorge>
Severity: high Docs Contact:
Priority: unspecified    
Version: 4.6CC: asachan, ebenahar, madam, muagarwa, ocs-bugs, openshift-bugzilla-robot, pdhange, sostapov
Target Milestone: ---Keywords: AutomationBackLog
Target Release: OCS 4.6.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: 4.6.0-142.ci Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-12-17 06:24:47 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1889604    

Description Rachael 2020-10-15 11:10:36 UTC
Description of problem (please be detailed as possible and provide log
snippets):

When deployment of external mode in OCS 4.6 is attempted using a monitoring-endpoint which is not reachable, the ocs-operator crashes

$ oc get pods
NAME                                   READY   STATUS             RESTARTS   AGE
noobaa-operator-58958fb578-82xfw       1/1     Running            0          7m7s
ocs-metrics-exporter-b6ddd6869-n8nll   1/1     Running            0          7m7s
ocs-operator-6f48c6bff5-s48rg          0/1     CrashLoopBackOff   5          7m7s
rook-ceph-operator-8467f78647-4j56z    1/1     Running            0          7m7s


Version of all relevant components (if applicable):
ocs-operator.v4.6.0-601.ci
OCP: 4.6.0-0.nightly-2020-10-14-095718

Does this issue impact your ability to continue to work with the product
(please explain in detail what is the user impact)?
Yes, even though the endpoint is not valid, the operator should handle this and not result in a crash


Is there any workaround available to the best of your knowledge?
No

Rate from 1 - 5 the complexity of the scenario you performed that caused this
bug (1 - very simple, 5 - very complex)?
2

Can this issue reproducible?
Yes

Can this issue reproduce from the UI?


If this is a regression, please provide more details to justify this:


Steps to Reproduce:
1. Run the exporter script on the RHCS cluster and pass an unreachable IP as the monitoring endpoint
Eg: # python3 exporter46.py --rgw-endpoint 10.1.8.96:8080 --rbd-data-pool-name ocs-cbp-46-rg --run-as-user client.ocs46 --monitoring-endpoint 1.2.3.4 --monitoring-endpoint-port 9283

2. Upload the JSON generated from the script and deploy OCS in external mode


Actual results:
The ocs-operator crashes

Expected results:
The ocs-operator should handle this error gracefully

Comment 3 Mudit Agarwal 2020-10-15 11:53:08 UTC
This needs to be fixed, providing dev_ack.

Comment 6 Mudit Agarwal 2020-10-20 07:41:28 UTC
Backport PR is not yet merged.
https://github.com/openshift/ocs-operator/pull/853

Comment 7 Mudit Agarwal 2020-10-20 07:43:00 UTC
*** Bug 1889604 has been marked as a duplicate of this bug. ***

Comment 14 errata-xmlrpc 2020-12-17 06:24:47 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: Red Hat OpenShift Container Storage 4.6.0 security, bug fix, enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:5605