Bug 1599826

Summary:	When ceph is unreachable, Gnocchi raise an exception instead of the correct error message.
Product:	Red Hat OpenStack	Reporter:	Mehdi ABAAKOUK <mabaakou>
Component:	python-cradox	Assignee:	Mehdi ABAAKOUK <mabaakou>
Status:	CLOSED ERRATA	QA Contact:	Sasha Smolyak <ssmolyak>
Severity:	low	Docs Contact:
Priority:	low
Version:	14.0 (Rocky)	CC:	amaumene, apevec, augol, jjoyce, jschluet, lhh, mabaakou, nlevinki, pkilambi, psedlak, sasha
Target Milestone:	Upstream M3	Keywords:	Triaged
Target Release:	14.0 (Rocky)
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	python-cradox-2.1.0-1.el7ost	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:	1573878	Environment:
Last Closed:	2019-01-11 11:50:39 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1573878
Bug Blocks:

Comment 4 Sasha Smolyak 2018-11-13 09:59:26 UTC

How do I test it? I'm supposed to make ceph unreachable and then I'll get the message, right? Suggestions about making ceph unreachable?

Comment 5 Mehdi ABAAKOUK 2018-11-13 10:29:51 UTC

yes make ceph unreacheable and the exception raise should be a bit different.

You can stop all ceph-mon processes.

Comment 7 Sasha Smolyak 2018-12-12 14:27:58 UTC

So, the exact scenario for testing the fix:
on controller-0:
    sudo systemctl stop ceph-mon

on controller-1:
    sudo systemctl stop ceph-mon

on controller-2:
    sudo systemctl stop ceph-mon

Wait 3 minutes, then

on controller-0:
    sudo docker restart gnocchi_metricd
    
cat /var/log/containers/gnocchi/gnocchi-metricd.log
...
2018-12-12 14:04:19,515 [25] ERROR    gnocchi.utils: Unable to initialize storage driver
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/tenacity/__init__.py", line 333, in call
    result = fn(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/gnocchi/storage/__init__.py", line 102, in get_driver
    conf.storage)
  File "/usr/lib/python2.7/site-packages/gnocchi/storage/ceph.py", line 52, in __init__
    self.rados, self.ioctx = ceph.create_rados_connection(conf)
  File "/usr/lib/python2.7/site-packages/gnocchi/common/ceph.py", line 69, in create_rados_connection
    conn.connect()
  File "cradox.pyx", line 843, in cradox.Rados.connect (cradox.c:11327)
TimedOut: [errno 110] error connecting to the cluster

The line "ERROR    gnocchi.utils: Unable to initialize storage driver" is seen and not hidden anymore, so Fixed.

Comment 9 errata-xmlrpc 2019-01-11 11:50:39 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2019:0045