Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1461711

Summary:	Got "Dispatcher failed to handle the notification" after installation of latest puppet
Product:	Red Hat OpenStack	Reporter:	Sasha Smolyak <ssmolyak>
Component:	openstack-ceilometer	Assignee:	Julien Danjou <jdanjou>
Status:	CLOSED ERRATA	QA Contact:	Sasha Smolyak <ssmolyak>
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	10.0 (Newton)	CC:	jdanjou, jjoyce, jruzicka, pkilambi, srevivo
Target Milestone:	z4	Keywords:	Triaged, ZStream
Target Release:	10.0 (Newton)
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	openstack-ceilometer-7.1.0-2.el7ost	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2017-09-06 17:04:34 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Sasha Smolyak 2017-06-15 09:11:59 UTC

Description of problem:
Deployed the latest puddle of  2017-06-09.1 of RHOS10, watched the collector.log on undercloud:

2017-06-14 08:02:01.058 19696 ERROR ceilometer.event.storage.impl_sqlalchemy [-] Failed to record event: This Session's transaction has been rolled back by a nested rollback() call.  To begin a new transaction, issue Session.rollback() first.
2017-06-14 08:02:01.058 19696 ERROR ceilometer.event.storage.impl_sqlalchemy Traceback (most recent call last):
2017-06-14 08:02:01.058 19696 ERROR ceilometer.event.storage.impl_sqlalchemy   File "/usr/lib/python2.7/site-packages/ceilometer/event/storage/impl_sqlalchemy.py", line 197, in record_events
2017-06-14 08:02:01.058 19696 ERROR ceilometer.event.storage.impl_sqlalchemy     event_model.event_type, session=session)
2017-06-14 08:02:01.058 19696 ERROR ceilometer.event.storage.impl_sqlalchemy   File "/usr/lib/python2.7/site-packages/ceilometer/event/storage/impl_sqlalchemy.py", line 181, in _get_or_create_event_type
2017-06-14 08:02:01.058 19696 ERROR ceilometer.event.storage.impl_sqlalchemy     et = self._get_or_create_event_type(event_type, session)
2017-06-14 08:02:01.058 19696 ERROR ceilometer.event.storage.impl_sqlalchemy   File "/usr/lib/python2.7/site-packages/ceilometer/event/storage/impl_sqlalchemy.py", line 174, in _get_or_create_event_type
2017-06-14 08:02:01.058 19696 ERROR ceilometer.event.storage.impl_sqlalchemy     with session.begin(subtransactions=True):
2017-06-14 08:02:01.058 19696 ERROR ceilometer.event.storage.impl_sqlalchemy   File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 737, in begin
2017-06-14 08:02:01.058 19696 ERROR ceilometer.event.storage.impl_sqlalchemy     nested=nested)
2017-06-14 08:02:01.058 19696 ERROR ceilometer.event.storage.impl_sqlalchemy   File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 235, in _begin
2017-06-14 08:02:01.058 19696 ERROR ceilometer.event.storage.impl_sqlalchemy     self._assert_active()
2017-06-14 08:02:01.058 19696 ERROR ceilometer.event.storage.impl_sqlalchemy   File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 218, in _assert_active
2017-06-14 08:02:01.058 19696 ERROR ceilometer.event.storage.impl_sqlalchemy     "This Session's transaction has been rolled back "
2017-06-14 08:02:01.058 19696 ERROR ceilometer.event.storage.impl_sqlalchemy InvalidRequestError: This Session's transaction has been rolled back by a nested rollback() call.  To begin a new transaction, issue Session.rollback() first.
2017-06-14 08:02:01.058 19696 ERROR ceilometer.event.storage.impl_sqlalchemy 
2017-06-14 08:02:01.058 19696 ERROR ceilometer.collector [-] Dispatcher failed to handle the notification, re-queuing it.
2017-06-14 08:02:01.058 19696 ERROR ceilometer.collector Traceback (most recent call last):
2017-06-14 08:02:01.058 19696 ERROR ceilometer.collector   File "/usr/lib/python2.7/site-packages/ceilometer/collector.py", line 169, in sample
2017-06-14 08:02:01.058 19696 ERROR ceilometer.collector     self.dispatcher_manager.map_method(self.method, goods)
2017-06-14 08:02:01.058 19696 ERROR ceilometer.collector   File "/usr/lib/python2.7/site-packages/stevedore/extension.py", line 259, in map_method
2017-06-14 08:02:01.058 19696 ERROR ceilometer.collector     method_name, *args, **kwds)
2017-06-14 08:02:01.058 19696 ERROR ceilometer.collector   File "/usr/lib/python2.7/site-packages/stevedore/extension.py", line 232, in map
2017-06-14 08:02:01.058 19696 ERROR ceilometer.collector     self._invoke_one_plugin(response.append, func, e, args, kwds)
2017-06-14 08:02:01.058 19696 ERROR ceilometer.collector   File "/usr/lib/python2.7/site-packages/stevedore/extension.py", line 263, in _invoke_one_plugin
2017-06-14 08:02:01.058 19696 ERROR ceilometer.collector     response_callback(func(e, *args, **kwds))
2017-06-14 08:02:01.058 19696 ERROR ceilometer.collector   File "/usr/lib/python2.7/site-packages/stevedore/extension.py", line 237, in _call_extension_method
2017-06-14 08:02:01.058 19696 ERROR ceilometer.collector     return getattr(extension.obj, method_name)(*args, **kwds)
2017-06-14 08:02:01.058 19696 ERROR ceilometer.collector   File "/usr/lib/python2.7/site-packages/ceilometer/dispatcher/database.py", line 112, in record_events
2017-06-14 08:02:01.058 19696 ERROR ceilometer.collector     self.conn.record_events(event_list)
2017-06-14 08:02:01.058 19696 ERROR ceilometer.collector   File "/usr/lib/python2.7/site-packages/ceilometer/event/storage/impl_sqlalchemy.py", line 225, in record_events
2017-06-14 08:02:01.058 19696 ERROR ceilometer.collector     raise error
2017-06-14 08:02:01.058 19696 ERROR ceilometer.collector InvalidRequestError: This Session's transaction has been rolled back by a nested rollback() call.  To begin a new transaction, issue Session.rollback() first.
2017-06-14 08:02:01.058 19696 ERROR ceilometer.collector 



Version-Release number of selected component (if applicable):
openstack-ceilometer-common-7.0.4-4.el7ost.noarch
python-ceilometerclient-2.6.2-1.el7ost.noarch
openstack-ceilometer-api-7.0.4-4.el7ost.noarch
openstack-ceilometer-central-7.0.4-4.el7ost.noarch
python-ceilometer-7.0.4-4.el7ost.noarch
puppet-ceilometer-9.5.0-1.el7ost.noarch
python-ceilometermiddleware-0.5.1-1.el7ost.noarch
openstack-ceilometer-collector-7.0.4-4.el7ost.noarch
openstack-ceilometer-notification-7.0.4-4.el7ost.noarch
openstack-ceilometer-polling-7.0.4-4.el7ost.noarch


How reproducible:
100%

Steps to Reproduce:
1.Deploy the puddle, watch the log
2.
3.

Actual results:


Expected results:
No errors in the log

Additional info:

Comment 1 Julien Danjou 2017-06-19 14:58:42 UTC

Backport the original fix from Panko to Ceilometer in https://review.openstack.org/475420

Comment 2 Julien Danjou 2017-06-20 08:36:49 UTC

Ceilometer 7.0.5 has been released upstream with the fix.

Comment 7 errata-xmlrpc 2017-09-06 17:04:34 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:2651