Bug 1461711 - Got "Dispatcher failed to handle the notification" after installation of latest puppet
Got "Dispatcher failed to handle the notification" after installation of late...
Status: CLOSED ERRATA
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-ceilometer (Show other bugs)
10.0 (Newton)
Unspecified Unspecified
unspecified Severity unspecified
: z4
: 10.0 (Newton)
Assigned To: Julien Danjou
Sasha Smolyak
: Triaged, ZStream
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2017-06-15 05:11 EDT by Sasha Smolyak
Modified: 2017-09-06 13:04 EDT (History)
5 users (show)

See Also:
Fixed In Version: openstack-ceilometer-7.1.0-2.el7ost
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2017-09-06 13:04:34 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
OpenStack gerrit 475420 None None None 2017-06-19 10:58 EDT

  None (edit)
Description Sasha Smolyak 2017-06-15 05:11:59 EDT
Description of problem:
Deployed the latest puddle of  2017-06-09.1 of RHOS10, watched the collector.log on undercloud:

2017-06-14 08:02:01.058 19696 ERROR ceilometer.event.storage.impl_sqlalchemy [-] Failed to record event: This Session's transaction has been rolled back by a nested rollback() call.  To begin a new transaction, issue Session.rollback() first.
2017-06-14 08:02:01.058 19696 ERROR ceilometer.event.storage.impl_sqlalchemy Traceback (most recent call last):
2017-06-14 08:02:01.058 19696 ERROR ceilometer.event.storage.impl_sqlalchemy   File "/usr/lib/python2.7/site-packages/ceilometer/event/storage/impl_sqlalchemy.py", line 197, in record_events
2017-06-14 08:02:01.058 19696 ERROR ceilometer.event.storage.impl_sqlalchemy     event_model.event_type, session=session)
2017-06-14 08:02:01.058 19696 ERROR ceilometer.event.storage.impl_sqlalchemy   File "/usr/lib/python2.7/site-packages/ceilometer/event/storage/impl_sqlalchemy.py", line 181, in _get_or_create_event_type
2017-06-14 08:02:01.058 19696 ERROR ceilometer.event.storage.impl_sqlalchemy     et = self._get_or_create_event_type(event_type, session)
2017-06-14 08:02:01.058 19696 ERROR ceilometer.event.storage.impl_sqlalchemy   File "/usr/lib/python2.7/site-packages/ceilometer/event/storage/impl_sqlalchemy.py", line 174, in _get_or_create_event_type
2017-06-14 08:02:01.058 19696 ERROR ceilometer.event.storage.impl_sqlalchemy     with session.begin(subtransactions=True):
2017-06-14 08:02:01.058 19696 ERROR ceilometer.event.storage.impl_sqlalchemy   File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 737, in begin
2017-06-14 08:02:01.058 19696 ERROR ceilometer.event.storage.impl_sqlalchemy     nested=nested)
2017-06-14 08:02:01.058 19696 ERROR ceilometer.event.storage.impl_sqlalchemy   File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 235, in _begin
2017-06-14 08:02:01.058 19696 ERROR ceilometer.event.storage.impl_sqlalchemy     self._assert_active()
2017-06-14 08:02:01.058 19696 ERROR ceilometer.event.storage.impl_sqlalchemy   File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 218, in _assert_active
2017-06-14 08:02:01.058 19696 ERROR ceilometer.event.storage.impl_sqlalchemy     "This Session's transaction has been rolled back "
2017-06-14 08:02:01.058 19696 ERROR ceilometer.event.storage.impl_sqlalchemy InvalidRequestError: This Session's transaction has been rolled back by a nested rollback() call.  To begin a new transaction, issue Session.rollback() first.
2017-06-14 08:02:01.058 19696 ERROR ceilometer.event.storage.impl_sqlalchemy 
2017-06-14 08:02:01.058 19696 ERROR ceilometer.collector [-] Dispatcher failed to handle the notification, re-queuing it.
2017-06-14 08:02:01.058 19696 ERROR ceilometer.collector Traceback (most recent call last):
2017-06-14 08:02:01.058 19696 ERROR ceilometer.collector   File "/usr/lib/python2.7/site-packages/ceilometer/collector.py", line 169, in sample
2017-06-14 08:02:01.058 19696 ERROR ceilometer.collector     self.dispatcher_manager.map_method(self.method, goods)
2017-06-14 08:02:01.058 19696 ERROR ceilometer.collector   File "/usr/lib/python2.7/site-packages/stevedore/extension.py", line 259, in map_method
2017-06-14 08:02:01.058 19696 ERROR ceilometer.collector     method_name, *args, **kwds)
2017-06-14 08:02:01.058 19696 ERROR ceilometer.collector   File "/usr/lib/python2.7/site-packages/stevedore/extension.py", line 232, in map
2017-06-14 08:02:01.058 19696 ERROR ceilometer.collector     self._invoke_one_plugin(response.append, func, e, args, kwds)
2017-06-14 08:02:01.058 19696 ERROR ceilometer.collector   File "/usr/lib/python2.7/site-packages/stevedore/extension.py", line 263, in _invoke_one_plugin
2017-06-14 08:02:01.058 19696 ERROR ceilometer.collector     response_callback(func(e, *args, **kwds))
2017-06-14 08:02:01.058 19696 ERROR ceilometer.collector   File "/usr/lib/python2.7/site-packages/stevedore/extension.py", line 237, in _call_extension_method
2017-06-14 08:02:01.058 19696 ERROR ceilometer.collector     return getattr(extension.obj, method_name)(*args, **kwds)
2017-06-14 08:02:01.058 19696 ERROR ceilometer.collector   File "/usr/lib/python2.7/site-packages/ceilometer/dispatcher/database.py", line 112, in record_events
2017-06-14 08:02:01.058 19696 ERROR ceilometer.collector     self.conn.record_events(event_list)
2017-06-14 08:02:01.058 19696 ERROR ceilometer.collector   File "/usr/lib/python2.7/site-packages/ceilometer/event/storage/impl_sqlalchemy.py", line 225, in record_events
2017-06-14 08:02:01.058 19696 ERROR ceilometer.collector     raise error
2017-06-14 08:02:01.058 19696 ERROR ceilometer.collector InvalidRequestError: This Session's transaction has been rolled back by a nested rollback() call.  To begin a new transaction, issue Session.rollback() first.
2017-06-14 08:02:01.058 19696 ERROR ceilometer.collector 



Version-Release number of selected component (if applicable):
openstack-ceilometer-common-7.0.4-4.el7ost.noarch
python-ceilometerclient-2.6.2-1.el7ost.noarch
openstack-ceilometer-api-7.0.4-4.el7ost.noarch
openstack-ceilometer-central-7.0.4-4.el7ost.noarch
python-ceilometer-7.0.4-4.el7ost.noarch
puppet-ceilometer-9.5.0-1.el7ost.noarch
python-ceilometermiddleware-0.5.1-1.el7ost.noarch
openstack-ceilometer-collector-7.0.4-4.el7ost.noarch
openstack-ceilometer-notification-7.0.4-4.el7ost.noarch
openstack-ceilometer-polling-7.0.4-4.el7ost.noarch


How reproducible:
100%

Steps to Reproduce:
1.Deploy the puddle, watch the log
2.
3.

Actual results:


Expected results:
No errors in the log

Additional info:
Comment 1 Julien Danjou 2017-06-19 10:58:42 EDT
Backport the original fix from Panko to Ceilometer in https://review.openstack.org/475420
Comment 2 Julien Danjou 2017-06-20 04:36:49 EDT
Ceilometer 7.0.5 has been released upstream with the fix.
Comment 7 errata-xmlrpc 2017-09-06 13:04:34 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:2651

Note You need to log in before you can comment on or make changes to this bug.