Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1539000

Summary: upgrade: creating/updating resource type raise HTTP 500
Product: Red Hat OpenStack Reporter: Mehdi ABAAKOUK <mabaakou>
Component: openstack-gnocchiAssignee: Mehdi ABAAKOUK <mabaakou>
Status: CLOSED NOTABUG QA Contact: Sasha Smolyak <ssmolyak>
Severity: high Docs Contact:
Priority: high    
Version: 11.0 (Ocata)CC: apevec, jschluet, lhh, mcornea
Target Milestone: zstreamKeywords: Triaged, ZStream
Target Release: 11.0 (Ocata)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-01-26 11:34:56 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1517977    

Description Mehdi ABAAKOUK 2018-01-26 11:10:10 UTC 
Description of problem:

Creating/updating resource type raise HTTP 500


 26 02:53:42.738419 2018] [:error] [pid 332625] [remote 172.17.1.10:0] mod_wsgi (pid=332625): Exception occurred processing WSGI script '/var/www/cgi-bin/gnocchi/app'.
[Fri Jan 26 02:53:42.738480 2018] [:error] [pid 332625] [remote 172.17.1.10:0] Traceback (most recent call last):
[Fri Jan 26 02:53:42.738498 2018] [:error] [pid 332625] [remote 172.17.1.10:0]   File "/usr/lib/python2.7/site-packages/webob/dec.py", line 130, in __call__
[Fri Jan 26 02:53:42.738873 2018] [:error] [pid 332625] [remote 172.17.1.10:0]     resp = self.call_func(req, *args, **self.kwargs)
[Fri Jan 26 02:53:42.738887 2018] [:error] [pid 332625] [remote 172.17.1.10:0]   File "/usr/lib/python2.7/site-packages/webob/dec.py", line 195, in call_func
[Fri Jan 26 02:53:42.738903 2018] [:error] [pid 332625] [remote 172.17.1.10:0]     return self.func(req, *args, **kwargs)
[Fri Jan 26 02:53:42.738910 2018] [:error] [pid 332625] [remote 172.17.1.10:0]   File "/usr/lib/python2.7/site-packages/oslo_middleware/base.py", line 126, in __call__
[Fri Jan 26 02:53:42.738995 2018] [:error] [pid 332625] [remote 172.17.1.10:0]     response = req.get_response(self.application)
[Fri Jan 26 02:53:42.739015 2018] [:error] [pid 332625] [remote 172.17.1.10:0]   File "/usr/lib/python2.7/site-packages/webob/request.py", line 1299, in send
[Fri Jan 26 02:53:42.739604 2018] [:error] [pid 332625] [remote 172.17.1.10:0]     application, catch_exc_info=False)
[Fri Jan 26 02:53:42.739626 2018] [:error] [pid 332625] [remote 172.17.1.10:0]   File "/usr/lib/python2.7/site-packages/webob/request.py", line 1263, in call_application
[Fri Jan 26 02:53:42.739651 2018] [:error] [pid 332625] [remote 172.17.1.10:0]     app_iter = application(self.environ, start_response)
[Fri Jan 26 02:53:42.739662 2018] [:error] [pid 332625] [remote 172.17.1.10:0]   File "/usr/lib/python2.7/site-packages/paste/urlmap.py", line 203, in __call__
[Fri Jan 26 02:53:42.739790 2018] [:error] [pid 332625] [remote 172.17.1.10:0]     return app(environ, start_response)
[Fri Jan 26 02:53:42.739801 2018] [:error] [pid 332625] [remote 172.17.1.10:0]   File "/usr/lib/python2.7/site-packages/webob/dec.py", line 130, in __call__
[Fri Jan 26 02:53:42.739815 2018] [:error] [pid 332625] [remote 172.17.1.10:0]     resp = self.call_func(req, *args, **self.kwargs)
[Fri Jan 26 02:53:42.739821 2018] [:error] [pid 332625] [remote 172.17.1.10:0]   File "/usr/lib/python2.7/site-packages/webob/dec.py", line 195, in call_func
[Fri Jan 26 02:53:42.739829 2018] [:error] [pid 332625] [remote 172.17.1.10:0]     return self.func(req, *args, **kwargs)
[Fri Jan 26 02:53:42.739835 2018] [:error] [pid 332625] [remote 172.17.1.10:0]   File "/usr/lib/python2.7/site-packages/oslo_middleware/base.py", line 126, in __call__
[Fri Jan 26 02:53:42.739855 2018] [:error] [pid 332625] [remote 172.17.1.10:0]     response = req.get_response(self.application)
[Fri Jan 26 02:53:42.739860 2018] [:error] [pid 332625] [remote 172.17.1.10:0]   File "/usr/lib/python2.7/site-packages/webob/request.py", line 1299, in send
[Fri Jan 26 02:53:42.739868 2018] [:error] [pid 332625] [remote 172.17.1.10:0]     application, catch_exc_info=False)
[Fri Jan 26 02:53:42.739873 2018] [:error] [pid 332625] [remote 172.17.1.10:0]   File "/usr/lib/python2.7/site-packages/webob/request.py", line 1263, in call_application
[Fri Jan 26 02:53:42.739880 2018] [:error] [pid 332625] [remote 172.17.1.10:0]     app_iter = application(self.environ, start_response)
[Fri Jan 26 02:53:42.739885 2018] [:error] [pid 332625] [remote 172.17.1.10:0]   File "/usr/lib/python2.7/site-packages/webob/dec.py", line 130, in __call__
[Fri Jan 26 02:53:42.739892 2018] [:error] [pid 332625] [remote 172.17.1.10:0]     resp = self.call_func(req, *args, **self.kwargs)
[Fri Jan 26 02:53:42.739897 2018] [:error] [pid 332625] [remote 172.17.1.10:0]   File "/usr/lib/python2.7/site-packages/webob/dec.py", line 195, in call_func
[Fri Jan 26 02:53:42.739904 2018] [:error] [pid 332625] [remote 172.17.1.10:0]     return self.func(req, *args, **kwargs)
[Fri Jan 26 02:53:42.739909 2018] [:error] [pid 332625] [remote 172.17.1.10:0]   File "/usr/lib/python2.7/site-packages/keystonemiddleware/auth_token/__init__.py", line 335, in __call__
[Fri Jan 26 02:53:42.740187 2018] [:error] [pid 332625] [remote 172.17.1.10:0]     response = req.get_response(self._app)
[Fri Jan 26 02:53:42.740199 2018] [:error] [pid 332625] [remote 172.17.1.10:0]   File "/usr/lib/python2.7/site-packages/webob/request.py", line 1299, in send
[Fri Jan 26 02:53:42.740212 2018] [:error] [pid 332625] [remote 172.17.1.10:0]     application, catch_exc_info=False)
[Fri Jan 26 02:53:42.740218 2018] [:error] [pid 332625] [remote 172.17.1.10:0]   File "/usr/lib/python2.7/site-packages/webob/request.py", line 1263, in call_application
[Fri Jan 26 02:53:42.740226 2018] [:error] [pid 332625] [remote 172.17.1.10:0]     app_iter = application(self.environ, start_response)
[Fri Jan 26 02:53:42.740231 2018] [:error] [pid 332625] [remote 172.17.1.10:0]   File "/usr/lib/python2.7/site-packages/webob/exc.py", line 1162, in __call__
[Fri Jan 26 02:53:42.740762 2018] [:error] [pid 332625] [remote 172.17.1.10:0]     return self.application(environ, start_response)
[Fri Jan 26 02:53:42.740780 2018] [:error] [pid 332625] [remote 172.17.1.10:0]   File "/usr/lib/python2.7/site-packages/gnocchi/rest/app.py", line 68, in __call__
[Fri Jan 26 02:53:42.740801 2018] [:error] [pid 332625] [remote 172.17.1.10:0]     return self.app(environ, start_response)
[Fri Jan 26 02:53:42.740808 2018] [:error] [pid 332625] [remote 172.17.1.10:0]   File "/usr/lib/python2.7/site-packages/pecan/middleware/recursive.py", line 56, in __call__
[Fri Jan 26 02:53:42.741048 2018] [:error] [pid 332625] [remote 172.17.1.10:0]     return self.application(environ, start_response)
[Fri Jan 26 02:53:42.741064 2018] [:error] [pid 332625] [remote 172.17.1.10:0]   File "/usr/lib/python2.7/site-packages/pecan/core.py", line 835, in __call__
[Fri Jan 26 02:53:42.742630 2018] [:error] [pid 332625] [remote 172.17.1.10:0]     return super(Pecan, self).__call__(environ, start_response)
[Fri Jan 26 02:53:42.742656 2018] [:error] [pid 332625] [remote 172.17.1.10:0]   File "/usr/lib/python2.7/site-packages/pecan/core.py", line 678, in __call__
[Fri Jan 26 02:53:42.742675 2018] [:error] [pid 332625] [remote 172.17.1.10:0]     self.invoke_controller(controller, args, kwargs, state)
[Fri Jan 26 02:53:42.742686 2018] [:error] [pid 332625] [remote 172.17.1.10:0]   File "/usr/lib/python2.7/site-packages/pecan/core.py", line 569, in invoke_controller
[Fri Jan 26 02:53:42.742697 2018] [:error] [pid 332625] [remote 172.17.1.10:0]     result = controller(*args, **kwargs)
[Fri Jan 26 02:53:42.742703 2018] [:error] [pid 332625] [remote 172.17.1.10:0]   File "/usr/lib/python2.7/site-packages/gnocchi/rest/__init__.py", line 725, in get
[Fri Jan 26 02:53:42.743093 2018] [:error] [pid 332625] [remote 172.17.1.10:0]     rt = pecan.request.indexer.get_resource_type(self._name)
[Fri Jan 26 02:53:42.743106 2018] [:error] [pid 332625] [remote 172.17.1.10:0]   File "/usr/lib/python2.7/site-packages/gnocchi/indexer/sqlalchemy.py", line 466, in get_resource_type
[Fri Jan 26 02:53:42.743485 2018] [:error] [pid 332625] [remote 172.17.1.10:0]     return self._get_resource_type(session, name)
[Fri Jan 26 02:53:42.743502 2018] [:error] [pid 332625] [remote 172.17.1.10:0]   File "/usr/lib/python2.7/site-packages/gnocchi/indexer/sqlalchemy.py", line 469, in _get_resource_type
[Fri Jan 26 02:53:42.743520 2018] [:error] [pid 332625] [remote 172.17.1.10:0]     resource_type = session.query(ResourceType).get(name)
[Fri Jan 26 02:53:42.743530 2018] [:error] [pid 332625] [remote 172.17.1.10:0]   File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 831, in get
[Fri Jan 26 02:53:42.744294 2018] [:error] [pid 332625] [remote 172.17.1.10:0]     return self._get_impl(ident, loading.load_on_ident)
[Fri Jan 26 02:53:42.744306 2018] [:error] [pid 332625] [remote 172.17.1.10:0]   File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 864, in _get_impl
[Fri Jan 26 02:53:42.744321 2018] [:error] [pid 332625] [remote 172.17.1.10:0]     return fallback_fn(self, key)
[Fri Jan 26 02:53:42.744327 2018] [:error] [pid 332625] [remote 172.17.1.10:0]   File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/loading.py", line 219, in load_on_ident
[Fri Jan 26 02:53:42.744515 2018] [:error] [pid 332625] [remote 172.17.1.10:0]     return q.one()
[Fri Jan 26 02:53:42.744531 2018] [:error] [pid 332625] [remote 172.17.1.10:0]   File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2693, in one
[Fri Jan 26 02:53:42.744546 2018] [:error] [pid 332625] [remote 172.17.1.10:0]     ret = list(self)
[Fri Jan 26 02:53:42.744552 2018] [:error] [pid 332625] [remote 172.17.1.10:0]   File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/loading.py", line 86, in instances
[Fri Jan 26 02:53:42.744561 2018] [:error] [pid 332625] [remote 172.17.1.10:0]     util.raise_from_cause(err)
[Fri Jan 26 02:53:42.744567 2018] [:error] [pid 332625] [remote 172.17.1.10:0]   File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/compat.py", line 200, in raise_from_cause
[Fri Jan 26 02:53:42.744646 2018] [:error] [pid 332625] [remote 172.17.1.10:0]     reraise(type(exception), exception, tb=exc_tb)
[Fri Jan 26 02:53:42.744654 2018] [:error] [pid 332625] [remote 172.17.1.10:0]   File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/loading.py", line 71, in instances
[Fri Jan 26 02:53:42.744665 2018] [:error] [pid 332625] [remote 172.17.1.10:0]     rows = [proc(row) for row in fetch]
[Fri Jan 26 02:53:42.744671 2018] [:error] [pid 332625] [remote 172.17.1.10:0]   File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/loading.py", line 428, in _instance
[Fri Jan 26 02:53:42.744679 2018] [:error] [pid 332625] [remote 172.17.1.10:0]     loaded_instance, populate_existing, populators)
[Fri Jan 26 02:53:42.744684 2018] [:error] [pid 332625] [remote 172.17.1.10:0]   File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/loading.py", line 486, in _populate_full
[Fri Jan 26 02:53:42.744692 2018] [:error] [pid 332625] [remote 172.17.1.10:0]     dict_[key] = getter(row)
[Fri Jan 26 02:53:42.744707 2018] [:error] [pid 332625] [remote 172.17.1.10:0]   File "/usr/lib64/python2.7/site-packages/sqlalchemy/sql/type_api.py", line 1030, in process
[Fri Jan 26 02:53:42.745127 2018] [:error] [pid 332625] [remote 172.17.1.10:0]     return process_value(value, dialect)
[Fri Jan 26 02:53:42.745140 2018] [:error] [pid 332625] [remote 172.17.1.10:0]   File "/usr/lib/python2.7/site-packages/gnocchi/indexer/sqlalchemy_base.py", line 125, in process_result_value
[Fri Jan 26 02:53:42.745288 2018] [:error] [pid 332625] [remote 172.17.1.10:0]     return value.replace(tzinfo=iso8601.iso8601.UTC)
[Fri Jan 26 02:53:42.745326 2018] [:error] [pid 332625] [remote 172.17.1.10:0] AttributeError: 'Decimal' object has no attribute 'replace'
Comment 1 Mehdi ABAAKOUK 2018-01-26 11:25:37 UTC 
full logs are here: https://bugzilla.redhat.com/show_bug.cgi?id=1517977#c16

Looks like Gnocchi API is 3.1 and database schema is still 3.0.

According timestamp ceilometer-upgrade run before gnocchi-upgrade.

So we must ensure gnocchi-upgrade run before ceilometer-upgrade.
Comment 2 Mehdi ABAAKOUK 2018-01-26 11:34:56 UTC 
The original depends bug was having the wrong NVR. Closing as not a bug.