Bug 1581331 - [UPDATE] glace is not accessible after update is get "Internal Server Error" for every command
Summary: [UPDATE] glace is not accessible after update is get "Internal Server Error"...
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-glance
Version: 11.0 (Ocata)
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: ---
Assignee: Cyril Roelandt
QA Contact: Mike Abrams
Kim Nylander
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-05-22 14:17 UTC by Raviv Bar-Tal
Modified: 2018-05-29 14:04 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-05-29 14:04:28 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
controller 2 logs (9.48 MB, application/x-xz)
2018-05-22 15:14 UTC, Raviv Bar-Tal
no flags Details

Description Raviv Bar-Tal 2018-05-22 14:17:31 UTC
Description of problem:
After updating to lates release (z5) glance commands exit with this error:
 "Internal Server Error"
in glance log on controller-2 we found this error:
2018-05-22 11:47:05.580 747087 ERROR glance.common.wsgi [req-0c73043f-94eb-49fa-90c8-9b2005723365 8c3cbf366ef54c40b292de07f1aa161d 5e62927ce5b14e7b9cc5a80ad2e25d49 - default default] Caught error: 'location_order'
2018-05-22 11:47:05.580 747087 ERROR glance.common.wsgi Traceback (most recent call last):
2018-05-22 11:47:05.580 747087 ERROR glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance/common/wsgi.py", line 1116, in __call__
2018-05-22 11:47:05.580 747087 ERROR glance.common.wsgi     request, **action_args)
2018-05-22 11:47:05.580 747087 ERROR glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance/common/wsgi.py", line 1155, in dispatch
2018-05-22 11:47:05.580 747087 ERROR glance.common.wsgi     return method(*args, **kwargs)
2018-05-22 11:47:05.580 747087 ERROR glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance/api/v2/images.py", line 111, in index
2018-05-22 11:47:05.580 747087 ERROR glance.common.wsgi     member_status=member_status)
2018-05-22 11:47:05.580 747087 ERROR glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance/api/authorization.py", line 111, in list
2018-05-22 11:47:05.580 747087 ERROR glance.common.wsgi     images = self.image_repo.list(*args, **kwargs)
2018-05-22 11:47:05.580 747087 ERROR glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance/domain/proxy.py", line 89, in list
2018-05-22 11:47:05.580 747087 ERROR glance.common.wsgi     items = self.base.list(*args, **kwargs)
2018-05-22 11:47:05.580 747087 ERROR glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance/api/policy.py", line 114, in list
2018-05-22 11:47:05.580 747087 ERROR glance.common.wsgi     return super(ImageRepoProxy, self).list(*args, **kwargs)
2018-05-22 11:47:05.580 747087 ERROR glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance/domain/proxy.py", line 89, in list
2018-05-22 11:47:05.580 747087 ERROR glance.common.wsgi     items = self.base.list(*args, **kwargs)
2018-05-22 11:47:05.580 747087 ERROR glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance/domain/proxy.py", line 89, in list
2018-05-22 11:47:05.580 747087 ERROR glance.common.wsgi     items = self.base.list(*args, **kwargs)
2018-05-22 11:47:05.580 747087 ERROR glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance/domain/proxy.py", line 89, in list
2018-05-22 11:47:05.580 747087 ERROR glance.common.wsgi     items = self.base.list(*args, **kwargs)
2018-05-22 11:47:05.580 747087 ERROR glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance/db/__init__.py", line 184, in list
2018-05-22 11:47:05.580 747087 ERROR glance.common.wsgi     image = self._format_image_from_db(db_image, db_image['tags'])
2018-05-22 11:47:05.580 747087 ERROR glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance/db/__init__.py", line 210, in _format_image_from_db
2018-05-22 11:47:05.580 747087 ERROR glance.common.wsgi     locations=location_strategy.get_ordered_locations(locations),
2018-05-22 11:47:05.580 747087 ERROR glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance/common/location_strategy/__init__.py", line 117, in get_ordered_locations
2018-05-22 11:47:05.580 747087 ERROR glance.common.wsgi     strategy_module = _available_strategies[CONF.location_strategy]
2018-05-22 11:47:05.580 747087 ERROR glance.common.wsgi KeyError: 'location_order'
2018-05-22 11:47:05.580 747087 ERROR glance.common.wsgi 


Version-Release number of selected component (if applicable):

python-glance-14.0.1-1.el7ost.noarch
python-glanceclient-2.6.0-1.el7ost.noarch
openstack-glance-14.0.1-1.el7ost.noarch
python-glance-store-0.20.0-1.el7ost.noarch
puppet-glance-10.4.0-1.el7ost.noarch

How reproducible:


Steps to Reproduce:
1. install z4 cloud
2. update cloud to z5
3. try to list or load image

Actual results:

see attached file.
Expected results:


Additional info:

Comment 1 Raviv Bar-Tal 2018-05-22 15:14:19 UTC
Created attachment 1440246 [details]
controller 2 logs

Comment 2 Cyril Roelandt 2018-05-28 14:24:14 UTC
Does this happen on a machine we can tinker with? I'd love to add debug statements in _load_strategies and see what happens there.

Comment 4 Raviv Bar-Tal 2018-05-29 14:04:28 UTC
I did try to reproduce this bug and failed,
The glance error could be caused by tlv lab network problems.


Note You need to log in before you can comment on or make changes to this bug.