Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1581331

Summary: [UPDATE] glace is not accessible after update is get "Internal Server Error" for every command
Product: Red Hat OpenStack Reporter: Raviv Bar-Tal <rbartal>
Component: openstack-glanceAssignee: Cyril Roelandt <cyril>
Status: CLOSED NOTABUG QA Contact: Mike Abrams <mabrams>
Severity: high Docs Contact: Kim Nylander <knylande>
Priority: high    
Version: 11.0 (Ocata)CC: eglynn, fpercoco, rbartal, srevivo, thiago
Target Milestone: ---Keywords: Triaged, ZStream
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-05-29 14:04:28 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
controller 2 logs none

Description Raviv Bar-Tal 2018-05-22 14:17:31 UTC 
Description of problem:
After updating to lates release (z5) glance commands exit with this error:
 "Internal Server Error"
in glance log on controller-2 we found this error:
2018-05-22 11:47:05.580 747087 ERROR glance.common.wsgi [req-0c73043f-94eb-49fa-90c8-9b2005723365 8c3cbf366ef54c40b292de07f1aa161d 5e62927ce5b14e7b9cc5a80ad2e25d49 - default default] Caught error: 'location_order'
2018-05-22 11:47:05.580 747087 ERROR glance.common.wsgi Traceback (most recent call last):
2018-05-22 11:47:05.580 747087 ERROR glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance/common/wsgi.py", line 1116, in __call__
2018-05-22 11:47:05.580 747087 ERROR glance.common.wsgi     request, **action_args)
2018-05-22 11:47:05.580 747087 ERROR glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance/common/wsgi.py", line 1155, in dispatch
2018-05-22 11:47:05.580 747087 ERROR glance.common.wsgi     return method(*args, **kwargs)
2018-05-22 11:47:05.580 747087 ERROR glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance/api/v2/images.py", line 111, in index
2018-05-22 11:47:05.580 747087 ERROR glance.common.wsgi     member_status=member_status)
2018-05-22 11:47:05.580 747087 ERROR glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance/api/authorization.py", line 111, in list
2018-05-22 11:47:05.580 747087 ERROR glance.common.wsgi     images = self.image_repo.list(*args, **kwargs)
2018-05-22 11:47:05.580 747087 ERROR glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance/domain/proxy.py", line 89, in list
2018-05-22 11:47:05.580 747087 ERROR glance.common.wsgi     items = self.base.list(*args, **kwargs)
2018-05-22 11:47:05.580 747087 ERROR glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance/api/policy.py", line 114, in list
2018-05-22 11:47:05.580 747087 ERROR glance.common.wsgi     return super(ImageRepoProxy, self).list(*args, **kwargs)
2018-05-22 11:47:05.580 747087 ERROR glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance/domain/proxy.py", line 89, in list
2018-05-22 11:47:05.580 747087 ERROR glance.common.wsgi     items = self.base.list(*args, **kwargs)
2018-05-22 11:47:05.580 747087 ERROR glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance/domain/proxy.py", line 89, in list
2018-05-22 11:47:05.580 747087 ERROR glance.common.wsgi     items = self.base.list(*args, **kwargs)
2018-05-22 11:47:05.580 747087 ERROR glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance/domain/proxy.py", line 89, in list
2018-05-22 11:47:05.580 747087 ERROR glance.common.wsgi     items = self.base.list(*args, **kwargs)
2018-05-22 11:47:05.580 747087 ERROR glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance/db/__init__.py", line 184, in list
2018-05-22 11:47:05.580 747087 ERROR glance.common.wsgi     image = self._format_image_from_db(db_image, db_image['tags'])
2018-05-22 11:47:05.580 747087 ERROR glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance/db/__init__.py", line 210, in _format_image_from_db
2018-05-22 11:47:05.580 747087 ERROR glance.common.wsgi     locations=location_strategy.get_ordered_locations(locations),
2018-05-22 11:47:05.580 747087 ERROR glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance/common/location_strategy/__init__.py", line 117, in get_ordered_locations
2018-05-22 11:47:05.580 747087 ERROR glance.common.wsgi     strategy_module = _available_strategies[CONF.location_strategy]
2018-05-22 11:47:05.580 747087 ERROR glance.common.wsgi KeyError: 'location_order'
2018-05-22 11:47:05.580 747087 ERROR glance.common.wsgi 


Version-Release number of selected component (if applicable):

python-glance-14.0.1-1.el7ost.noarch
python-glanceclient-2.6.0-1.el7ost.noarch
openstack-glance-14.0.1-1.el7ost.noarch
python-glance-store-0.20.0-1.el7ost.noarch
puppet-glance-10.4.0-1.el7ost.noarch

How reproducible:


Steps to Reproduce:
1. install z4 cloud
2. update cloud to z5
3. try to list or load image

Actual results:

see attached file.
Expected results:


Additional info:
Comment 1 Raviv Bar-Tal 2018-05-22 15:14:19 UTC 
Created attachment 1440246 [details]
controller 2 logs
Comment 2 Cyril Roelandt 2018-05-28 14:24:14 UTC 
Does this happen on a machine we can tinker with? I'd love to add debug statements in _load_strategies and see what happens there.
Comment 4 Raviv Bar-Tal 2018-05-29 14:04:28 UTC 
I did try to reproduce this bug and failed,
The glance error could be caused by tlv lab network problems.