Bug 1614928
Summary: | Unable to upload OC images to glance: Faile d to upload image data due to internal error | ||
---|---|---|---|
Product: | Red Hat OpenStack | Reporter: | Alexander Chuzhoy <sasha> |
Component: | python-glance-store | Assignee: | Cyril Roelandt <cyril> |
Status: | CLOSED ERRATA | QA Contact: | Filip Hubík <fhubik> |
Severity: | urgent | Docs Contact: | Kim Nylander <knylande> |
Priority: | urgent | ||
Version: | 14.0 (Rocky) | CC: | apevec, aschultz, cschwede, cyril, eglynn, ekuvaja, fhubik, fpercoco, jschluet, lhh, markmc, mburns, mlammon, srevivo |
Target Milestone: | beta | Keywords: | Automation, AutomationBlocker, Triaged |
Target Release: | 14.0 (Rocky) | ||
Hardware: | Unspecified | ||
OS: | Unspecified | ||
Whiteboard: | |||
Fixed In Version: | python-glance-store-0.26.1-1.el7ost | Doc Type: | If docs needed, set a value |
Doc Text: | Story Points: | --- | |
Clone Of: | Environment: | ||
Last Closed: | 2019-01-11 11:51:11 UTC | Type: | Bug |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: |
Description
Alexander Chuzhoy
2018-08-10 18:36:21 UTC
Errors in /var/log/containers/glance/api.log: 2018-08-10 14:36:11.104 26 ERROR glance.api.v2.image_data [req-cd51852d-7df0-47a1-8187-c6f7e71228a1 4c59164086924d5083d12cf70922679b e57037826de841c392b6cb7860452a93 - default default] Faile d to upload image data due to internal error 2018-08-10 14:36:11.205 26 ERROR glance.common.wsgi [req-cd51852d-7df0-47a1-8187-c6f7e71228a1 4c59164086924d5083d12cf70922679b e57037826de841c392b6cb7860452a93 - default default] Caught erro r: 'module' object has no attribute 'add_to_backend_with_multihash': AttributeError: 'module' object has no attribute 'add_to_backend_with_multihash' 2018-08-10 14:36:11.205 26 ERROR glance.common.wsgi Traceback (most recent call last): 2018-08-10 14:36:11.205 26 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/common/wsgi.py", line 1274, in __call__ 2018-08-10 14:36:11.205 26 ERROR glance.common.wsgi request, **action_args) 2018-08-10 14:36:11.205 26 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/common/wsgi.py", line 1317, in dispatch 2018-08-10 14:36:11.205 26 ERROR glance.common.wsgi return method(*args, **kwargs) 2018-08-10 14:36:11.205 26 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/common/utils.py", line 417, in wrapped 2018-08-10 14:36:11.205 26 ERROR glance.common.wsgi return func(self, req, *args, **kwargs) 2018-08-10 14:36:11.205 26 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/api/v2/image_data.py", line 279, in upload 2018-08-10 14:36:11.205 26 ERROR glance.common.wsgi self._restore(image_repo, image) 2018-08-10 14:36:11.205 26 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2018-08-10 14:36:11.205 26 ERROR glance.common.wsgi self.force_reraise() 2018-08-10 14:36:11.205 26 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2018-08-10 14:36:11.205 26 ERROR glance.common.wsgi six.reraise(self.type_, self.value, self.tb) 2018-08-10 14:36:11.205 26 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/api/v2/image_data.py", line 144, in upload 2018-08-10 14:36:11.205 26 ERROR glance.common.wsgi image.set_data(data, size, backend=backend) 2018-08-10 14:36:11.205 26 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/domain/proxy.py", line 198, in set_data 2018-08-10 14:36:11.205 26 ERROR glance.common.wsgi self.base.set_data(data, size, backend=backend) 2018-08-10 14:36:11.205 26 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/notifier.py", line 485, in set_data 2018-08-10 14:36:11.205 26 ERROR glance.common.wsgi _send_notification(notify_error, 'image.upload', msg) 2018-08-10 14:36:11.205 26 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2018-08-10 14:36:11.205 26 ERROR glance.common.wsgi self.force_reraise() 2018-08-10 14:36:11.205 26 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2018-08-10 14:36:11.205 26 ERROR glance.common.wsgi six.reraise(self.type_, self.value, self.tb) 2018-08-10 14:36:11.205 26 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/notifier.py", line 432, in set_data 2018-08-10 14:36:11.205 26 ERROR glance.common.wsgi self.repo.set_data(data, size, backend=backend) 2018-08-10 14:36:11.205 26 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/api/policy.py", line 193, in set_data 2018-08-10 14:36:11.205 26 ERROR glance.common.wsgi return self.image.set_data(*args, **kwargs) 2018-08-10 14:36:11.205 26 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/quota/__init__.py", line 310, in set_data 2018-08-10 14:36:11.205 26 ERROR glance.common.wsgi self.image.set_data(data, size=size, backend=backend) 2018-08-10 14:36:11.205 26 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/location.py", line 475, in set_data 2018-08-10 14:36:11.205 26 ERROR glance.common.wsgi loc_meta) = self.store_api.add_to_backend_with_multihash( 2018-08-10 14:36:11.205 26 ERROR glance.common.wsgi AttributeError: 'module' object has no attribute 'add_to_backend_with_multihash' 2018-08-10 14:36:11.205 26 ERROR glance.common.wsgi missing python-chardet-3.0.4-7.el7 that came along with python-requests-2.19.1-3.el7 https://review.rdoproject.org/r/#/c/15077/3/deps.yml Looks like we might be using old version of glance_store. Could someone please check which version of glance_store is included in these puddles for me? We've python-chardet-3.0.4-7.el7ost since 2018-08-13.1 puddle I had a look into the glance container, and it looks like we're missing an update of python-glance-store. This patch is missing: https://review.openstack.org/#/c/587098/ Thus we need an updated build of python-glance-store (latest is https://brewweb.engineering.redhat.com/brew/buildinfo?buildID=732749). In freshest puddle (2018-08-15.1) passing UC install and image upload stages we have package python2-glance-store-0.26.1-1.el7ost present inside openstack-glance-api container. Images are uploaded to glance registry and they are able to reach active state, they are not stuck in queued state anymore: uc $ openstack image list +--------------------------------------+------------------------+--------+ | ID | Name | Status | +--------------------------------------+------------------------+--------+ | 2c7f5391-c8b1-486b-b793-024d1b06ea8e | bm-deploy-kernel | active | | 7fb9e254-3348-41bd-bf2f-8c335bb166f8 | bm-deploy-ramdisk | active | | d37889df-ccc4-468d-9998-3dae962dc0df | overcloud-full | active | | e38d2bb7-ef2d-4f98-b2fc-32a780e566b5 | overcloud-full-initrd | active | | 4c5cad73-7b96-417f-8f70-0672ac8811fd | overcloud-full-vmlinuz | active | +--------------------------------------+------------------------+--------+ Verified. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2019:0045 |