Description of problem: Swift stores full (username, password, host) locations URIs inside the database. This turns out to be: 1) A security issue since anyone with access to the database could have the store's username, password and host info. 2) An issue when any of those params change since they'll be changed in the config file but not in the existing images' records
There's a plan for doing a bigger change than the one proposed in the review is being planned to address this issue. From yesterday's meeting: Log: http://eavesdrop.openstack.org/meetings/glance/2013/glance.2013-11-21-14.02.log.html 14:28:56 <flaper87> for example: https://review.openstack.org/#/c/34801/ 14:29:23 <flaper87> that was opened on June 27th 14:29:29 <markwash> "Remove user and key from location in Swift" 14:29:36 <markwash> yeah that review is definitely on my mind 14:29:46 <markwash> there has been some recent discussion which I guess has been unfortunately kind of hidden 14:29:54 <markwash> which has kept me from pursuing that 14:30:00 <flaper87> oh 14:30:40 <markwash> rosmaita: would you say the conversations we've been having with smclaren about solving the credentials problem are relevant to that patch? 14:30:47 <markwash> and might change the direction we want to go? 14:31:00 <rosmaita> yes, i would hold off on that patch 14:31:34 <rosmaita> i can post Stuart's doc link if others are interested 14:31:45 <flaper87> rosmaita: yup, please. 14:31:49 <markwash> I might -1 it with a note just to pull it out of the stats 14:31:57 <rosmaita> don't mean to work hidden, but don't want to broadcast our prob The proposal will address the issue in several stores besides swift. Moving to 5.0 since it won't be fixed for 4.0 and backporting the patch doesn't make sense in this case.
I configured swift as the backend for glance. Then created 'image2' before changing admin credentials and 'image3' after changing the admin credentials. Now I'm not able to delete images that were created with old credentials. But deleting new images work. Please let me know if I can reopen the bug since the fix may not be working here. CONSOLE OUTPUT ============== [root@lynx13 ~(keystone_admin)]# vi /etc/glance/glance-api.conf [root@lynx13 ~(keystone_admin)]# !138 service openstack-glance-api restart Redirecting to /bin/systemctl restart openstack-glance-api.service [root@lynx13 ~(keystone_admin)]# !139 service openstack-glance-registry restart Redirecting to /bin/systemctl restart openstack-glance-registry.service [root@lynx13 ~(keystone_admin)]# source keystonerc_admin [root@lynx13 ~(keystone_admin)]# glance image-create --name image3 --min-disk 1 --min-ram 768 --file cirros.qcow2 --is-public True --disk-format=qcow2 --container-format bare --progress +------------------+--------------------------------------+ | Property | Value | +------------------+--------------------------------------+ | checksum | d41d8cd98f00b204e9800998ecf8427e | | container_format | bare | | created_at | 2015-02-17T14:43:11 | | deleted | False | | deleted_at | None | | disk_format | qcow2 | | id | 97af10f6-72ea-476b-a52f-fecb4ba70a09 | | is_public | True | | min_disk | 1 | | min_ram | 768 | | name | image3 | | owner | 1343b6debee64b14966bf6742425aa21 | | protected | False | | size | 0 | | status | active | | updated_at | 2015-02-17T14:43:11 | | virtual_size | None | +------------------+--------------------------------------+ [root@lynx13 ~(keystone_admin)]# glance image-list +--------------------------------------+--------+-------------+------------------+----------+--------+ | ID | Name | Disk Format | Container Format | Size | Status | +--------------------------------------+--------+-------------+------------------+----------+--------+ | ce70e62d-14b6-43fa-b3b5-2afa6a9fa308 | cirros | qcow2 | bare | 13200896 | active | | 4016e592-d8c0-4f30-b898-e40c2d63f911 | image1 | qcow2 | bare | | active | | d9dc2553-7d10-4b9b-a34d-3d60c4fcbe76 | image2 | qcow2 | bare | | active | | 97af10f6-72ea-476b-a52f-fecb4ba70a09 | image3 | qcow2 | bare | | active | +--------------------------------------+--------+-------------+------------------+----------+--------+ DELETING NEW IMAGES PASS ========================= [root@lynx13 ~(keystone_admin)]# glance image-delete 97af10f6-72ea-476b-a52f-fecb4ba70a09 [root@lynx13 ~(keystone_admin)]# DELETING AN OLD IMAGE FAILS ============================ [root@lynx13 ~(keystone_admin)]# glance image-delete 4016e592-d8c0-4f30-b898-e40c2d63f911 HTTPInternalServerError (HTTP 500): Unable to delete image 4016e592-d8c0-4f30-b898-e40c2d63f911 [root@lynx13 ~(keystone_admin)]# glance image-delete d9dc2553-7d10-4b9b-a34d-3d60c4fcbe76 HTTPInternalServerError (HTTP 500): Unable to delete image d9dc2553-7d10-4b9b-a34d-3d60c4fcbe76 SAMPLE LOGS FOR FAILURE (/var/log/glance-api.log) =================================================== 2015-02-17 16:49:37.055 77137 ERROR swiftclient [08ee6676-515e-4089-87cb-7b1d23e72723 828ad19213794cd290b541d736f8fb6f 1343b6debee64b14966bf6742425aa21 - - -] Unauthorized. Check username, password and tenant name/id. 2015-02-17 16:49:37.055 77137 TRACE swiftclient Traceback (most recent call last): 2015-02-17 16:49:37.055 77137 TRACE swiftclient File "/usr/lib/python2.7/site-packages/swiftclient/client.py", line 1235, in _retry 2015-02-17 16:49:37.055 77137 TRACE swiftclient self.url, self.token = self.get_auth() 2015-02-17 16:49:37.055 77137 TRACE swiftclient File "/usr/lib/python2.7/site-packages/swiftclient/client.py", line 1209, in get_auth 2015-02-17 16:49:37.055 77137 TRACE swiftclient insecure=self.insecure) 2015-02-17 16:49:37.055 77137 TRACE swiftclient File "/usr/lib/python2.7/site-packages/swiftclient/client.py", line 406, in get_auth 2015-02-17 16:49:37.055 77137 TRACE swiftclient auth_version=auth_version) 2015-02-17 16:49:37.055 77137 TRACE swiftclient File "/usr/lib/python2.7/site-packages/swiftclient/client.py", line 339, in get_auth_keystone 2015-02-17 16:49:37.055 77137 TRACE swiftclient raise ClientException(msg) 2015-02-17 16:49:37.055 77137 TRACE swiftclient ClientException: Unauthorized. Check username, password and tenant name/id. 2015-02-17 16:49:37.055 77137 TRACE swiftclient 2015-02-17 16:49:37.103 77137 INFO glance.wsgi.server [08ee6676-515e-4089-87cb-7b1d23e72723 828ad19213794cd290b541d736f8fb6f 1343b6debee64b14966bf6742425aa21 - - -] Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/eventlet/wsgi.py", line 433, in handle_one_response result = self.application(self.environ, start_response) File "/usr/lib/python2.7/site-packages/webob/dec.py", line 130, in __call__ resp = self.call_func(req, *args, **self.kwargs) File "/usr/lib/python2.7/site-packages/webob/dec.py", line 195, in call_func return self.func(req, *args, **kwargs) File "/usr/lib/python2.7/site-packages/glance/common/wsgi.py", line 394, in __call__ response = req.get_response(self.application) File "/usr/lib/python2.7/site-packages/webob/request.py", line 1296, in send application, catch_exc_info=False) File "/usr/lib/python2.7/site-packages/webob/request.py", line 1260, in call_application app_iter = application(self.environ, start_response) File "/usr/lib/python2.7/site-packages/webob/dec.py", line 130, in __call__ resp = self.call_func(req, *args, **self.kwargs) File "/usr/lib/python2.7/site-packages/webob/dec.py", line 195, in call_func return self.func(req, *args, **kwargs) File "/usr/lib/python2.7/site-packages/osprofiler/web.py", line 99, in __call__ return request.get_response(self.application) File "/usr/lib/python2.7/site-packages/webob/request.py", line 1296, in send application, catch_exc_info=False) File "/usr/lib/python2.7/site-packages/webob/request.py", line 1260, in call_application app_iter = application(self.environ, start_response) File "/usr/lib/python2.7/site-packages/keystonemiddleware/auth_token.py", line 748, in __call__ return self._call_app(env, start_response) File "/usr/lib/python2.7/site-packages/keystonemiddleware/auth_token.py", line 684, in _call_app return self._app(env, _fake_start_response) File "/usr/lib/python2.7/site-packages/webob/dec.py", line 130, in __call__ resp = self.call_func(req, *args, **self.kwargs) File "/usr/lib/python2.7/site-packages/webob/dec.py", line 195, in call_func return self.func(req, *args, **kwargs) File "/usr/lib/python2.7/site-packages/glance/common/wsgi.py", line 394, in __call__ response = req.get_response(self.application) File "/usr/lib/python2.7/site-packages/webob/request.py", line 1296, in send application, catch_exc_info=False) File "/usr/lib/python2.7/site-packages/webob/request.py", line 1260, in call_application app_iter = application(self.environ, start_response) File "/usr/lib/python2.7/site-packages/paste/urlmap.py", line 203, in __call__ return app(environ, start_response) File "/usr/lib/python2.7/site-packages/webob/dec.py", line 144, in __call__ return resp(environ, start_response) File "/usr/lib/python2.7/site-packages/routes/middleware.py", line 131, in __call__ response = self.app(environ, start_response) File "/usr/lib/python2.7/site-packages/webob/dec.py", line 144, in __call__ return resp(environ, start_response) File "/usr/lib/python2.7/site-packages/webob/dec.py", line 130, in __call__ resp = self.call_func(req, *args, **self.kwargs) File "/usr/lib/python2.7/site-packages/webob/dec.py", line 195, in call_func return self.func(req, *args, **kwargs) File "/usr/lib/python2.7/site-packages/glance/common/wsgi.py", line 683, in __call__ request, **action_args) File "/usr/lib/python2.7/site-packages/glance/common/wsgi.py", line 707, in dispatch return method(*args, **kwargs) File "/usr/lib/python2.7/site-packages/glance/common/utils.py", line 449, in wrapped return func(self, req, *args, **kwargs) File "/usr/lib/python2.7/site-packages/glance/api/v1/images.py", line 1073, in delete {'status': ori_status}) File "/usr/lib/python2.7/site-packages/glance/openstack/common/excutils.py", line 82, in __exit__ six.reraise(self.type_, self.value, self.tb) File "/usr/lib/python2.7/site-packages/glance/api/v1/images.py", line 1069, in delete upload_utils.initiate_deletion(req, loc_data, id) File "/usr/lib/python2.7/site-packages/glance/api/v1/upload_utils.py", line 46, in initiate_deletion id, location_data) File "/usr/lib/python2.7/site-packages/glance/common/store_utils.py", line 124, in delete_image_location_from_backend safe_delete_from_backend(context, image_id, location) File "/usr/lib/python2.7/site-packages/glance/common/store_utils.py", line 58, in safe_delete_from_backend ret = store_api.delete_from_backend(location['url'], context=context) File "/usr/lib/python2.7/site-packages/glance_store/backend.py", line 280, in delete_from_backend return store.delete(loc, context=context) File "/usr/lib/python2.7/site-packages/glance_store/_drivers/swift/store.py", line 593, in delete location.container, location.obj) File "/usr/lib/python2.7/site-packages/swiftclient/client.py", line 1333, in head_object return self._retry(None, head_object, container, obj) File "/usr/lib/python2.7/site-packages/swiftclient/client.py", line 1235, in _retry self.url, self.token = self.get_auth() File "/usr/lib/python2.7/site-packages/swiftclient/client.py", line 1209, in get_auth insecure=self.insecure) File "/usr/lib/python2.7/site-packages/swiftclient/client.py", line 406, in get_auth auth_version=auth_version) File "/usr/lib/python2.7/site-packages/swiftclient/client.py", line 339, in get_auth_keystone raise ClientException(msg) ClientException: Unauthorized. Check username, password and tenant name/id. 2015-02-17 16:49:37.103 77137 INFO glance.wsgi.server [08ee6676-515e-4089-87cb-7b1d23e72723 828ad19213794cd290b541d736f8fb6f 1343b6debee64b14966bf6742425aa21 - - -] 10.35.117.50 - - [17/Feb/2015 16:49:37] "DELETE /v1/images/d9dc2553-7d10-4b9b-a34d-3d60c4fcbe76 HTTP/1.1" 500 139 0.200249
@Prashanth This bug is referring to something quite different from what you've tested. Your tests are based on keystone users credentials, whereas the bug refers to the *store* credentials. In your test, you're correctly getting an error when you changed the tenant credentials. I say correctly because you're trying to access an image with a different user, AFAICT, which may not be the admin anymore. What this bug fixes is an issue where the store credentials were kept in the database and therefore, whenever the store credentials were changed, it was not possible to access the image data anymore.
Flavio, Could you let me know how to change the store credentials, so that I can verify this bug? As for my notes, I dont think it is a correct behavior. It is still the same "admin" user, but only a different password this time. The "admin" user still has admin privileges, so he should be able to delete any image. I will open a new bug for this.
@Prasanth, May I see your config file? Specifically the swift section and the swift config file. If you changed the password of one of the users listed in the swift config file, you'll need to update the file as well. Did you do this?
Flavio, What is the reference name mentioned in "[ref1]", '[ref2] etc. Is this something I need to configure in glance-api.conf? Also, is the auth_address same as "swift_store_auth_address" in glance-api.conf? Thanks Prasanth
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHSA-2015-0644.html