Bug 1295561 - tempest test fails, tempest.api.image.v2.test_images.BasicOperationsImagesTest.test_update_image
tempest test fails, tempest.api.image.v2.test_images.BasicOperationsImagesTes...
Status: CLOSED DUPLICATE of bug 1284845
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-glance (Show other bugs)
7.0 (Kilo)
Unspecified Unspecified
high Severity high
: y3
: 7.0 (Kilo)
Assigned To: Sergey Gotliv
nlevinki
: Automation, AutomationBlocker, ZStream
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2016-01-04 16:01 EST by wes hayutin
Modified: 2016-04-26 14:48 EDT (History)
10 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-02-05 08:44:48 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description wes hayutin 2016-01-04 16:01:47 EST
Description of problem:

Failed 8 times in the last 13 runs. Stability: 38 %
tempest.api.image.v2.test_images.BasicOperationsImagesTest.test_update_image

https://github.com/redhat-openstack/tempest/blob/master/tempest/api/image/v2/test_images.py#L112

Traceback (most recent call last):
  File "/home/stack/tempest/tempest/api/image/v2/test_images.py", line 112, in test_update_image
    self.client.store_image(image_id, image_file)
  File "/home/stack/tempest/tempest/services/image/v2/json/image_client.py", line 142, in store_image
    return service_client.ResponseBody(resp, body)
  File "/home/stack/tempest/tempest/common/service_client.py", line 51, in __init__
    self.update(body_data)
ValueError: dictionary update sequence element #0 has length 26; 2 is required
Traceback (most recent call last):
_StringException: Empty attachments:
  stderr
  stdout
Comment 5 James Slagle 2016-02-03 08:18:50 EST
this is the tempest test failing:

https://rhos-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/osp_director-rhos-7_director-puddle-rhel-7.2-templates-virthost-minimal_ha-single_nic_vlans-neutron-ml2-vxlan/lastCompletedBuild/testReport/tempest.api.image.v2.test_images/BasicOperationsImagesTest/test_update_image_id_f66891a7_a35c_41a8_b590_a065c2a1caa6_smoke_/

In the glance api log on the overcloud I see this traceback:

2016-01-30 19:57:44.461 17102 TRACE glance.common.wsgi Traceback (most recent call last):
2016-01-30 19:57:44.461 17102 TRACE glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance/common/wsgi.py", line 881, in __call__
2016-01-30 19:57:44.461 17102 TRACE glance.common.wsgi     request, **action_args)
2016-01-30 19:57:44.461 17102 TRACE glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance/common/wsgi.py", line 909, in dispatch
2016-01-30 19:57:44.461 17102 TRACE glance.common.wsgi     return method(*args, **kwargs)
2016-01-30 19:57:44.461 17102 TRACE glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance/common/utils.py", line 508, in wrapped
2016-01-30 19:57:44.461 17102 TRACE glance.common.wsgi     return func(self, req, *args, **kwargs)
2016-01-30 19:57:44.461 17102 TRACE glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance/api/v2/image_data.py", line 178, in upload
2016-01-30 19:57:44.461 17102 TRACE glance.common.wsgi     self._restore(image_repo, image)
2016-01-30 19:57:44.461 17102 TRACE glance.common.wsgi   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 85, in __exit__
2016-01-30 19:57:44.461 17102 TRACE glance.common.wsgi     six.reraise(self.type_, self.value, self.tb)
2016-01-30 19:57:44.461 17102 TRACE glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance/api/v2/image_data.py", line 74, in upload
2016-01-30 19:57:44.461 17102 TRACE glance.common.wsgi     image.set_data(data, size)
2016-01-30 19:57:44.461 17102 TRACE glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance/domain/proxy.py", line 166, in set_data
2016-01-30 19:57:44.461 17102 TRACE glance.common.wsgi     self.base.set_data(data, size)
2016-01-30 19:57:44.461 17102 TRACE glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance/notifier.py", line 429, in set_data
2016-01-30 19:57:44.461 17102 TRACE glance.common.wsgi     _send_notification(notify_error, 'image.upload', msg)
2016-01-30 19:57:44.461 17102 TRACE glance.common.wsgi   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 85, in __exit__
2016-01-30 19:57:44.461 17102 TRACE glance.common.wsgi     six.reraise(self.type_, self.value, self.tb)
2016-01-30 19:57:44.461 17102 TRACE glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance/notifier.py", line 378, in set_data
2016-01-30 19:57:44.461 17102 TRACE glance.common.wsgi     self.repo.set_data(data, size)
2016-01-30 19:57:44.461 17102 TRACE glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance/api/policy.py", line 196, in set_data
2016-01-30 19:57:44.461 17102 TRACE glance.common.wsgi     return self.image.set_data(*args, **kwargs)
2016-01-30 19:57:44.461 17102 TRACE glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance/quota/__init__.py", line 296, in set_data
2016-01-30 19:57:44.461 17102 TRACE glance.common.wsgi     self.image.set_data(data, size=size)
2016-01-30 19:57:44.461 17102 TRACE glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance/location.py", line 377, in set_data
2016-01-30 19:57:44.461 17102 TRACE glance.common.wsgi     context=self.context)
2016-01-30 19:57:44.461 17102 TRACE glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance_store/backend.py", line 364, in add_to_backend
2016-01-30 19:57:44.461 17102 TRACE glance.common.wsgi     return store_add_to_backend(image_id, data, size, store, context)
2016-01-30 19:57:44.461 17102 TRACE glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance_store/backend.py", line 339, in store_add_to_backend
2016-01-30 19:57:44.461 17102 TRACE glance.common.wsgi     context=context)
2016-01-30 19:57:44.461 17102 TRACE glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance_store/capabilities.py", line 226, in op_checker
2016-01-30 19:57:44.461 17102 TRACE glance.common.wsgi     return store_op_fun(store, *args, **kwargs)
2016-01-30 19:57:44.461 17102 TRACE glance.common.wsgi   File "/usr/lib/python2.7/site-packages/glance_store/_drivers/swift/store.py", line 607, in add
2016-01-30 19:57:44.461 17102 TRACE glance.common.wsgi     raise exceptions.Duplicate(message=msg)
2016-01-30 19:57:44.461 17102 TRACE glance.common.wsgi Duplicate: Swift already has an image at this location
2016-01-30 19:57:44.461 17102 TRACE glance.common.wsgi
Comment 6 James Slagle 2016-02-03 08:22:13 EST
there is also this traceback in glance-api.log:

2016-01-30 19:57:44.314 17102 WARNING requests.packages.urllib3.connectionpool [req-847d259b-41a1-404f-84a5-40399d9429bb 09ec28e5b7a04e239d1eb15d6281dc90 8322d041a3684cab902e335bee747625 - - -] Connection pool is full, discarding connection: 172.16.21.10
2016-01-30 19:57:44.384 17102 ERROR swiftclient [req-847d259b-41a1-404f-84a5-40399d9429bb 09ec28e5b7a04e239d1eb15d6281dc90 8322d041a3684cab902e335bee747625 - - -] Object DELETE failed: http://172.16.21.10:8080/v1/AUTH_8fb4858ca5f4486893cd1f723865e45e/glance/d6e238b2-1c06-4d18-be0c-6ab1ca8ca3b9-00002 409 Conflict  [first 60 chars of response] <html><h1>Conflict</h1><p>There was a conflict when trying t
2016-01-30 19:57:44.384 17102 TRACE swiftclient Traceback (most recent call last):
2016-01-30 19:57:44.384 17102 TRACE swiftclient   File "/usr/lib/python2.7/site-packages/swiftclient/client.py", line 1261, in _retry
2016-01-30 19:57:44.384 17102 TRACE swiftclient     rv = func(self.url, self.token, *args, **kwargs)
2016-01-30 19:57:44.384 17102 TRACE swiftclient   File "/usr/lib/python2.7/site-packages/swiftclient/client.py", line 1124, in delete_object
2016-01-30 19:57:44.384 17102 TRACE swiftclient     http_response_content=body)
2016-01-30 19:57:44.384 17102 TRACE swiftclient ClientException: Object DELETE failed: http://172.16.21.10:8080/v1/AUTH_8fb4858ca5f4486893cd1f723865e45e/glance/d6e238b2-1c06-4d18-be0c-6ab1ca8ca3b9-00002 409 Conflict  [first 60 chars of response] <html><h1>Conflict</h1><p>There was a conflict when trying t
2016-01-30 19:57:44.384 17102 TRACE swiftclient 

so the image delete failed, and the new one can't be created
Comment 7 Emilien Macchi 2016-02-03 09:34:31 EST
So Mirantis Fuel has the same bug:
https://bugs.launchpad.net/fuel/+bug/1517013/comments/7

It looks like it has been fixed in Glance:
https://review.openstack.org/#/c/254873

We might need this patch in glance-store/kilo release.

Reassigning the bug.
Comment 8 Flavio Percoco 2016-02-05 08:44:48 EST
This is a duplicate of rhbz#1284845

Going to close it in favor of that one, which has a backported patch already. Feel free to re-open it if you want to track it elsewhere.

(P.S: pls, don't re-assign glance bugs to other folks)

*** This bug has been marked as a duplicate of bug 1284845 ***

Note You need to log in before you can comment on or make changes to this bug.