Bug 1394470 - keystone does not retry on deadlock Transactions [500 Error]
Summary: keystone does not retry on deadlock Transactions [500 Error]
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-keystone
Version: 10.0 (Newton)
Hardware: Unspecified
OS: Unspecified
Target Milestone: Upstream M2
: 11.0 (Ocata)
Assignee: Adam Young
QA Contact: nlevinki
Depends On:
TreeView+ depends on / blocked
Reported: 2016-11-12 11:52 UTC by Attila Fazekas
Modified: 2017-05-17 19:46 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Last Closed: 2017-05-17 19:46:13 UTC
Target Upstream Version:

Attachments (Terms of Use)
keystone_exception.txt (16.43 KB, text/plain)
2016-11-12 11:52 UTC, Attila Fazekas
no flags Details

System ID Private Priority Status Summary Last Updated
Launchpad 1604862 0 None None None 2016-12-08 17:34:21 UTC
Red Hat Product Errata RHEA-2017:1245 0 normal SHIPPED_LIVE Red Hat OpenStack Platform 11.0 Bug Fix and Enhancement Advisory 2017-05-17 23:01:50 UTC

Description Attila Fazekas 2016-11-12 11:52:42 UTC
Created attachment 1219990 [details]

Description of problem:
DBDeadlock: (pymysql.err.InternalError) (1213, u'Deadlock found when trying to get lock; try restarting transaction')

The above error is retry-able error, but no evidence for keystone would really did a retry before throwing a 500.

Version-Release number of selected component (if applicable):
python-keystone.noarch              1:10.0.0-3.el7ost  @rhos-10.0-puddle        
python-keystoneauth1.noarch         2.12.2-1.el7ost    @rhos-10.0-puddle        
python-keystoneclient.noarch        1:3.5.0-1.el7ost   @rhos-10.0-puddle        
python-keystonemiddleware.noarch    4.9.0-1.el7ost     @rhos-10.0-puddle  

How reproducible:

Comment 1 Adam Young 2016-12-08 16:52:39 UTC
How can I reproduce this?

Comment 2 Adam Young 2016-12-08 16:52:50 UTC
How can I reproduce this?

Comment 3 Adam Young 2016-12-08 17:34:21 UTC
Looks like it was fixed upstream.  Changing the external bug to link to the older discussions

Comment 4 Adam Young 2016-12-08 17:38:52 UTC
Should be fixed in all versions of OSP 11.

Comment 7 Rodrigo Duarte 2017-02-13 19:25:18 UTC
verified for openstack-keystone-11.0.0-0.20170127043446.cefbc3c.el7ost.noarch

When using a MariaDB Galera cluster, the error happens due a race condition [1]. To try to reproduce the error, I've run a big concurrency rally scenario test that would create and delete several roles and projects - this triggers several revocation events. No error was reported or found in keystone logs.

Marking as verified due the lack of evidence the issue persists rather than an actual validation.

[1] http://lists.openstack.org/pipermail/openstack-dev/2015-February/056007.html

Comment 8 errata-xmlrpc 2017-05-17 19:46:13 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.