Bug 1556720

Summary:	[OSP12] gnocchi-upgrade fails with InternalError: (1050, u"Table 'archive_policy' already exists")
Product:	Red Hat OpenStack	Reporter:	Michele Baldessari <michele>
Component:	openstack-tripleo-heat-templates	Assignee:	Michele Baldessari <michele>
Status:	CLOSED ERRATA	QA Contact:	Sasha Smolyak <ssmolyak>
Severity:	high	Docs Contact:
Priority:	high
Version:	12.0 (Pike)	CC:	aschultz, jamsmith, mburns, michele, rhel-osp-director-maint, rscarazz
Target Milestone:	z3	Keywords:	Triaged, ZStream
Target Release:	12.0 (Pike)
Hardware:	All
OS:	Linux
Whiteboard:
Fixed In Version:	openstack-tripleo-heat-templates-7.0.9-10.el7ost	Doc Type:	Bug Fix
Doc Text:	To prevent failures caused by a gnocchi-upgrade race condition, gnocchi-upgrade is now called from the bootstrap node instead of from multiple nodes. Previously, gnocchi-upgrade was called from each node where gnocchi-api is part of the role. This sometimes resulted in failures with the error shown in the following example: 2018-03-14 12:39:39,683 [1] ERROR oslo_db.sqlalchemy.exc_filters: DBAPIError exception wrapped from (pymysql.err.InternalError) (1050, u"Table 'archive_policy' already exists")	Story Points:	---
Clone Of:
Clones:	1556721 (view as bug list)		Environment:
Last Closed:	2018-08-20 12:59:02 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Michele Baldessari 2018-03-15 06:46:52 UTC

Description of problem:
Rasca has been observing the following from time to time on his OSP12 setups:

2018-03-14 12:39:39,683 [1] ERROR oslo_db.sqlalchemy.exc_filters: DBAPIError exception wrapped from (pymysql.err.InternalError) (1050, u"Table 'archive_policy' already exists") [SQL: u'\nCREATE TABLE archive_policy (\n\tname VARCHAR(255) NOT NULL, \n\tback_window INTEGER NOT NULL, \n\tdefinition TEXT NOT NULL, \n\taggregation_methods TEXT NOT NULL, \n\tPRIMARY KEY (name)\n)ENGINE=InnoDB CHARSET=utf8\n\n'] Traceback (most recent call last): File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1182, in _execute_context context) File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/default.py", line 470, in do_execute cursor.execute(statement, parameters) File "/usr/lib/python2.7/site-packages/pymysql/cursors.py", line 166, in execute result = self._query(query) File "/usr/lib/python2.7/site-packages/pymysql/cursors.py", line 322, in _query conn.query(q) File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 856, in query self._affected_rows = self._read_query_result(unbuffered=unbuffered) File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 1057, in _read_query_result result.read() File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 1340, in read first_packet = self.connection._read_packet() File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 1014, in _read_packet packet.check_error() File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 393, in check_error err.raise_mysql_exception(self._data) File "/usr/lib/python2.7/site-packages/pymysql/err.py", line 107, in raise_mysql_exception raise errorclass(errno, errval) InternalError: (1050, u"Table 'archive_policy' already exists") 


This seems a race around gnocchi-upgrade

Comment 1 Raoul Scarazzini 2018-03-15 10:21:25 UTC

I verified the patch [1] on the same osp12 environment in which it was failing and it's working.

[1] https://review.openstack.org/#/c/553028/1

Comment 2 Michele Baldessari 2018-03-15 13:33:24 UTC

Linked the proper pike/review

Comment 8 Sasha Smolyak 2018-08-01 11:45:46 UTC

The fixed-in-version is present, the fail doesn't happen anymore. Fixed

Comment 10 errata-xmlrpc 2018-08-20 12:59:02 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2018:2331