Bug 1703019 - Scale out failure cause with "Data too long for column 'instance_info'"
Summary: Scale out failure cause with "Data too long for column 'instance_info'"
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-ironic
Version: 10.0 (Newton)
Hardware: x86_64
OS: Linux
urgent
urgent
Target Milestone: ---
: ---
Assignee: RHOS Maint
QA Contact: mlammon
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-04-25 10:11 UTC by Khomesh Thakre
Modified: 2019-05-17 11:50 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-05-17 11:50:36 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Bugzilla 1433088 high CLOSED Stop storing configdrive in the database, as it can cause failures with "Data too long for column 'instance_info'" 2020-10-14 00:28:05 UTC

Description Khomesh Thakre 2019-04-25 10:11:01 UTC
Description of problem:
While performing a scale-out operation it failed with below traceback

~~~
2019-04-24 17:53:22.735 1770 ERROR oslo_db.sqlalchemy.exc_filters [req-a1a3c0bc-7017-45e5-bdc7-3eddf79789a6 - - - - -] DBAPIError exception wrapped from (pymysql.err.DataError) (1406, u"Data too long for column 'instance_info' at row 1") [SQL: u'UPDATE nodes SET updated_at=%(updated_at)s, instance_info=%(instance_info)s WHERE nodes.id = %(nodes_id)s'] [parameters: {'nodes_id': 199, 'updated_at': datetime.datetime(2019, 4, 24, 12, 23, 22, 723638),
.
.
.
2019-04-24 17:53:22.735 1770 ERROR oslo_db.sqlalchemy.exc_filters Traceback (most recent call last):
2019-04-24 17:53:22.735 1770 ERROR oslo_db.sqlalchemy.exc_filters   File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context
2019-04-24 17:53:22.735 1770 ERROR oslo_db.sqlalchemy.exc_filters     context)
2019-04-24 17:53:22.735 1770 ERROR oslo_db.sqlalchemy.exc_filters   File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/default.py", line 450, in do_execute
2019-04-24 17:53:22.735 1770 ERROR oslo_db.sqlalchemy.exc_filters     cursor.execute(statement, parameters)
2019-04-24 17:53:22.735 1770 ERROR oslo_db.sqlalchemy.exc_filters   File "/usr/lib/python2.7/site-packages/pymysql/cursors.py", line 166, in execute
2019-04-24 17:53:22.735 1770 ERROR oslo_db.sqlalchemy.exc_filters     result = self._query(query)
2019-04-24 17:53:22.735 1770 ERROR oslo_db.sqlalchemy.exc_filters   File "/usr/lib/python2.7/site-packages/pymysql/cursors.py", line 322, in _query
2019-04-24 17:53:22.735 1770 ERROR oslo_db.sqlalchemy.exc_filters     conn.query(q)
2019-04-24 17:53:22.735 1770 ERROR oslo_db.sqlalchemy.exc_filters   File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 841, in query
2019-04-24 17:53:22.735 1770 ERROR oslo_db.sqlalchemy.exc_filters     self._affected_rows = self._read_query_result(unbuffered=unbuffered)
2019-04-24 17:53:22.735 1770 ERROR oslo_db.sqlalchemy.exc_filters   File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 1029, in _read_query_result
2019-04-24 17:53:22.735 1770 ERROR oslo_db.sqlalchemy.exc_filters     result.read()
2019-04-24 17:53:22.735 1770 ERROR oslo_db.sqlalchemy.exc_filters   File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 1312, in read
2019-04-24 17:53:22.735 1770 ERROR oslo_db.sqlalchemy.exc_filters     first_packet = self.connection._read_packet()
2019-04-24 17:53:22.735 1770 ERROR oslo_db.sqlalchemy.exc_filters   File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 991, in _read_packet
2019-04-24 17:53:22.735 1770 ERROR oslo_db.sqlalchemy.exc_filters     packet.check_error()
2019-04-24 17:53:22.735 1770 ERROR oslo_db.sqlalchemy.exc_filters   File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 393, in check_error
2019-04-24 17:53:22.735 1770 ERROR oslo_db.sqlalchemy.exc_filters     err.raise_mysql_exception(self._data)
2019-04-24 17:53:22.735 1770 ERROR oslo_db.sqlalchemy.exc_filters   File "/usr/lib/python2.7/site-packages/pymysql/err.py", line 107, in raise_mysql_exception
2019-04-24 17:53:22.735 1770 ERROR oslo_db.sqlalchemy.exc_filters     raise errorclass(errno, errval)
2019-04-24 17:53:22.735 1770 ERROR oslo_db.sqlalchemy.exc_filters DataError: (1406, u"Data too long for column 'instance_info' at row 1")
2019-04-24 17:53:22.735 1770 ERROR oslo_db.sqlalchemy.exc_filters
~~~

Version-Release number of selected component (if applicable):
Red Hat OpenStack Platform 10 Newton

Comment 3 Bob Fournier 2019-05-07 19:26:34 UTC
From the case it looks like this is no longer an issue.

Because of the database configuration its not possible to backport the fix for 
https://bugzilla.redhat.com/show_bug.cgi?id=1433088 to OSP-10.  The recommended fix is to update the database as described in https://bugzilla.redhat.com/show_bug.cgi?id=1433088#c32.  This has been well tested at multiple customer sites.

As this is no longer a problem with customer recommend closing this bug.

Comment 5 Bob Fournier 2019-05-17 11:50:36 UTC
Closing based on workaround to update database field.


Note You need to log in before you can comment on or make changes to this bug.