Bug 1703019 - Scale out failure cause with "Data too long for column 'instance_info'"
Summary: Scale out failure cause with "Data too long for column 'instance_info'"
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-ironic
Version: 10.0 (Newton)
Hardware: x86_64
OS: Linux
urgent
urgent
Target Milestone: ---
: ---
Assignee: RHOS Maint
QA Contact: mlammon
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-04-25 10:11 UTC by Khomesh Thakre
Modified: 2019-05-17 11:50 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-05-17 11:50:36 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 1433088 0 high CLOSED Stop storing configdrive in the database, as it can cause failures with "Data too long for column 'instance_info'" 2022-08-09 14:21:03 UTC

Description Khomesh Thakre 2019-04-25 10:11:01 UTC 
Description of problem:
While performing a scale-out operation it failed with below traceback

~~~
2019-04-24 17:53:22.735 1770 ERROR oslo_db.sqlalchemy.exc_filters [req-a1a3c0bc-7017-45e5-bdc7-3eddf79789a6 - - - - -] DBAPIError exception wrapped from (pymysql.err.DataError) (1406, u"Data too long for column 'instance_info' at row 1") [SQL: u'UPDATE nodes SET updated_at=%(updated_at)s, instance_info=%(instance_info)s WHERE nodes.id = %(nodes_id)s'] [parameters: {'nodes_id': 199, 'updated_at': datetime.datetime(2019, 4, 24, 12, 23, 22, 723638),
.
.
.
2019-04-24 17:53:22.735 1770 ERROR oslo_db.sqlalchemy.exc_filters Traceback (most recent call last):
2019-04-24 17:53:22.735 1770 ERROR oslo_db.sqlalchemy.exc_filters   File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context
2019-04-24 17:53:22.735 1770 ERROR oslo_db.sqlalchemy.exc_filters     context)
2019-04-24 17:53:22.735 1770 ERROR oslo_db.sqlalchemy.exc_filters   File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/default.py", line 450, in do_execute
2019-04-24 17:53:22.735 1770 ERROR oslo_db.sqlalchemy.exc_filters     cursor.execute(statement, parameters)
2019-04-24 17:53:22.735 1770 ERROR oslo_db.sqlalchemy.exc_filters   File "/usr/lib/python2.7/site-packages/pymysql/cursors.py", line 166, in execute
2019-04-24 17:53:22.735 1770 ERROR oslo_db.sqlalchemy.exc_filters     result = self._query(query)
2019-04-24 17:53:22.735 1770 ERROR oslo_db.sqlalchemy.exc_filters   File "/usr/lib/python2.7/site-packages/pymysql/cursors.py", line 322, in _query
2019-04-24 17:53:22.735 1770 ERROR oslo_db.sqlalchemy.exc_filters     conn.query(q)
2019-04-24 17:53:22.735 1770 ERROR oslo_db.sqlalchemy.exc_filters   File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 841, in query
2019-04-24 17:53:22.735 1770 ERROR oslo_db.sqlalchemy.exc_filters     self._affected_rows = self._read_query_result(unbuffered=unbuffered)
2019-04-24 17:53:22.735 1770 ERROR oslo_db.sqlalchemy.exc_filters   File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 1029, in _read_query_result
2019-04-24 17:53:22.735 1770 ERROR oslo_db.sqlalchemy.exc_filters     result.read()
2019-04-24 17:53:22.735 1770 ERROR oslo_db.sqlalchemy.exc_filters   File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 1312, in read
2019-04-24 17:53:22.735 1770 ERROR oslo_db.sqlalchemy.exc_filters     first_packet = self.connection._read_packet()
2019-04-24 17:53:22.735 1770 ERROR oslo_db.sqlalchemy.exc_filters   File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 991, in _read_packet
2019-04-24 17:53:22.735 1770 ERROR oslo_db.sqlalchemy.exc_filters     packet.check_error()
2019-04-24 17:53:22.735 1770 ERROR oslo_db.sqlalchemy.exc_filters   File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 393, in check_error
2019-04-24 17:53:22.735 1770 ERROR oslo_db.sqlalchemy.exc_filters     err.raise_mysql_exception(self._data)
2019-04-24 17:53:22.735 1770 ERROR oslo_db.sqlalchemy.exc_filters   File "/usr/lib/python2.7/site-packages/pymysql/err.py", line 107, in raise_mysql_exception
2019-04-24 17:53:22.735 1770 ERROR oslo_db.sqlalchemy.exc_filters     raise errorclass(errno, errval)
2019-04-24 17:53:22.735 1770 ERROR oslo_db.sqlalchemy.exc_filters DataError: (1406, u"Data too long for column 'instance_info' at row 1")
2019-04-24 17:53:22.735 1770 ERROR oslo_db.sqlalchemy.exc_filters
~~~

Version-Release number of selected component (if applicable):
Red Hat OpenStack Platform 10 Newton
Comment 3 Bob Fournier 2019-05-07 19:26:34 UTC 
From the case it looks like this is no longer an issue.

Because of the database configuration its not possible to backport the fix for 
https://bugzilla.redhat.com/show_bug.cgi?id=1433088 to OSP-10.  The recommended fix is to update the database as described in https://bugzilla.redhat.com/show_bug.cgi?id=1433088#c32.  This has been well tested at multiple customer sites.

As this is no longer a problem with customer recommend closing this bug.
Comment 5 Bob Fournier 2019-05-17 11:50:36 UTC 
Closing based on workaround to update database field.
Note You need to log in before you can comment on or make changes to this bug.