Bug 1328237

Summary: rm /var/lib/ironic-inspector/inspector.sqlite; systemctl restart openstack-ironic-inspector fails to recreate sqlite db schema
Product: Red Hat OpenStack Reporter: Dan Yocum <dyocum>
Component: documentationAssignee: RHOS Documentation Team <rhos-docs>
Status: CLOSED WONTFIX QA Contact: RHOS Documentation Team <rhos-docs>
Severity: high Docs Contact:
Priority: unspecified    
Version: 8.0 (Liberty)CC: bfournie, djuran, dmacpher, dtantsur, mburns, slinaber, srevivo
Target Milestone: ---Keywords: Documentation, ZStream
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-07-11 20:34:22 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Dan Yocum 2016-04-18 19:21:43 UTC 
Description of problem:

Per the GA Director Installation and Configuration guide, executing section 10.3 - https://access.redhat.com/documentation/en/red-hat-openstack-platform/8/director-installation-and-usage/102-troubleshooting-hardware-introspection - fails to re-create the sqlite database.

Version-Release number of selected component (if applicable):


How reproducible:

100%

Steps to Reproduce:
1. Follow procedure 10.3 in the guide:

Procedure 10.3. Stopping the Discovery Process

Change the power state of each node to off:
$ ironic node-set-power-state [NODE UUID] off

Remove ironic-inspector cache and restart it:
$ rm /var/lib/ironic-inspector/inspector.sqlite
$ sudo systemctl restart openstack-ironic-inspector


Actual results:

[17301]: 2016-04-18 16:14:21.815 17301 DEBUG ironic_inspector.main [-] Running periodic update of filters periodic_update /usr/lib/python2.7/site-packages/ironic_inspector/main.py:270
[17301]: 2016-04-18 16:14:21.815 17301 DEBUG keystoneclient.auth.identity.v2 [-] Making authentication request to http://10.3.3.1:5000/v2.0/tokens get_auth_ref /usr/lib/python2.7/site-packages/keystoneclient/auth/identity/v2.py:86
[17301]: 2016-04-18 16:14:21.946 17301 DEBUG iso8601.iso8601 [-] Parsed 2016-04-19T00:14:21Z into {'tz_sign': None, 'second_fraction': None, 'hour': u'00', 'daydash': u'19', 'tz_hour': None, 'month': None, 'timezone': u'Z', 'second': u'21', 'tz_minute': None, 'year': u'2016', 'separator': u'T', 'monthdash': u'04', 'day': None, 'minute': u'14'} with default timezone <iso8601.iso8601.Utc object at 0x2088ad0> parse_date /usr/lib/python2.7/site-packages/iso8601/iso8601.py:184
[17301]: 2016-04-18 16:14:21.946 17301 DEBUG iso8601.iso8601 [-] Got u'2016' for 'year' with default None to_int /usr/lib/python2.7/site-packages/iso8601/iso8601.py:140
[17301]: 2016-04-18 16:14:21.947 17301 DEBUG iso8601.iso8601 [-] Got u'04' for 'monthdash' with default 1 to_int /usr/lib/python2.7/site-packages/iso8601/iso8601.py:140
[17301]: 2016-04-18 16:14:21.947 17301 DEBUG iso8601.iso8601 [-] Got 4 for 'month' with default 4 to_int /usr/lib/python2.7/site-packages/iso8601/iso8601.py:140
[17301]: 2016-04-18 16:14:21.947 17301 DEBUG iso8601.iso8601 [-] Got u'19' for 'daydash' with default 1 to_int /usr/lib/python2.7/site-packages/iso8601/iso8601.py:140
[17301]: 2016-04-18 16:14:21.947 17301 DEBUG iso8601.iso8601 [-] Got 19 for 'day' with default 19 to_int /usr/lib/python2.7/site-packages/iso8601/iso8601.py:140
[17301]: 2016-04-18 16:14:21.947 17301 DEBUG iso8601.iso8601 [-] Got u'00' for 'hour' with default None to_int /usr/lib/python2.7/site-packages/iso8601/iso8601.py:140
[17301]: 2016-04-18 16:14:21.947 17301 DEBUG iso8601.iso8601 [-] Got u'14' for 'minute' with default None to_int /usr/lib/python2.7/site-packages/iso8601/iso8601.py:140
[17301]: 2016-04-18 16:14:21.947 17301 DEBUG iso8601.iso8601 [-] Got u'21' for 'second' with default None to_int /usr/lib/python2.7/site-packages/iso8601/iso8601.py:140
[17301]: 2016-04-18 16:14:21.950 17301 ERROR ironic_inspector.main [-] Periodic update failed
[17301]: 2016-04-18 16:14:21.950 17301 ERROR ironic_inspector.main Traceback (most recent call last):
[17301]: 2016-04-18 16:14:21.950 17301 ERROR ironic_inspector.main   File "/usr/lib/python2.7/site-packages/ironic_inspector/main.py", line 272, in periodic_update
[17301]: 2016-04-18 16:14:21.950 17301 ERROR ironic_inspector.main     firewall.update_filters()
[17301]: 2016-04-18 16:14:21.950 17301 ERROR ironic_inspector.main   File "/usr/lib/python2.7/site-packages/ironic_inspector/firewall.py", line 179, in update_filters
[17301]: 2016-04-18 16:14:21.950 17301 ERROR ironic_inspector.main     if not _should_enable_dhcp():
[17301]: 2016-04-18 16:14:21.950 17301 ERROR ironic_inspector.main   File "/usr/lib/python2.7/site-packages/ironic_inspector/firewall.py", line 111, in _should_enable_dhcp
[17301]: 2016-04-18 16:14:21.950 17301 ERROR ironic_inspector.main     return (node_cache.introspection_active() or
[17301]: 2016-04-18 16:14:21.950 17301 ERROR ironic_inspector.main   File "/usr/lib/python2.7/site-packages/ironic_inspector/node_cache.py", line 351, in introspection_active
[17301]: 2016-04-18 16:14:21.950 17301 ERROR ironic_inspector.main     return (db.model_query(db.Node.uuid).filter_by(finished_at=None).first()
[17301]: 2016-04-18 16:14:21.950 17301 ERROR ironic_inspector.main   File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2445, in first
[17301]: 2016-04-18 16:14:21.950 17301 ERROR ironic_inspector.main     ret = list(self[0:1])
[17301]: 2016-04-18 16:14:21.950 17301 ERROR ironic_inspector.main   File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2281, in __getitem__
[17301]: 2016-04-18 16:14:21.950 17301 ERROR ironic_inspector.main     return list(res)
[17301]: 2016-04-18 16:14:21.950 17301 ERROR ironic_inspector.main   File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2516, in __iter__
[17301]: 2016-04-18 16:14:21.950 17301 ERROR ironic_inspector.main     return self._execute_and_instances(context)
[17301]: 2016-04-18 16:14:21.950 17301 ERROR ironic_inspector.main   File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2531, in _execute_and_instances
[17301]: 2016-04-18 16:14:21.950 17301 ERROR ironic_inspector.main     result = conn.execute(querycontext.statement, self._params)
[17301]: 2016-04-18 16:14:21.950 17301 ERROR ironic_inspector.main   File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 914, in execute
[17301]: 2016-04-18 16:14:21.950 17301 ERROR ironic_inspector.main     return meth(self, multiparams, params)
[17301]: 2016-04-18 16:14:21.950 17301 ERROR ironic_inspector.main   File "/usr/lib64/python2.7/site-packages/sqlalchemy/sql/elements.py", line 323, in _execute_on_connection
[17301]: 2016-04-18 16:14:21.950 17301 ERROR ironic_inspector.main     return connection._execute_clauseelement(self, multiparams, params)
[17301]: 2016-04-18 16:14:21.950 17301 ERROR ironic_inspector.main   File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1010, in _execute_clauseelement
[17301]: 2016-04-18 16:14:21.950 17301 ERROR ironic_inspector.main     compiled_sql, distilled_params
[17301]: 2016-04-18 16:14:21.950 17301 ERROR ironic_inspector.main   File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1146, in _execute_context
[17301]: 2016-04-18 16:14:21.950 17301 ERROR ironic_inspector.main     context)
[17301]: 2016-04-18 16:14:21.950 17301 ERROR ironic_inspector.main   File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1337, in _handle_dbapi_exception
[17301]: 2016-04-18 16:14:21.950 17301 ERROR ironic_inspector.main     util.raise_from_cause(newraise, exc_info)
[17301]: 2016-04-18 16:14:21.950 17301 ERROR ironic_inspector.main   File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/compat.py", line 199, in raise_from_cause
[17301]: 2016-04-18 16:14:21.950 17301 ERROR ironic_inspector.main     reraise(type(exception), exception, tb=exc_tb)
[17301]: 2016-04-18 16:14:21.950 17301 ERROR ironic_inspector.main   File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context
[17301]: 2016-04-18 16:14:21.950 17301 ERROR ironic_inspector.main     context)
[17301]: 2016-04-18 16:14:21.950 17301 ERROR ironic_inspector.main   File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/default.py", line 450, in do_execute
[17301]: 2016-04-18 16:14:21.950 17301 ERROR ironic_inspector.main     cursor.execute(statement, parameters)
[17301]: 2016-04-18 16:14:21.950 17301 ERROR ironic_inspector.main OperationalError: (sqlite3.OperationalError) no such table: nodes [SQL: u'SELECT nodes.uuid AS nodes_uuid \nFROM nodes \nWHERE nodes.finished_at IS NULL\n LIMIT ? OFFSET ?'] [parameters: (1, 0)]
[17301]: 2016-04-18 16:14:21.950 17301 ERROR ironic_inspector.main


Expected results:

inspector.sqlite db is re-created.

Additional info:

This worked in v7.3
Comment 2 Dan Yocum 2016-04-18 23:45:55 UTC 
Changing this to documentation and adding dmacpher.

The sol'n is to run the following command, so it needs to be added to the introspection troubleshooting section, mentioned above:

sudo ironic-inspector-dbsync --config-file /etc/ironic-inspector/inspector.conf upgrade

I gleened this from http://docs.openstack.org/developer/ironic-inspector/install.html#managing-the-ironic-inspector-database
Comment 3 Dmitry Tantsur 2016-04-19 07:02:45 UTC 
Upstream documentation update: https://review.openstack.org/307596
Comment 4 David Juran 2016-04-27 15:02:04 UTC 
*** Bug 1330590 has been marked as a duplicate of this bug. ***
Comment 5 Dmitry Tantsur 2016-09-30 09:55:47 UTC 
Note: this procedure is for OSPd8 *only*. Please do not recommend removing the database for OSPd9 and newer, we should use the "openstack baremetal introspection abort" command instead.