Bug 1319841 - openstack-ironic-conductor service failing silently with bad config
Summary: openstack-ironic-conductor service failing silently with bad config
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-ironic
Version: 7.0 (Kilo)
Hardware: Unspecified
OS: Unspecified
unspecified
low
Target Milestone: Upstream M3
: 10.0 (Newton)
Assignee: Lucas Alvares Gomes
QA Contact: Raviv Bar-Tal
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-03-21 16:05 UTC by Harry Rybacki
Modified: 2016-12-14 15:29 UTC (History)
6 users (show)

Fixed In Version: openstack-ironic-6.1.1-0.20160823055226.46dc395.el7ost
Doc Type: Bug Fix
Doc Text:
The openstack-ironic-conductor service now checks whether all drivers specified in the 'enabled_drivers' option are unique. The service then removes duplicated entries and logs a warning. In previous releases, duplicate entries in the 'enabled_drivers' option simply caused the openstack-ironic-conductor service to fail, thereby preventing the Bare Metal provisioning service from loading any drivers.
Clone Of:
Environment:
Last Closed: 2016-12-14 15:29:00 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Launchpad 1561564 0 None None None 2016-03-24 15:04:55 UTC
OpenStack gerrit 297194 0 None MERGED Fix: Duplicated driver causes conductor to fail 2020-12-04 14:24:15 UTC
Red Hat Product Errata RHEA-2016:2948 0 normal SHIPPED_LIVE Red Hat OpenStack Platform 10 enhancement update 2016-12-14 19:55:27 UTC

Description Harry Rybacki 2016-03-21 16:05:45 UTC
Description of problem:

When restarting the openstack-ironic-conductor service after modifying the `enabled_drivers` section of the ironic.conf file it failed silently. Adding a duplicate entry, pxe_ssh in this case, resulted in no drivers being enabled after the service restarted. Nothing was reported to stderr but was obvious after checking `ironic driver-list` and noting that there were no drivers enabled  

Version-Release number of selected component (if applicable):


How reproducible:
Easily

Steps to Reproduce:
1. from an existing OSPd-7 deployment add a duplicate entry to `enabled_drivers` within /etc/ironic/ironic.conf
2. sudo systemctl restart openstack-ironic-conductor
3. ironic driver-list

Actual results:
+---------------------+----------------+
| Supported driver(s) | Active host(s) |
+---------------------+----------------+
+---------------------+----------------+


Expected results:

Either something informative is pushed out to stderr when the malformed config is parsed indicating that is the case or, modify the config parser to handle and disregard duplicate entries. 

Additional info:

### /var/log/ironic/ironic-conductor.log ###

2016-03-21 11:38:48.822 774 DEBUG ironic.cmd.conductor [-] Configuration: main /usr/lib/python2.7/site-packages/ironic/cmd/conductor.py:43
2016-03-21 11:38:48.823 774 DEBUG ironic.cmd.conductor [-] ******************************************************************************** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2216
2016-03-21 11:38:48.823 774 DEBUG ironic.cmd.conductor [-] Configuration options gathered from: log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2217
2016-03-21 11:38:48.823 774 DEBUG ironic.cmd.conductor [-] command line args: [] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2218
2016-03-21 11:38:48.824 774 DEBUG ironic.cmd.conductor [-] config files: ['/usr/share/ironic/ironic-dist.conf', '/etc/ironic/ironic.conf'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2219
2016-03-21 11:38:48.824 774 DEBUG ironic.cmd.conductor [-] ================================================================================ log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2220
2016-03-21 11:38:48.824 774 DEBUG ironic.cmd.conductor [-] bindir                         = /usr/lib/python2.7/site-packages/ironic/bin log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2229
2016-03-21 11:38:48.824 774 DEBUG ironic.cmd.conductor [-] config_dir                     = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2229
2016-03-21 11:38:48.825 774 DEBUG ironic.cmd.conductor [-] config_file                    = ['/usr/share/ironic/ironic-dist.conf', '/etc/ironic/ironic.conf'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2229
2016-03-21 11:38:48.825 774 DEBUG ironic.cmd.conductor [-] control_exchange               = openstack log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2229
2016-03-21 11:38:48.825 774 DEBUG ironic.cmd.conductor [-] debug                          = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2229
2016-03-21 11:38:48.825 774 DEBUG ironic.cmd.conductor [-] default_log_levels             = ['amqp=WARN', 'amqplib=WARN', 'qpid.messaging=INFO', 'oslo.messaging=INFO', 'sqlalchemy=WARN', 'keystoneclient=INFO', 'stevedore=INFO', 'eventlet.wsgi.server=WARN', 'iso8601=WARN', 'paramiko=WARN', 'requests=WARN', 'neutronclient=WARN', 'glanceclient=WARN', 'ironic.openstack.common=WARN', 'urllib3.connectionpool=WARN'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2229
2016-03-21 11:38:48.826 774 DEBUG ironic.cmd.conductor [-] enabled_drivers                = ['pxe_ipmitool', 'pxe_ssh', 'pxe_drac', 'pxe_ilo', 'pxe_wol', 'pxe_ssh'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2229

Comment 2 Lucas Alvares Gomes 2016-03-24 15:06:47 UTC
Thanks for filling this out, I've opened a bug upstream and submitted a fix for it. I can backport it once it's merged there.

(added links to the external trackers)

Comment 3 Dmitry Tantsur 2016-09-06 16:07:40 UTC
Merged in OSP 10 and I've proposed a backport to OSP 9.

Comment 5 Raviv Bar-Tal 2016-09-18 12:52:04 UTC
This bug is verified.
New bug was open with some more enabled_driver formatting problem causing conductor to fail without error message
 https://bugzilla.redhat.com/show_bug.cgi?id=1377103

Comment 8 errata-xmlrpc 2016-12-14 15:29:00 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHEA-2016-2948.html


Note You need to log in before you can comment on or make changes to this bug.