Red Hat Bugzilla – Bug 1229296
ironic services did not start up after reboot
Last modified: 2015-08-05 09:52:50 EDT
Description of problem: installed poodle following https://repos.fedorapeople.org/repos/openstack-m/docs/internal/master/installation/installing.html then rebooted the instack VM and ironic services didn't start. while trying to discover node details got : $ instack-ironic-deployment --discover-nodes Preparing for deployment... Discovering nodes. Sending node ID 55d21860-66c6-4977-ad07-0e343dea2f5a to discoverd for discovery ... ERROR: openstack ('Connection aborted.', error(111, 'Connection refused')) when node registration worked: $ instack-ironic-deployment --nodes-json instackenv.json --register-nodes Preparing for deployment... Registering nodes from instackenv.json Nodes registered. +--------------------------------------+------+---------------+-------------+-----------------+-------------+ | UUID | Name | Instance UUID | Power State | Provision State | Maintenance | +--------------------------------------+------+---------------+-------------+-----------------+-------------+ | 55d21860-66c6-4977-ad07-0e343dea2f5a | None | None | power off | available | False | | 9ab3df2c-4a5c-4951-b24a-3eb42f116122 | None | None | power off | available | False | | be19bfdf-1422-4df9-bcc2-a2b875ae3b9e | None | None | power off | available | False | | 13c64e11-1e13-4390-bcaf-4f81ae043e7e | None | None | power off | available | False | +--------------------------------------+------+---------------+-------------+-----------------+-------------+ Version-Release number of selected component (if applicable): $ rpm -qa | grep openstack-ironic openstack-ironic-api-2015.1.0-4.el7ost.noarch openstack-ironic-conductor-2015.1.0-4.el7ost.noarch openstack-ironic-common-2015.1.0-4.el7ost.noarch openstack-ironic-discoverd-1.1.0-3.el7ost.noarch How reproducible: 100% Steps to Reproduce: 1.install the undercloud 2.reboot the VM 3. register nodes 4. discover nodes Actual results: ironic services stopped Expected results: ironic services are up and running. discovery process succeeded. Additional info: $ sudo systemctl status openstack-ironic-discoverd openstack-ironic-discoverd.service - Hardware introspection service for OpenStack Ironic Loaded: loaded (/usr/lib/systemd/system/openstack-ironic-discoverd.service; enabled) Active: failed (Result: exit-code) since Mon 2015-06-08 07:03:56 EDT; 50min ago Process: 490 ExecStart=/usr/bin/ironic-discoverd --config-file /etc/ironic-discoverd/discoverd.conf (code=exited, status=1/FAILURE) Main PID: 490 (code=exited, status=1/FAILURE) CGroup: /system.slice/openstack-ironic-discoverd.service Jun 08 07:03:56 instack.localdomain ironic-discoverd[490]: self.authenticate() Jun 08 07:03:56 instack.localdomain ironic-discoverd[490]: File "/usr/lib/python2.7/site-packages/keyst...er Jun 08 07:03:56 instack.localdomain ironic-discoverd[490]: return func(*args, **kwargs) Jun 08 07:03:56 instack.localdomain ironic-discoverd[490]: File "/usr/lib/python2.7/site-packages/keyst...te Jun 08 07:03:56 instack.localdomain ironic-discoverd[490]: resp = self.get_raw_token_from_identity_serv...s) Jun 08 07:03:56 instack.localdomain ironic-discoverd[490]: File "/usr/lib/python2.7/site-packages/keyst...ce Jun 08 07:03:56 instack.localdomain ironic-discoverd[490]: _("Authorization Failed: %s") % e) Jun 08 07:03:56 instack.localdomain ironic-discoverd[490]: keystoneclient.openstack.common.apiclient.ex...ns Jun 08 07:03:56 instack.localdomain systemd[1]: openstack-ironic-discoverd.service: main process exite...URE Jun 08 07:03:56 instack.localdomain systemd[1]: Unit openstack-ironic-discoverd.service entered failed...te. Hint: Some lines were ellipsized, use -l to show in full. [stack@instack ~]$ sudo systemctl status openstack-keystone openstack-keystone.service - OpenStack Identity Service (code-named Keystone) Loaded: loaded (/usr/lib/systemd/system/openstack-keystone.service; enabled) Active: active (running) since Mon 2015-06-08 07:03:56 EDT; 50min ago Main PID: 1075 (keystone-all) CGroup: /system.slice/openstack-keystone.service ├─1075 /usr/bin/python /usr/bin/keystone-all ├─1734 /usr/bin/python /usr/bin/keystone-all ├─1735 /usr/bin/python /usr/bin/keystone-all ├─1736 /usr/bin/python /usr/bin/keystone-all └─1737 /usr/bin/python /usr/bin/keystone-all Jun 08 07:03:37 instack.localdomain systemd[1]: Starting OpenStack Identity Service (code-named Keystone)... Jun 08 07:03:56 instack.localdomain systemd[1]: Started OpenStack Identity Service (code-named Keystone). Hint: Some lines were ellipsized, use -l to show in full. [stack@instack ~]$ sudo systemctl status openstack-ironic-api openstack-ironic-api.service - OpenStack Ironic API service Loaded: loaded (/usr/lib/systemd/system/openstack-ironic-api.service; enabled) Active: active (running) since Mon 2015-06-08 07:03:37 EDT; 50min ago Main PID: 1042 (ironic-api) CGroup: /system.slice/openstack-ironic-api.service └─1042 /usr/bin/python2 /usr/bin/ironic-api Jun 08 07:52:55 instack.localdomain ironic-api[1042]: 2015-06-08 07:52:55.046 1042 DEBUG keystonemiddl...296 Jun 08 07:52:55 instack.localdomain ironic-api[1042]: 2015-06-08 07:52:55.046 1042 DEBUG keystonemiddl...603 Jun 08 07:52:55 instack.localdomain ironic-api[1042]: 2015-06-08 07:52:55.046 1042 DEBUG keystonemiddlewa... Jun 08 07:52:55 instack.localdomain ironic-api[1042]: 192.0.2.1 - - [08/Jun/2015 07:52:55] "GET /v1/no...391 Jun 08 07:53:55 instack.localdomain ironic-api[1042]: 2015-06-08 07:53:55.045 1042 DEBUG keystonemiddl...X-S Jun 08 07:53:55 instack.localdomain ironic-api[1042]: 2015-06-08 07:53:55.045 1042 DEBUG keystonemiddl...581 Jun 08 07:53:55 instack.localdomain ironic-api[1042]: 2015-06-08 07:53:55.046 1042 DEBUG keystonemiddl...296 Jun 08 07:53:55 instack.localdomain ironic-api[1042]: 2015-06-08 07:53:55.046 1042 DEBUG keystonemiddl...603 Jun 08 07:53:55 instack.localdomain ironic-api[1042]: 2015-06-08 07:53:55.046 1042 DEBUG keystonemiddlewa... Jun 08 07:53:55 instack.localdomain ironic-api[1042]: 192.0.2.1 - - [08/Jun/2015 07:53:55] "GET /v1/no...391 Hint: Some lines were ellipsized, use -l to show in full. the output of journalctl -b -u openstack-ironic-discoverd -u openstack-keystone -u openstack-ironic-api is attached
Created attachment 1036348 [details] journal.log
instack-undercloud patch to increase timeout: https://review.gerrithub.io/#/c/235907/
Verified in puddle 2015-06-26: openstack-ironic-discoverd-1.1.0-4.el7ost.noarch
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2015:1549