HA | Neutron-server resource is being reported as failed. It seems like neutron-server service is being restarted on every puppet run. rhel-osp-installer-client-0.5.3-1.el7ost.noarch rhel-osp-installer-0.5.3-1.el7ost.noarch openstack-puppet-modules-2014.2.7-2.el7ost.noarch ruby193-rubygem-foreman_openstack_simplify-0.0.6-8.el7ost.noarch openstack-foreman-installer-3.0.5-1.el7ost.noarch server log shows the following: -------------------------------- 2014-12-16 14:56:53.444 75106 CRITICAL neutron [req-fc6a53d0-b85e-4b19-a56f-11c2a4961c2d None] AttributeError: 'MessageHandlingServer' object has no attribute 'kill' 2014-12-16 14:56:53.444 75106 TRACE neutron Traceback (most recent call last): 2014-12-16 14:56:53.444 75106 TRACE neutron File "/usr/bin/neutron-server", line 10, in <module> 2014-12-16 14:56:53.444 75106 TRACE neutron sys.exit(main()) 2014-12-16 14:56:53.444 75106 TRACE neutron File "/usr/lib/python2.7/site-packages/neutron/server/__init__.py", line 51, in main 2014-12-16 14:56:53.444 75106 TRACE neutron neutron_rpc = service.serve_rpc() 2014-12-16 14:56:53.444 75106 TRACE neutron File "/usr/lib/python2.7/site-packages/neutron/service.py", line 163, in serve_rpc 2014-12-16 14:56:53.444 75106 TRACE neutron LOG.exception(_('Unrecoverable error: please check log ' 2014-12-16 14:56:53.444 75106 TRACE neutron File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 82, in __exit__ 2014-12-16 14:56:53.444 75106 TRACE neutron six.reraise(self.type_, self.value, self.tb) 2014-12-16 14:56:53.444 75106 TRACE neutron File "/usr/lib/python2.7/site-packages/neutron/service.py", line 159, in serve_rpc 2014-12-16 14:56:53.444 75106 TRACE neutron launcher.launch_service(rpc, workers=cfg.CONF.rpc_workers) 2014-12-16 14:56:53.444 75106 TRACE neutron File "/usr/lib/python2.7/site-packages/neutron/openstack/common/service.py", line 341, in launch_service 2014-12-16 14:56:53.444 75106 TRACE neutron self._start_child(wrap) 2014-12-16 14:56:53.444 75106 TRACE neutron File "/usr/lib/python2.7/site-packages/neutron/openstack/common/service.py", line 322, in _start_child 2014-12-16 14:56:53.444 75106 TRACE neutron status, signo = self._child_wait_for_exit_or_signal(launcher) 2014-12-16 14:56:53.444 75106 TRACE neutron File "/usr/lib/python2.7/site-packages/neutron/openstack/common/service.py", line 280, in _child_wait_for_exit_or_signal 2014-12-16 14:56:53.444 75106 TRACE neutron launcher.stop() 2014-12-16 14:56:53.444 75106 TRACE neutron File "/usr/lib/python2.7/site-packages/neutron/openstack/common/service.py", line 128, in stop 2014-12-16 14:56:53.444 75106 TRACE neutron self.services.stop() 2014-12-16 14:56:53.444 75106 TRACE neutron File "/usr/lib/python2.7/site-packages/neutron/openstack/common/service.py", line 470, in stop 2014-12-16 14:56:53.444 75106 TRACE neutron service.stop() 2014-12-16 14:56:53.444 75106 TRACE neutron File "/usr/lib/python2.7/site-packages/neutron/service.py", line 132, in stop 2014-12-16 14:56:53.444 75106 TRACE neutron server.kill() 2014-12-16 14:56:53.444 75106 TRACE neutron AttributeError: 'MessageHandlingServer' object has no attribute 'kill' ------------------------------------------------------------------------------- /var/log/messages shows that neutron server was restarted several times upon puppet run. Dec 16 14:56:46 mac848f69fbc49f stonith-ng[14740]: notice: update_cib_cache_cb: [cib_diff_notify] Patch aborted: Application of an update diff failed (-206) Dec 16 14:56:47 mac848f69fbc49f stonith-ng[14740]: notice: update_cib_cache_cb: [cib_diff_notify] Patch aborted: Application of an update diff failed (-206) Dec 16 14:56:47 mac848f69fbc49f stonith-ng[14740]: notice: update_cib_cache_cb: [cib_diff_notify] Patch aborted: Application of an update diff failed (-206) Dec 16 14:56:47 mac848f69fbc49f puppet-agent[68008]: (/Stage[main]/Quickstack::Neutron::All/Exec[neutron-db-manage upgrade]/returns) executed successfully Dec 16 14:56:47 mac848f69fbc49f puppet-agent[68008]: (/Stage[main]/Neutron::Server::Notifications/Nova_admin_tenant_id_setter[nova_admin_tenant_id]/ensure) created Dec 16 14:56:48 mac848f69fbc49f puppet-agent[68008]: (/Stage[main]/Quickstack::Pacemaker::Cinder/Quickstack::Pacemaker::Resource::Service[openstack-cinder-api]/Exec[wait for pcmk_resource openstack-cinder-api]/returns) executed successfu lly Dec 16 14:56:49 mac848f69fbc49f puppet-agent[68008]: (/Stage[main]/Quickstack::Pacemaker::Cinder/Quickstack::Pacemaker::Resource::Service[openstack-cinder-scheduler]/Exec[wait for pcmk_resource openstack-cinder-scheduler]/returns) execut ed successfully Dec 16 14:56:50 mac848f69fbc49f puppet-agent[68008]: (/Stage[main]/Quickstack::Pacemaker::Cinder/Quickstack::Pacemaker::Resource::Service[openstack-cinder-volume]/Exec[wait for pcmk_resource openstack-cinder-volume]/returns) executed suc cessfully Dec 16 14:56:51 mac848f69fbc49f systemd: Stopping OpenStack Neutron Server... Dec 16 14:56:52 mac848f69fbc49f systemd: Starting OpenStack Neutron Server... Dec 16 14:56:53 mac848f69fbc49f crmd[66761]: notice: process_lrm_event: LRM operation neutron-server_monitor_30000 (call=419, rc=196, cib-update=1010, confirmed=false) OCF_PENDING Dec 16 14:56:53 mac848f69fbc49f attrd[14742]: notice: attrd_cs_dispatch: Update relayed from pcmk-mac848f69fbc643 Dec 16 14:56:53 mac848f69fbc49f attrd[14742]: notice: attrd_trigger_update: Sending flush op to all hosts for: fail-count-neutron-server (1) Dec 16 14:56:53 mac848f69fbc49f attrd[14742]: notice: attrd_perform_update: Sent update 63115: fail-count-neutron-server=1 Dec 16 14:56:53 mac848f69fbc49f attrd[14742]: notice: attrd_cs_dispatch: Update relayed from pcmk-mac848f69fbc643 Dec 16 14:56:53 mac848f69fbc49f attrd[14742]: notice: attrd_trigger_update: Sending flush op to all hosts for: last-failure-neutron-server (1418734613) Dec 16 14:56:53 mac848f69fbc49f attrd[14742]: notice: attrd_perform_update: Sent update 63117: last-failure-neutron-server=1418734613 Dec 16 14:56:53 mac848f69fbc49f systemd: Reloading. Dec 16 14:56:53 mac848f69fbc49f systemd: Stopping OpenStack Neutron Server... Dec 16 14:56:53 mac848f69fbc49f crmd[66761]: notice: process_lrm_event: LRM operation neutron-server_stop_0 (call=660, rc=0, cib-update=1011, confirmed=true) ok Dec 16 14:56:53 mac848f69fbc49f puppet-agent[68008]: (/Stage[main]/Neutron::Server/Service[neutron-server]) Failed to call refresh: Could not restart Service[neutron-server]: Execution of '/usr/bin/systemctl restart neutron-server' retur ned 1: Job for neutron-server.service canceled. Dec 16 14:56:53 mac848f69fbc49f puppet-agent[68008]: (/Stage[main]/Neutron::Server/Service[neutron-server]) Could not restart Service[neutron-server]: Execution of '/usr/bin/systemctl restart neutron-server' returned 1: Job for neutron-s erver.service canceled. Dec 16 14:56:53 mac848f69fbc49f puppet-agent[68008]: (/Stage[main]/Neutron::Server/Service[neutron-server]) Wrapped exception: Dec 16 14:56:53 mac848f69fbc49f puppet-agent[68008]: (/Stage[main]/Neutron::Server/Service[neutron-server]) Execution of '/usr/bin/systemctl restart neutron-server' returned 1: Job for neutron-server.service canceled. Dec 16 14:56:53 mac848f69fbc49f systemd: Reloading. Dec 16 14:56:53 mac848f69fbc49f systemd: Starting OpenStack Neutron Server... Dec 16 14:56:53 mac848f69fbc49f crmd[66761]: notice: do_state_transition: State transition S_NOT_DC -> S_PENDING [ input=I_PENDING cause=C_FSA_INTERNAL origin=do_election_count_vote ] Dec 16 14:56:53 mac848f69fbc49f cib[14739]: warning: cib_process_diff: Diff 0.257.1848 -> 0.257.1 from pcmk-mac848f69fbc643 not applied to 0.257.1848: Failed application of an update diff Dec 16 14:56:53 mac848f69fbc49f puppet-agent[68008]: (/Stage[main]/Quickstack::Pacemaker::Neutron/Exec[pcs-neutron-server-set-up]/returns) executed successfully Dec 16 14:56:53 mac848f69fbc49f cib[14739]: notice: cib_server_process_diff: Not applying diff 0.257.1 -> 0.257.2 (sync in progress) Dec 16 14:56:53 mac848f69fbc49f systemd: Starting OpenStack Neutron Server...
*** This bug has been marked as a duplicate of bug 1174389 ***