Description of problem: Did a new deployment with three controller and one compute. after deployment is successful, notice some neutron services on overcloud-control-2 is not running. manually restart the services work [stack@dell-undercloud ~]$ heat stack-list +--------------------------------------+------------+-----------------+----------------------+ | id | stack_name | stack_status | creation_time | +--------------------------------------+------------+-----------------+----------------------+ | 22a91797-dd4a-451e-927c-4bc5990ff471 | overcloud | CREATE_COMPLETE | 2015-09-24T16:18:32Z | +--------------------------------------+------------+-----------------+----------------------+ [stack@dell-undercloud ~]$ source overcloudrc [stack@dell-undercloud ~]$ neutron agent-list +--------------------------------------+--------------------+------------------------------------+-------+----------------+---------------------------+ | id | agent_type | host | alive | admin_state_up | binary | +--------------------------------------+--------------------+------------------------------------+-------+----------------+---------------------------+ | 126eb7ba-8eff-4599-a71f-62b23e992379 | Open vSwitch agent | overcloud-controller-0.localdomain | :-) | True | neutron-openvswitch-agent | | 2dac3402-cbc3-4903-b0ca-be626eda4b01 | Open vSwitch agent | overcloud-compute-0.localdomain | :-) | True | neutron-openvswitch-agent | | 3cadc9dc-d561-4553-ac6d-dfd2a4a6fa07 | Metadata agent | overcloud-controller-0.localdomain | :-) | True | neutron-metadata-agent | | 4129e624-df06-4350-bd60-3c7840a2ea3a | Open vSwitch agent | overcloud-controller-1.localdomain | :-) | True | neutron-openvswitch-agent | | 519e626f-0a09-4cbe-9730-81f2bdc97eb5 | Metadata agent | overcloud-controller-2.localdomain | xxx | True | neutron-metadata-agent | | 6566e981-ea12-4bd8-a8f4-46d7575a816b | DHCP agent | overcloud-controller-2.localdomain | xxx | True | neutron-dhcp-agent | | 66db8bf8-59c9-4a1e-a671-0c15acd9501f | L3 agent | overcloud-controller-0.localdomain | :-) | True | neutron-l3-agent | | 6f47e13c-e6d5-4fe0-911c-7ab0d81fdfc5 | L3 agent | overcloud-controller-1.localdomain | :-) | True | neutron-l3-agent | | 90865d7e-db52-4fe1-8801-f7e159f9f4dc | DHCP agent | overcloud-controller-0.localdomain | :-) | True | neutron-dhcp-agent | | b3fffb3b-fd83-4f61-ad02-a3085e9e6658 | Open vSwitch agent | overcloud-controller-2.localdomain | :-) | True | neutron-openvswitch-agent | | bb89dc91-4154-4d3a-bf9d-efdadfa98658 | Metadata agent | overcloud-controller-1.localdomain | :-) | True | neutron-metadata-agent | | bec80895-d553-43ef-975f-5cf3c4545c26 | L3 agent | overcloud-controller-2.localdomain | xxx | True | neutron-l3-agent | | d541ef16-8b42-4799-b2e7-d3c3f17f5502 | DHCP agent | overcloud-controller-1.localdomain | :-) | True | neutron-dhcp-agent | | ff418ec1-f807-44a9-aea4-47a5345bad51 | Open vSwitch agent | overcloud-compute-35.localdomain | :-) | True | neutron-openvswitch-agent | +--------------------------------------+--------------------+------------------------------------+-------+----------------+---------------------------+ [stack@dell-undercloud ~]$ [heat-admin@overcloud-controller-2 ~]$ systemctl -a | grep neutron neutron-bsn-lldp.service loaded active running bsn lldp neutron-dhcp-agent.service loaded failed failed OpenStack Neutron DHCP Agent neutron-openvswitch-agent.service loaded active running Cluster Controlled neutron-openvswitch-agent neutron-server.service loaded active running Cluster Controlled neutron-server [stack@dell-undercloud ~]$ neutron agent-list +--------------------------------------+--------------------+------------------------------------+-------+----------------+---------------------------+ | id | agent_type | host | alive | admin_state_up | binary | +--------------------------------------+--------------------+------------------------------------+-------+----------------+---------------------------+ | 126eb7ba-8eff-4599-a71f-62b23e992379 | Open vSwitch agent | overcloud-controller-0.localdomain | :-) | True | neutron-openvswitch-agent | | 2dac3402-cbc3-4903-b0ca-be626eda4b01 | Open vSwitch agent | overcloud-compute-0.localdomain | :-) | True | neutron-openvswitch-agent | | 3cadc9dc-d561-4553-ac6d-dfd2a4a6fa07 | Metadata agent | overcloud-controller-0.localdomain | :-) | True | neutron-metadata-agent | | 4129e624-df06-4350-bd60-3c7840a2ea3a | Open vSwitch agent | overcloud-controller-1.localdomain | :-) | True | neutron-openvswitch-agent | | 519e626f-0a09-4cbe-9730-81f2bdc97eb5 | Metadata agent | overcloud-controller-2.localdomain | :-) | True | neutron-metadata-agent | | 6566e981-ea12-4bd8-a8f4-46d7575a816b | DHCP agent | overcloud-controller-2.localdomain | :-) | True | neutron-dhcp-agent | | 66db8bf8-59c9-4a1e-a671-0c15acd9501f | L3 agent | overcloud-controller-0.localdomain | :-) | True | neutron-l3-agent | | 6f47e13c-e6d5-4fe0-911c-7ab0d81fdfc5 | L3 agent | overcloud-controller-1.localdomain | :-) | True | neutron-l3-agent | | 90865d7e-db52-4fe1-8801-f7e159f9f4dc | DHCP agent | overcloud-controller-0.localdomain | :-) | True | neutron-dhcp-agent | | b3fffb3b-fd83-4f61-ad02-a3085e9e6658 | Open vSwitch agent | overcloud-controller-2.localdomain | :-) | True | neutron-openvswitch-agent | | bb89dc91-4154-4d3a-bf9d-efdadfa98658 | Metadata agent | overcloud-controller-1.localdomain | :-) | True | neutron-metadata-agent | | bec80895-d553-43ef-975f-5cf3c4545c26 | L3 agent | overcloud-controller-2.localdomain | :-) | True | neutron-l3-agent | | d541ef16-8b42-4799-b2e7-d3c3f17f5502 | DHCP agent | overcloud-controller-1.localdomain | :-) | True | neutron-dhcp-agent | | ff418ec1-f807-44a9-aea4-47a5345bad51 | Open vSwitch agent | overcloud-compute-35.localdomain | :-) | True | neutron-openvswitch-agent | +--------------------------------------+--------------------+------------------------------------+-------+----------------+---------------------------+ Version-Release number of selected component (if applicable): How reproducible: seen it once Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Additional info:
sosreport from undercloud as well as overcloud-control-2 is at https://bigswitch.box.com/s/gm9ajwsiaayygyjhdb7a83a8jh74mspg
Did a pcs resource cleanup do anything to resolve the service availability?
Verified on 7.2 - 2015.1.2 After overcloud deployment - virt environment 3 controllers 1 compute [root@overcloud-controller-2 ~]# pcs status Cluster name: tripleo_cluster Last updated: Mon Jan 11 13:27:57 2016 Last change: Mon Jan 11 11:35:15 2016 by root via crm_resource on overcloud-controller-0 Stack: corosync Current DC: overcloud-controller-2 (version 1.1.13-10.el7-44eb2dd) - partition with quorum 3 nodes and 108 resources configured Online: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ] Full list of resources: ip-192.0.2.6 (ocf::heartbeat:IPaddr2): Started overcloud-controller-0 Clone Set: haproxy-clone [haproxy] Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ] ip-192.0.2.7 (ocf::heartbeat:IPaddr2): Started overcloud-controller-1 Master/Slave Set: galera-master [galera] Masters: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ] Master/Slave Set: redis-master [redis] Masters: [ overcloud-controller-1 ] Slaves: [ overcloud-controller-0 overcloud-controller-2 ] Clone Set: mongod-clone [mongod] Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ] Clone Set: rabbitmq-clone [rabbitmq] Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ] Clone Set: memcached-clone [memcached] Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ] Clone Set: openstack-nova-scheduler-clone [openstack-nova-scheduler] Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ] Clone Set: neutron-l3-agent-clone [neutron-l3-agent] Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ] Clone Set: openstack-ceilometer-alarm-notifier-clone [openstack-ceilometer-alarm-notifier] Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ] Clone Set: openstack-heat-engine-clone [openstack-heat-engine] Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ] Clone Set: openstack-ceilometer-api-clone [openstack-ceilometer-api] Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ] Clone Set: neutron-metadata-agent-clone [neutron-metadata-agent] Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ] Clone Set: neutron-ovs-cleanup-clone [neutron-ovs-cleanup] Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ] Clone Set: neutron-netns-cleanup-clone [neutron-netns-cleanup] Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ] Clone Set: openstack-heat-api-clone [openstack-heat-api] Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ] Clone Set: openstack-cinder-scheduler-clone [openstack-cinder-scheduler] Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ] Clone Set: openstack-nova-api-clone [openstack-nova-api] Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ] Clone Set: openstack-heat-api-cloudwatch-clone [openstack-heat-api-cloudwatch] Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ] Clone Set: openstack-ceilometer-collector-clone [openstack-ceilometer-collector] Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ] Clone Set: openstack-keystone-clone [openstack-keystone] Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ] Clone Set: openstack-nova-consoleauth-clone [openstack-nova-consoleauth] Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ] Clone Set: openstack-glance-registry-clone [openstack-glance-registry] Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ] Clone Set: openstack-ceilometer-notification-clone [openstack-ceilometer-notification] Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ] Clone Set: openstack-cinder-api-clone [openstack-cinder-api] Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ] Clone Set: neutron-dhcp-agent-clone [neutron-dhcp-agent] Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ] Clone Set: openstack-glance-api-clone [openstack-glance-api] Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ] Clone Set: neutron-openvswitch-agent-clone [neutron-openvswitch-agent] Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ] Clone Set: openstack-nova-novncproxy-clone [openstack-nova-novncproxy] Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ] Clone Set: delay-clone [delay] Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ] Clone Set: neutron-server-clone [neutron-server] Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ] Clone Set: httpd-clone [httpd] Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ] Clone Set: openstack-ceilometer-central-clone [openstack-ceilometer-central] Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ] Clone Set: openstack-ceilometer-alarm-evaluator-clone [openstack-ceilometer-alarm-evaluator] Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ] Clone Set: openstack-heat-api-cfn-clone [openstack-heat-api-cfn] Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ] openstack-cinder-volume (systemd:openstack-cinder-volume): Started overcloud-controller-2 Clone Set: openstack-nova-conductor-clone [openstack-nova-conductor] Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ] PCSD Status: overcloud-controller-0: Online overcloud-controller-1: Online overcloud-controller-2: Online Daemon Status: corosync: active/enabled pacemaker: active/enabled pcsd: active/enabled [stack@instack ~]$ neutron agent-list +--------------------------------------+--------------------+------------------------------------+-------+----------------+---------------------------+ | id | agent_type | host | alive | admin_state_up | binary | +--------------------------------------+--------------------+------------------------------------+-------+----------------+---------------------------+ | 13ab282f-2405-44e8-b790-defa5db3ac08 | L3 agent | overcloud-controller-2.localdomain | :-) | True | neutron-l3-agent | | 3e777156-f477-4e63-b6ff-538201774b61 | Metadata agent | overcloud-controller-0.localdomain | :-) | True | neutron-metadata-agent | | 55412dad-5305-4763-a219-8b52071d726e | DHCP agent | overcloud-controller-1.localdomain | :-) | True | neutron-dhcp-agent | | 707b992d-8856-4d7d-8da6-6f134b06c1a9 | Open vSwitch agent | overcloud-controller-1.localdomain | :-) | True | neutron-openvswitch-agent | | 7a8a7d09-d593-4d3b-a71e-b34f63a39570 | Open vSwitch agent | overcloud-controller-0.localdomain | :-) | True | neutron-openvswitch-agent | | 83b4a90a-023a-45c2-908f-7e67f190eebd | Metadata agent | overcloud-controller-2.localdomain | :-) | True | neutron-metadata-agent | | a1724073-a211-41ab-b64e-362591a49ead | L3 agent | overcloud-controller-0.localdomain | :-) | True | neutron-l3-agent | | a6c8baf3-86f6-4c6f-a2d8-04aed9c3b8d1 | Open vSwitch agent | overcloud-compute-0.localdomain | :-) | True | neutron-openvswitch-agent | | b4367e87-1cd6-491d-a318-1826592378b0 | Open vSwitch agent | overcloud-controller-2.localdomain | :-) | True | neutron-openvswitch-agent | | c96831fe-fa71-4275-b0f4-b7066c34ebe4 | Metadata agent | overcloud-controller-1.localdomain | :-) | True | neutron-metadata-agent | | dafc6340-6534-4e62-a066-37439c001446 | DHCP agent | overcloud-controller-0.localdomain | :-) | True | neutron-dhcp-agent | | e55627c3-f803-45cf-a490-13113d7a6fff | DHCP agent | overcloud-controller-2.localdomain | :-) | True | neutron-dhcp-agent | | fcb21af6-d01d-42fc-8b33-873344410809 | L3 agent | overcloud-controller-1.localdomain | :-) | True | neutron-l3-agent | +--------------------------------------+--------------------+------------------------------------+-------+----------------+---------------------------+
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHEA-2016-0604.html