Bug 1292390

Summary: overcloud neutron services are down after deployment
Product: Red Hat OpenStack Reporter: Amit Ugol <augol>
Component: rhosp-directorAssignee: Hugh Brock <hbrock>
Status: CLOSED ERRATA QA Contact: Amit Ugol <augol>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 7.0 (Kilo)CC: augol, dmacpher, mburns, oblaut, rhel-osp-director-maint
Target Milestone: y3Keywords: TestOnly, Triaged
Target Release: 7.0 (Kilo)   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-02-18 16:47:53 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Amit Ugol 2015-12-17 10:01:25 UTC
Description of problem:
After deployment, it seems that all neutron services on the overcloud have stopped:
[stack@instack ~]$ . overcloudrc
[stack@instack ~]$ neutron agent-list
+--------------------------------------+--------------------+------------------------------------+-------+----------------+---------------------------+
| id                                   | agent_type         | host                               | alive | admin_state_up | binary                    |
+--------------------------------------+--------------------+------------------------------------+-------+----------------+---------------------------+
| 4d09fcd9-780d-4810-ace4-532303fb2abe | Open vSwitch agent | overcloud-compute-0.localdomain    | :-)   | True           | neutron-openvswitch-agent |
| 7d7e264f-5746-4ced-a0c5-ceacf81b147d | Open vSwitch agent | overcloud-controller-0.localdomain | xxx   | True           | neutron-openvswitch-agent |
| 90937ab5-aba1-4871-b58e-67699f964d04 | DHCP agent         | overcloud-controller-0.localdomain | xxx   | True           | neutron-dhcp-agent        |
| 911d307c-c9c3-444e-b089-03f7304fc3c8 | L3 agent           | overcloud-controller-0.localdomain | xxx   | True           | neutron-l3-agent          |
| fc237f6d-fa49-4b1b-98dd-1aca1bb01679 | Metadata agent     | overcloud-controller-0.localdomain | xxx   | True           | neutron-metadata-agent    |
+--------------------------------------+--------------------+------------------------------------+-------+----------------+---------------------------+

[heat-admin@overcloud-controller-0 ~]$ sudo pcs status --full | grep Stop
     neutron-l3-agent   (systemd:neutron-l3-agent):     Stopped
     Stopped: [ overcloud-controller-0 ]
     neutron-metadata-agent     (systemd:neutron-metadata-agent):       Stopped
     Stopped: [ overcloud-controller-0 ]
     neutron-dhcp-agent (systemd:neutron-dhcp-agent):   Stopped
     Stopped: [ overcloud-controller-0 ]


*** In this case, upgrade should in theory not even start, but not only can I start it, upgrade to 7.1 works.

**** full logs too big to be attached. Please find them in http://ikook.tlv.redhat.com/unrelated/files/logs.tar.xz

Version-Release number of selected component (if applicable):
7.0 GA -
neutron-2015.1.0-12
openstack-puppet-modules-2015.1.8-8
pcs-0.9.137-13

How reproducible:
always on 7.0. 7.1 seems to not be effected.

Steps to Reproduce:
1. Deploy 7.0 GA.

Actual results:
Neutron services on the controller nodes stops after a few minutes.

Expected results:
All services should be up and running.

Additional info:

Comment 1 Amit Ugol 2015-12-17 10:43:46 UTC
After restarting the services manually and waiting for some time, they seem to be up and running just fine now with no other change to the system.

Comment 3 Hugh Brock 2016-02-05 14:39:40 UTC
We can't test this against 8.0 because there is only 1 8.0 version :). Please retest against 7.3.

Comment 6 errata-xmlrpc 2016-02-18 16:47:53 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-0264.html

Comment 7 Amit Ugol 2016-03-09 17:05:36 UTC
since it got closed, I'll just clear that need info state