Bug 1403944

Summary:	Once in a while, overcloud deployment fails because nova-compute doesn't start successfully on the computes
Product:	Red Hat OpenStack	Reporter:	David Hill <dhill>
Component:	openstack-nova	Assignee:	Eoghan Glynn <eglynn>
Status:	CLOSED NOTABUG	QA Contact:	Prasanth Anbalagan <panbalag>
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	9.0 (Mitaka)	CC:	berrange, dasmith, dhill, eglynn, kchamart, sbauza, sferdjao, sgordon, srevivo, vromanso
Target Milestone:	---	Keywords:	ZStream
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2016-12-16 20:07:28 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:

Description David Hill 2016-12-12 16:51:06 UTC

Description of problem:
Once in a while, overcloud deployment fails because nova-compute doesn't start successfully on the computes.  This is problematic because if we kill the restart, it fails the deployment and there're no ways of getting this back to a deploying state.,

root      3716  0.2  1.0 279524 30980 ?        Ss   16:01   0:06 /usr/bin/python2 /usr/bin/os-collect-config
root     17886  0.0  0.3 193380  9928 ?        S    16:06   0:00  \_ /usr/bin/python2 /usr/bin/os-refresh-config
root     17932  0.0  0.0 115248  1580 ?        S    16:06   0:00      \_ /bin/bash /usr/local/bin/dib-run-parts /usr/libexec/os-refresh-config/configure.d
root     19482  0.0  0.4 224380 14220 ?        S    16:06   0:00          \_ python /usr/libexec/os-refresh-config/configure.d/55-heat-config
root     19487  0.0  0.2 149384  6140 ?        S    16:06   0:00              \_ python /var/lib/heat-config/hooks/puppet
root     19489  0.5  4.1 412576 121572 ?       Sl   16:06   0:14                  \_ /usr/bin/ruby /usr/bin/puppet apply --detailed-exitcodes /var/lib/heat-config/heat-config-puppet/386b3ea2-cc29-4096-8946-26bf22f3a4bd.pp
root     20180  0.0  0.0 128408  1348 ?        Ss   16:07   0:00                      \_ /usr/bin/systemctl start openstack-nova-compute
root      3718  0.0  0.0 126220  1672 ?        Ss   16:01   0:00 /usr/sbin/crond -n
ntp      19908  0.0  0.0  29408  2064 ?        Ss   16:07   0:00 /usr/sbin/ntpd -u ntp:ntp -g
root     20042  0.0  0.6 611252 19920 ?        Ssl  16:07   0:00 /usr/sbin/libvirtd --listen
root     20113  0.1  0.3 223668 11436 ?        Ss   16:07   0:02 /usr/sbin/snmpd -LS0-6d -f
neutron  20151  1.3  2.1 358788 62840 ?        Ss   16:07   0:33 /usr/bin/python2 /usr/bin/neutron-openvswitch-agent --config-file /usr/share/neutron/neutron-dist.conf --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/openvswitch_agent.ini --config-dir /etc/neutron/conf.d/common --config-dir /etc/neutron/conf.d/neutron-openvswitch-agent --log-file /var/log/neutron/openvswitch-agent.log
root     20198  0.0  0.0 193332  2780 ?        S    16:07   0:00  \_ sudo neutron-rootwrap-daemon /etc/neutron/rootwrap.conf
root     20199  0.0  0.4 364596 12812 ?        Sl   16:07   0:00  |   \_ /usr/bin/python2 /usr/bin/neutron-rootwrap-daemon /etc/neutron/rootwrap.conf
root     20433  0.0  0.0 193332  2784 ?        S    16:27   0:00  \_ sudo neutron-rootwrap /etc/neutron/rootwrap.conf ovsdb-client monitor Interface name,ofport,external_ids --format=json
root     20435  0.0  0.3 207780 11592 ?        S    16:27   0:00      \_ /usr/bin/python2 /usr/bin/neutron-rootwrap /etc/neutron/rootwrap.conf ovsdb-client monitor Interface name,ofport,external_ids --format=json
root     20436  0.0  0.0  47500  2624 ?        S    16:27   0:00          \_ /bin/ovsdb-client monitor Interface name,ofport,external_ids --format=json
nova     20184  1.2  3.8 507868 112560 ?       Ss   16:07   0:30 /usr/bin/python2 /usr/bin/nova-compute


Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 2 David Hill 2016-12-16 20:07:28 UTC

This is a normal behavior. Until all controllers are up/running, the nova-compute deployment on the computes might be stuck in a starting state until rabbitmq/nova-conductor is connected.