Bug 1318046

Summary: openstack-nova-network fails to start
Product: Red Hat OpenStack Reporter: Sean Waite <swaite>
Component: openstack-novaAssignee: Eoghan Glynn <eglynn>
Status: CLOSED INSUFFICIENT_DATA QA Contact: nlevinki <nlevinki>
Severity: urgent Docs Contact:
Priority: unspecified    
Version: 4.0CC: berrange, dasmith, eglynn, jschluet, kchamart, ndipanov, sbauza, sferdjao, sgordon, swaite, vromanso, yeylon
Target Milestone: ---Flags: eglynn: needinfo? (swaite)
Target Release: 8.0 (Liberty)   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-03-31 14:03:35 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Sean Waite 2016-03-15 20:33:23 UTC 
Description of problem:
Our QE Openstack environment, on OSP4, is down and we cannot start the openstack-nova-network service.

 openstack-nova-network_stop_0 on openstack-network-01 'unknown error' (1): call=-1, status=Timed Out, last-rc-change='Tue Mar 15 18:52:30 2016', queued=0ms, exec=0ms

There isn't anything in /var/log/nova/network.log from any recent period. We had been seeing timeout issues (No protocol received, closing) on the controllers, but increasing the timeout stopped that. We've also seen conntrack table full errors on the network node, but that doesn't appear to be preventing us from starting the service and I've increased the max on that as well.



Version-Release number of selected component (if applicable):
openstack-nova-network-2013.2.4-4.el6ost.noarch

How reproducible:
100%


Steps to Reproduce:
1. pcs resource disable openstack-nova-network
2. pcs resource enable openstack-nova-network
3. Still hasn't started.

Actual results:
openstack-nova-network fails to start

Expected results:
service starts

Additional info:
Comment 1 Sean Waite 2016-03-15 21:25:46 UTC 
One of the behaviors we're seeing is that we cannot create instances. They appear to go from Task State networking to Status: ERROR.

I've unmanaged nova-networking in PCS and started the service manually via standard sysv commands, but we get the same behavior. The service appears to be running here, though.
Comment 2 Eoghan Glynn 2016-03-16 17:46:00 UTC 
@swaite: can you provide relevant log excerpts from the compute and network services?
Comment 4 Eoghan Glynn 2016-03-31 14:03:35 UTC 
Closing due to lack of response on the request for logs. Feel free to reopen if you can provide the diagnostics.