Bug 1308566

Summary: VM's unable to obtain DHCP addresses
Product: Red Hat OpenStack Reporter: Prateek Arora <parora>
Component: openstack-neutronAssignee: lpeer <lpeer>
Status: CLOSED NOTABUG QA Contact: Toni Freger <tfreger>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 7.0 (Kilo)CC: amuller, anande, chrisw, jlibosva, nyechiel, parora, yeylon
Target Milestone: ---   
Target Release: 8.0 (Liberty)   
Hardware: All   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-02-23 10:02:44 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Prateek Arora 2016-02-15 14:07:44 UTC
Description of problem:

VM's are unable to obtain DHCP addresses


Version-Release number of selected component (if applicable):

Red Hat Openstack 7.0

How reproducible:

Reproducible everytime in the attached case's customer environment

Steps to Reproduce:
1. Run "dhclient enp10s0" in the overcloud host
2. The DHCP service is able to assign IP addresses.
3. But the VM's are uanble to get it

Actual results:
The VM's should get an IP address


Expected results:


Additional info:

Here is my analysis of the SOS reports.


Going through the flow for the DHCP allocation, the follwing components and their logs are attached

1) Nova compute- looks to be fine

2) Neutron Server

There are two noteworthy things in the sos reports for neutron_server regarding this
a) 14403 WARNING keystonemiddleware.auth_token [-] Identity response: {"error": {"message": "Could not find token
b) [req-56fca01f-9be7-48c1-b3b6-362ec91b9215 ] show failed (client error): Port 0b07657f-a23c-4508-b53a-335a75337fe2 could not be found

This is where we can see the first trace of the problem we are having. So the problem could either be port not being created or not being found due to failed authentication. We would have to check the logs futher to see where the problem actually lies.

3) DHCP Agent

A log here states that unable to sync network state with trace saying further
task <bound method DhcpAgentWithStateReport._report_state of <neutron.agent.dhcp.agent.DhcpAgentWithStateReport object at 0x34fcb50>> run outlasted interval by 30.06 sec
Timed out waiting for a reply to message ID : ...

4) Dnsmasq


The neutron server not finding the port looks to be the error but then if the neutron server has created the port itself, it should fail in creation itself.

Comment 3 Prateek Arora 2016-02-16 09:55:53 UTC
Please use this link to access the sos reports

https://fubar.gsslab.pnq.redhat.com/01577351/