Bug 1227633 – DHCPNAK after neutron-dhcp-agent restart

Bugzilla will be upgraded to version 5.0. The upgrade date is tentatively scheduled for 2 December 2018, pending final testing and feedback.

Bug 1227633 - DHCPNAK after neutron-dhcp-agent restart

Summary:	DHCPNAK after neutron-dhcp-agent restart

Status:	CLOSED ERRATA

Aliases:	None

Product:	Red Hat OpenStack
Classification:	Red Hat
Component:	openstack-neutron (Show other bugs)
Sub Component:
Version:	7.0 (Kilo)
Hardware:	Unspecified Unspecified

Priority	unspecified Severity high
Target Milestone:	ga
Target Release:	7.0 (Kilo)
Assigned To:	Nir Magnezi
QA Contact:	Toni Freger
Docs Contact:

URL:
Whiteboard:
Keywords:

Depends On:
Blocks:
	Show dependency tree / graph

Reported:	2015-06-03 04:10 EDT by Nir Magnezi
Modified:	2016-04-26 14:51 EDT (History)
CC List:	9 users (show)

See Also:
Fixed In Version:	openstack-neutron-2015.1.0-2.el7ost
Doc Type:	Bug Fix
Doc Text:	Previously, dnsmasq did not save lease information in persistent storage, and when it was restarted, the lease information was lost. This behavior was a result of the removal of the dnsmasq '--dhcp-script' option under BZ#1202392. As a result, instances were stuck in the network boot process for a long period of time. In addition, NACK messages were noted in the dnsmasq log. This update addresses this issue by removing the authoritative option, so that NAKs are not sent in response to DHCPREQUESTs to other servers. This change is expected to prevent dnsmasq from NAKing clients renewing leases issued before it was restarted/rescheduled, with the result that no DHCPNAK messages can be found in the log files.
Story Points:	---
Clone Of:
Clones:	1227635 1227638 (view as bug list)
Environment:
Last Closed:	2015-08-05 09:24:44 EDT
Type:	Bug
Regression:	---
Mount Type:	---
Documentation:	---
CRM:
Verified Versions:
Category:	---
oVirt Team:	---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---

Attachments	(Terms of Use)
Add an attachment (proposed patch, testcase, etc.)

External Trackers
Tracker	ID	Priority	Status	Summary	Last Updated
Launchpad	1345947	None	None	None	Never
Launchpad	1457900	None	None	None	Never
Red Hat Product Errata	RHEA-2015:1548	normal	SHIPPED_LIVE	Red Hat Enterprise Linux OpenStack Platform Enhancement Advisory	2015-08-05 13:07:06 EDT

Groups: None (edit)

Description Nir Magnezi 2015-06-03 04:10:50 EDT

Description of problem:
=======================

After rolling out a configuration change, we restarted neutron-dhcp-agent service, and then dnsmasq logs start flooding: DHCPNAK ... lease not found.
DHCPNAK is replied by dnsmasq for all DHCPREQUEST renews from all VMs. However the MAC and IP pairs exist in host files.
The log flooding increases when more and more VMs start renewing and they keep retrying until IP expire and send DHCPDISCOVER and reinit the IP.
The log flooding gradually disappears when the VMs IP expire and send DHCPDISCOVER, to which dnsmasq respond DHCPOFFER properly.

Analysis:
=========
I noticed that option --leasefile-ro is used in dnsmasq command when started by neutron dhcp-agent. According to dnsmasq manual, this option should be used together with --dhcp-script to customize the lease database. However, the option --dhcp-script was removed when fixing bug 1202392.
Because of this, dnsmasq will not save lease information in persistent storage, and when it is restarted, lease information is lost.

Solution:
=========
Simply replace --leasefile-ro by --dhcp-leasefile=<path to dhcp runtime files>/lease would solve the problem. (patch attached)

Comment 7 Toni Freger 2015-06-17 02:55:45 EDT

Verified on RHEL7.1
puddle from rel-eng/OpenStack/7.0-RHEL-7-Beta/2015-06-12.1/

After the lease was renewed no DHCPNAK messages were found.

The DHCP messages from /var/log/messages file of the agents.

Agent1 :

Jun 17 09:46:24 rose11 dnsmasq-dhcp[20559]: DHCPDISCOVER(tapc705ecac-68) fa:16:3e:4a:27:39
Jun 17 09:46:24 rose11 dnsmasq-dhcp[20559]: DHCPOFFER(tapc705ecac-68) 20.20.20.4 fa:16:3e:4a:27:39
Jun 17 09:46:24 rose11 dnsmasq-dhcp[20559]: DHCPREQUEST(tapc705ecac-68) 20.20.20.4 fa:16:3e:4a:27:39
Jun 17 09:46:24 rose11 dnsmasq-dhcp[20559]: DHCPACK(tapc705ecac-68) 20.20.20.4 fa:16:3e:4a:27:39 host-20-20-20-4


Agent2: 

Jun 17 09:46:24 rose12 dnsmasq-dhcp[20043]: DHCPDISCOVER(tapd266cd9a-8b) fa:16:3e:4a:27:39
Jun 17 09:46:24 rose12 dnsmasq-dhcp[20043]: DHCPOFFER(tapd266cd9a-8b) 20.20.20.4 fa:16:3e:4a:27:39

Comment 9 errata-xmlrpc 2015-08-05 09:24:44 EDT

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2015:1548

Note You need to log in before you can comment on or make changes to this bug.