Bug 1181107 - Keepalived tries to resolve dns entries at restart, which leads to MASTER flapping
Summary: Keepalived tries to resolve dns entries at restart, which leads to MASTER fla...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: keepalived
Version: 7.1
Hardware: Unspecified
OS: Unspecified
urgent
urgent
Target Milestone: rc
: 7.2
Assignee: Ryan O'Hara
QA Contact: Brandon Perkins
URL:
Whiteboard:
Depends On:
Blocks: 1181592
TreeView+ depends on / blocked
 
Reported: 2015-01-12 11:40 UTC by Miguel Angel Ajo
Modified: 2015-11-19 12:20 UTC (History)
7 users (show)

Fixed In Version: keepalived-1.2.13-7.el7
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-11-19 12:20:17 UTC
Target Upstream Version:


Attachments (Terms of Use)
Set global data default values after parsing config file (3.39 KB, patch)
2015-05-14 17:38 UTC, Ryan O'Hara
no flags Details | Diff


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2015:2379 normal SHIPPED_LIVE keepalived bug fix update 2015-11-19 10:51:21 UTC

Description Miguel Angel Ajo 2015-01-12 11:40:53 UTC
Description of problem:

When keepalived is running inside a network namespace it could have no network access to the system configured name servers. So during a -HUP restart to reload new configuration keepalived tries to resolve it's own hostname twice, waiting for DNS timeout.


This makes keepalived MASTER move around the BACKUP instances.

The DNS resolutions are done to fill up default configuration regardless of configuration being set or not. So there is no way to avoid it without system workarounds.

This workaround fixes the issue, but could interfere with other services:

# echo 127.0.0.1 $( hostname ) >>/etc/hosts

Version-Release number of selected component (if applicable):

keepalived-1.2.13-6.el7.x86_64

How reproducible:

100%

Steps to Reproduce:
1. set /etc/resolv.conf to a non responding DNS server.
2.
3.

Actual results:

when you kill -HUP $(keepalived_pid) the MASTER moves to a SLAVE because keepalived get's stuck in name resolution.

Expected results:

If you provide router_id and you're not configuring email notifications no DNS resolution should be invoked to fill up any of those default settings within keepalived.


Additional info:

Comment 9 Ryan O'Hara 2015-05-14 17:38:07 UTC
Created attachment 1025539 [details]
Set global data default values after parsing config file

This patch will defer setting the global data default values until after the config file has been parsed. This will potentially avoid two calls to getaddrinfo. For example, if the router_id and/or email_from parameters are set in the config file, there is no need to call getaddrinfo twice in order to set a default value. Instead, this patch will check to see if they values are unset after parsing the config file. Note that email_from and smtp_connection_to are only set to a default value if they are unitialized and smtp_server is specified.

I'm planning to submit this upstream, but it might be worthwhile to have Neutron developers test this to make sure if fixes the problems. Note that in order to avoid all DNS queries you must have router_id set in global_defs, eg.

global_defs {
    router_id whatever
}

Comment 11 Ryan O'Hara 2015-05-14 18:09:52 UTC
Upstream pull request:

https://github.com/acassen/keepalived/pull/148

Comment 16 errata-xmlrpc 2015-11-19 12:20:17 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-2379.html


Note You need to log in before you can comment on or make changes to this bug.