RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1464869 - Configuration reload (SIGHUP) opens two (never closed) new pipes in subprocess
Summary: Configuration reload (SIGHUP) opens two (never closed) new pipes in subprocess
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: keepalived
Version: 7.3
Hardware: x86_64
OS: Linux
urgent
urgent
Target Milestone: rc
: ---
Assignee: Ryan O'Hara
QA Contact: Brandon Perkins
URL:
Whiteboard:
Depends On:
Blocks: 1477883 1477907
TreeView+ depends on / blocked
 
Reported: 2017-06-26 06:46 UTC by yann.morice
Modified: 2017-12-20 20:04 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1477883 1477907 (view as bug list)
Environment:
Last Closed: 2017-08-03 14:39:49 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description yann.morice 2017-06-26 06:46:57 UTC
Description of problem: 

At each live reload of configuration (<=> send of SIGHUP signal), the main keepalived subprocess opens two new pipes. These two pipes are never closed until maximum of open files reached and full restart of keepalived.

The problem is that this is widely used by openstack to live update configuration of routers (floating-ips, etc...) and do the router go off-line after too many open files...

The problem was introduced in Keepalived version 1.2.13-9.el7_3 to deal with bug #1429880. Previous version 1.2.13-8.el7 was OK.

Version-Release number of selected component (if applicable): 1.2.13-9.el7_3

Steps to Reproduce:

1. If we run a simple config using vrrp only :

# keepalived -P -f /etc/keepalived/keepalived.conf

vrrp_instance VR_1 {
    state BACKUP
    interface eth0
    virtual_router_id 1
    priority 50
    garp_master_delay 60
    nopreempt
    advert_int 2
    track_interface {
        eth0
    }
    virtual_ipaddress {
        169.254.0.2/24 dev eth0
    }
}

2. We have then two new processes : 

# ps -eafwww |grep keepalived
root     13722     1  0 09:00 ?        00:00:00 keepalived -P -f /etc/keepalived/keepalived-simple.conf
root     13723 13722  0 09:00 ?        00:00:00 keepalived -P -f /etc/keepalived/keepalived-simple.conf

3. First one has two pipes  :

# lsof |grep 13722|grep pipe|wc -l
2

4. Second one has four pipes at the beginning :
# lsof |grep 13723|grep pipe|wc -l
4

5. If we do SIGHUP to the main process to live update configuration :
# kill -HUP 13722

6. and do again :
# lsof |grep 13723|grep pipe|wc -l
6

Actual results:

We have now six pipes... (+2 pipes at each SIGHUP in fact)

Expected results:

With the version 1.2.13-8.el7, doing the same, the second process stay at only two pipes across all SIGHUPs.

Additional info: 

I suspect this to be caused by SIGHUP signal from configuration changes (add of floating ip) as a similar router with no config changes stay ok.

In fact, I think the bug was introduced is in lib/signals.c. I don't see any mechanism to replace signal_handler_destroy for closing these two pipes. This piece of code is still in the trunk (and latest versions) of keepalived. I think these four lines should not be deleted :

-       close(signal_pipe[1]);
-       close(signal_pipe[0]);
-       signal_pipe[1] = -1;
-       signal_pipe[0] = -1;

For information (but not directly linked), we never use the code in #ifdef HAVE_PIPE2 because set of variable in configure is not propagated in Makefiles*. In fact, if we do strings /usr/sbin/keepalived |grep pipe on installed keepalived, we got pipe (instead of pipe2). I suspect we should have something like DEFS	 = @DFLAGS@ -D@SNMP_SUPPORT@ @DEFS@ in lib/Makefile.in to do this.

Comment 2 Ryan O'Hara 2017-08-02 12:48:08 UTC
Since we rebased in 7.4, this bug only needs to be fixed in 7.3.z.

Comment 4 Ryan O'Hara 2017-08-02 13:30:26 UTC
Just to show that this problem does not exist in RHEL 7.4:

# cat /etc/redhat-release && rpm -q keepalived
Red Hat Enterprise Linux Server release 7.4 (Maipo)
keepalived-1.3.5-1.el7.x86_64

# keepalived -P -f /etc/keepalived/keepalived.conf
# ps -eafwww | grep keepalived
root     13321     1  0 08:26 ?        00:00:00 keepalived -P -f /etc/keepalived/keepalived.conf
root     13322 13321  0 08:26 ?        00:00:00 keepalived -P -f /etc/keepalived/keepalived.conf

# lsof | grep 13321 | grep pipe | wc -l
2
# lsof | grep 13322 | grep pipe | wc -l
2

Now SIGHUP the parent process:

# cat /var/run/keepalived.pid 
13321
# kill -HUP 13321

# lsof | grep 13321 | grep pipe | wc -l
2
# lsof | grep 13322 | grep pipe | wc -l
2

This is correct.

Comment 6 Oneata Mircea Teodor 2017-08-03 07:07:39 UTC
This bug has been copied as 7.3 z-stream (EUS) bug #1477883 and now must be
resolved in the current update release, set blocker flag.

Comment 8 Ryan O'Hara 2017-08-03 14:39:49 UTC
Setting as CLOSED/CURRENTRELEASE since the rebase in RHEL 7.4 solved this problem. Fixing only for z-stream.


Note You need to log in before you can comment on or make changes to this bug.