Bug 567429 - slapd didn't close connection and get into CLOSE_WAIT state
Summary: slapd didn't close connection and get into CLOSE_WAIT state
Alias: None
Product: 389
Classification: Retired
Component: Directory Server
Version: 1.2.1
Hardware: x86_64
OS: Linux
Target Milestone: ---
Assignee: Rich Megginson
QA Contact: Viktor Ashirov
Depends On:
Blocks: 434914 389_1.2.6
TreeView+ depends on / blocked
Reported: 2010-02-22 23:18 UTC by beyonddc.storage
Modified: 2015-12-07 17:03 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Last Closed: 2015-12-07 17:03:12 UTC

Attachments (Terms of Use)
JNDI Test Reproducer (8.30 KB, text/x-java)
2010-02-22 23:20 UTC, beyonddc.storage
no flags Details
a netstat script I use to monitor slapd open connection (227 bytes, application/x-shellscript)
2010-02-22 23:21 UTC, beyonddc.storage
no flags Details
patch (2.20 KB, patch)
2010-03-25 22:14 UTC, Rich Megginson
nhosoi: review+
Details | Diff

Description beyonddc.storage 2010-02-22 23:18:48 UTC
Description of problem:
I am running 389 DS version 1.2.5, build number 2010.012.2034 on RHEL 5.2.

I have a problem that slapd didn't close a connection and eventually get into a CLOSE_WAIT state after my JAVA application exit.

The scenario only happen when my application registers a NamingListener via the JAVA JNDI (Java Naming Directory Interface).  I believe the NamingListener is equivalent to the Persistent Search.  This problem doesn't exist if I don't use the JNDI NamingListener capability.

From my understanding, I did everything correctly in my application.  I create a context, add a listener, do some stuffs, remove the listener and then close the context.

Version-Release number of selected component (if applicable):
This issue can be replicated using 389 Directory Server 1.2.5

How reproducible:
Run my JNDI_Simple reproducer, and monitor slapd open connection using netstat.  I will attach the JAVA JNDI_Simple test procedure and a netstat script I use to monitor slapd.  The netstat script I am providing uses the "-p" option which requires to be run as root or an user with sudo privilege.

Steps to Reproduce:
1. Download the attached JNDI_Simple.java
2. Edit JNDI_Simple.java and search for "MODIFY ME" in all caps.  Replace anything that is necessary to run in your environment such as DN location, LDAP host, directory manager's username/password
3. Compile JNDI_Simple.java by "javac JNDI_Simple.java
4. Execute the application by "java -cp . JNDI_Simple"

Actual results:
After the JNDI test reproducer finishes running, using netstat, you'll see that there is a stale LDAP connection sitting in CLOSE_WAIT state.

tcp        0      0 localhost:ldap              localhost:58104             CLOSE_WAIT  9743/ns-slapd

Expected results:
After JNDI test reproducer finishes, all connection should be released properly. 

Additional info:
This issue has been raised on 389-users mailing list and is documented in below link. http://www.mail-archive.com/389-users@lists.fedoraproject.org/msg00234.html

I have done some testing on it
1) Tried OpenDS and confirm that OpenDS doesn't have this issue.
2) Modified JNDI source code in OpenJDK to set persistent search to be a NONCRITICAL control instead of default CRITICAL control.

Comment 1 beyonddc.storage 2010-02-22 23:20:06 UTC
Created attachment 395601 [details]
JNDI Test Reproducer

Might require minor modification to set the correct directory manager username and password and the base dn to run the test.

Comment 2 beyonddc.storage 2010-02-22 23:21:29 UTC
Created attachment 395602 [details]
a netstat script I use to monitor slapd open connection

Require to run as root or an user with sudo privilege.

Comment 4 Rich Megginson 2010-03-25 22:14:58 UTC
Created attachment 402698 [details]

Comment 5 Rich Megginson 2010-03-26 00:03:11 UTC
To ssh://git.fedorahosted.org/git/389/ds.git
   65e04b8..48d50e8  master -> master
commit 48d50e8f06e0c2fd8e2541c3c239217a6b72ebbc
Author: Rich Megginson <rmeggins@redhat.com>
Date:   Thu Mar 25 15:11:58 2010 -0600
    Reviewed by: nhosoi (Thanks!)
    Branch: HEAD
    Fix Description: The JNDI code attached to the bug uses persistent search.
    The connection pool code handles persistent searches differently than
    regular connections.  The connection pool code was acquiring a reference
    to a conn, but was not releasing it in the persistent search case, assumin
    the persistent search code did not also have a reference, but it does.  Th
    caused connection_table_move_connection_out_of_active_list() to not move
    the connection out of the active list, and therefore available for closing
    because there was an outstanding reference.  The solution is for the
    connection pool code to release its reference.
    Platforms tested: RHEL5 x86_64
    Flag Day: no
    Doc impact: no

To ssh://git.fedorahosted.org/git/389/ds.git
   41c5be0..6dec072  Directory_Server_8_2_Branch -> Directory_Server_8_2_Branch
commit 6dec0725ba70e97deefcaec605b5192368e2625c
Author: Rich Megginson <rmeggins@redhat.com>
Date:   Thu Mar 25 15:11:58 2010 -0600

Note You need to log in before you can comment on or make changes to this bug.