Bug 567429

Summary: slapd didn't close connection and get into CLOSE_WAIT state
Product: [Retired] 389 Reporter: beyonddc.storage
Component: Directory ServerAssignee: Rich Megginson <rmeggins>
Status: CLOSED CURRENTRELEASE QA Contact: Viktor Ashirov <vashirov>
Severity: medium Docs Contact:
Priority: low    
Version: 1.2.1CC: beyonddc.storage, jgalipea
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-12-07 17:03:12 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 434914, 543590    
Attachments:
Description Flags
JNDI Test Reproducer
none
a netstat script I use to monitor slapd open connection
none
patch nhosoi: review+

Description beyonddc.storage 2010-02-22 23:18:48 UTC 
Description of problem:
I am running 389 DS version 1.2.5, build number 2010.012.2034 on RHEL 5.2.

I have a problem that slapd didn't close a connection and eventually get into a CLOSE_WAIT state after my JAVA application exit.

The scenario only happen when my application registers a NamingListener via the JAVA JNDI (Java Naming Directory Interface).  I believe the NamingListener is equivalent to the Persistent Search.  This problem doesn't exist if I don't use the JNDI NamingListener capability.

From my understanding, I did everything correctly in my application.  I create a context, add a listener, do some stuffs, remove the listener and then close the context.

Version-Release number of selected component (if applicable):
This issue can be replicated using 389 Directory Server 1.2.5


How reproducible:
Run my JNDI_Simple reproducer, and monitor slapd open connection using netstat.  I will attach the JAVA JNDI_Simple test procedure and a netstat script I use to monitor slapd.  The netstat script I am providing uses the "-p" option which requires to be run as root or an user with sudo privilege.


Steps to Reproduce:
1. Download the attached JNDI_Simple.java
2. Edit JNDI_Simple.java and search for "MODIFY ME" in all caps.  Replace anything that is necessary to run in your environment such as DN location, LDAP host, directory manager's username/password
3. Compile JNDI_Simple.java by "javac JNDI_Simple.java
4. Execute the application by "java -cp . JNDI_Simple"

  
Actual results:
After the JNDI test reproducer finishes running, using netstat, you'll see that there is a stale LDAP connection sitting in CLOSE_WAIT state.

tcp        0      0 localhost:ldap              localhost:58104             CLOSE_WAIT  9743/ns-slapd


Expected results:
After JNDI test reproducer finishes, all connection should be released properly. 


Additional info:
This issue has been raised on 389-users mailing list and is documented in below link. http://www.mail-archive.com/389-users@lists.fedoraproject.org/msg00234.html

I have done some testing on it
1) Tried OpenDS and confirm that OpenDS doesn't have this issue.
2) Modified JNDI source code in OpenJDK to set persistent search to be a NONCRITICAL control instead of default CRITICAL control.
Comment 1 beyonddc.storage 2010-02-22 23:20:06 UTC 
Created attachment 395601 [details]
JNDI Test Reproducer

Might require minor modification to set the correct directory manager username and password and the base dn to run the test.
Comment 2 beyonddc.storage 2010-02-22 23:21:29 UTC 
Created attachment 395602 [details]
a netstat script I use to monitor slapd open connection

Require to run as root or an user with sudo privilege.
Comment 4 Rich Megginson 2010-03-25 22:14:58 UTC 
Created attachment 402698 [details]
patch
Comment 5 Rich Megginson 2010-03-26 00:03:11 UTC 
To ssh://git.fedorahosted.org/git/389/ds.git
   65e04b8..48d50e8  master -> master
commit 48d50e8f06e0c2fd8e2541c3c239217a6b72ebbc
Author: Rich Megginson <rmeggins>
Date:   Thu Mar 25 15:11:58 2010 -0600
    Reviewed by: nhosoi (Thanks!)
    Branch: HEAD
    Fix Description: The JNDI code attached to the bug uses persistent search.
    The connection pool code handles persistent searches differently than
    regular connections.  The connection pool code was acquiring a reference
    to a conn, but was not releasing it in the persistent search case, assumin
    the persistent search code did not also have a reference, but it does.  Th
    caused connection_table_move_connection_out_of_active_list() to not move
    the connection out of the active list, and therefore available for closing
    because there was an outstanding reference.  The solution is for the
    connection pool code to release its reference.
    Platforms tested: RHEL5 x86_64
    Flag Day: no
    Doc impact: no

To ssh://git.fedorahosted.org/git/389/ds.git
   41c5be0..6dec072  Directory_Server_8_2_Branch -> Directory_Server_8_2_Branch
commit 6dec0725ba70e97deefcaec605b5192368e2625c
Author: Rich Megginson <rmeggins>
Date:   Thu Mar 25 15:11:58 2010 -0600