Bug 1432206

Summary:	content sync plugin can hang server shutdown
Product:	Red Hat Enterprise Linux 7	Reporter:	mreynolds
Component:	389-ds-base	Assignee:	Noriko Hosoi <nhosoi>
Status:	CLOSED ERRATA	QA Contact:	Viktor Ashirov <vashirov>
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	7.4	CC:	mreynolds, nkinder, rmeggins, sramling
Target Milestone:	rc
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	389-ds-base-1.3.6.1-2.el7	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2017-08-01 21:14:10 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description mreynolds 2017-03-14 18:48:16 UTC

Description of problem:

If a sync result thread fails to acquire a connection the plugin's thread count is not properly updated.  This causes the plugin to hang when shutting down the server.

Version-Release number of selected component (if applicable):

389-ds-base-1.3.3 and up are affected

dirsrv error log:

[14/Mar/2017:17:37:02.765911998 +0100] - ERR - slapi_connection_acquire - conn=29 fd=107 Attempt to acquire connection in the closing state
[14/Mar/2017:17:37:02.768441567 +0100] - ERR - content-sync-plugin - sync_send_results - conn=29 op=0 Could not acquire the connection - aborted

Comment 3 Sankar Ramalingam 2017-05-15 15:50:54 UTC

Hi Mark, request you to add steps to reproduce.

Comment 4 mreynolds 2017-05-15 16:40:30 UTC

Hi Sankar,

This was only discovered and reproduced during an IPA install.  It was really a side effect of several other "nunc-stans" bugs that triggered this connection failure that lead to the hang in the content sync plugin.  All of nunc-stans bugs are now fixed, and the potential hang is fixed in the sync plugin.  

So I'm sorry there is no easy reproducer.  The only way would be to find a way to block the sync plugin's connections, and then immediately restart the server.  I'm afraid I don't know of a way to do that without adding instrumented code to internally trigger the error condition in DS.

Comment 5 Sankar Ramalingam 2017-06-14 14:24:47 UTC

Thanks Mark! for the clarification. 
Installed IPA server with the latest build of 389-ds-base on RHEL-7.4 and I didn't observe any hang. Installation went through fine as well.

==============================================================================
Setup complete

Next steps:
	1. You must make sure these network ports are open:
		TCP Ports:
		  * 80, 443: HTTP/HTTPS
		  * 389, 636: LDAP/LDAPS
		  * 88, 464: kerberos
		UDP Ports:
		  * 88, 464: kerberos
		  * 123: ntp

	2. You can now obtain a kerberos ticket using the command: 'kinit admin'
	   This ticket will allow you to use the IPA tools (e.g., ipa user-add)
	   and the web user interface.

Be sure to back up the CA certificates stored in /root/cacert.p12
These files are required to create replicas. The password for these
files is the Directory Manager password
[root@qeos-237 upstream]# rpm -qa |egrep 'ipa-server|389-ds-base'
ipa-server-dns-4.5.0-15.el7.noarch
389-ds-base-libs-1.3.6.1-16.el7.x86_64
389-ds-base-snmp-1.3.6.1-16.el7.x86_64
389-ds-base-debuginfo-1.3.6.1-16.el7.x86_64
389-ds-base-1.3.6.1-16.el7.x86_64
ipa-server-4.5.0-15.el7.x86_64
ipa-server-common-4.5.0-15.el7.noarch

Marking it as Verified as Sanity only since it cannot be reproduced by QE.

Comment 6 errata-xmlrpc 2017-08-01 21:14:10 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:2086