Bug 964250 (PM-310, PRODMGT-310)

Summary: RFE: PRODMGT-310 Implement page control in LDAP group search page
Product: [JBoss] JBoss Operations Network Reporter: Charles Crouch <ccrouch>
Component: Core ServerAssignee: Simeon Pinder <spinder>
Status: CLOSED CURRENTRELEASE QA Contact: Mike Foley <mfoley>
Severity: high Docs Contact:
Priority: urgent    
Version: JON 3.1.1CC: hbrock, loleary, skondkar
Target Milestone: ER01   
Target Release: JON 3.2.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Charles Crouch 2013-05-17 16:18:31 UTC
This BZ is to track the implementation of PRODMGT-310

Simeon has already put together a wiki page discussing this topic:
https://docs.jboss.org/author/display/RHQ/Supporting+LDAP+query+page+control

A couple of key things to make sure we consider:

1) The test scenario by which we can validate that we have a problem in JON312 and it is fixed in JON320. Larry effectively supplies this in the jira: "At a high-level, a target group will need to be determined and the LDAP server's maximum result size [on the actual LDAP server itself] should be set to a very low number. So low in fact that the target group will not be returned in the first or even second batch of results. With the existing implementation, this will result in the target group never being seen in the LDAP group-to-role mapping dialog. Once this feature has been implemented, the target group will simply appear without the user having to do anything."

2) Performance: we need to make sure that we handle large paging sizes and large return sets gracefully, i.e. in the 1000's. The jira issues talks of upto 20k groups.

3) Supported platforms. Support and testing for Active Directory is required, support and testing for Red Hat IdM would be great if the underlying LDAP server supports this feature.

Comment 1 Charles Crouch 2013-05-17 17:08:56 UTC
4) Demo and testcase review with QE.

Comment 2 JBoss JIRA Server 2013-07-26 13:21:35 UTC
Thomas Heute <theute> made a comment on jira PRODMGT-310

Note for myself: A wiki has been written: https://docs.jboss.org/author/display/RHQ/Supporting+LDAP+query+page+control will check with Simeon

Comment 3 Simeon Pinder 2013-07-26 16:51:10 UTC
Moving this to ASSIGNED. This work has already been done and was included in the RHQ 4.8 release.

Comment 4 Simeon Pinder 2013-07-29 06:09:33 UTC
Commits where this is fixed in master: 
97dbbbfe
44af5cbe
ff58a992
ec2d4a65
03b81154
54043a51

The fix:
As detailed in https://docs.jboss.org/author/display/RHQ/Supporting+LDAP+query+page+control, the motivations for RFC 2696 are varied, but the fix was to:
i) provide the ability for the JON server to enable Query paging(disabled by default).  Most ldap servers don't handle rfc 2696 properly. 
ii)provide the RHQ admin with the ability to specify how many results should be in each page. Defaults to 1K as is default on Active Directory.
iii)Modify the JON + LDAP integration to send the page controls to the external LDAP servers and to iterate over the results until done.
iv) Best results achieved when page size is set to largest page size supported by LDAP server.

As requested in the description, 
1) was just addressed. 
2) Will be tested in more depth by QE, but I've loaded 20 K groups with only small delays from local ldap servers with paging enabled. 
3) Supported platforms: AD and Redhat Directory Server.  At the time of patching there were a few issues with getting Directory server to handle RFC 2696 as consistently as MS did.  I worked with one of the developers to confirm that these issues were being fixed for the next release. 
4) I also did a test case review with Sunil Kondar at 7/9/13.

Comment 5 Simeon Pinder 2013-07-29 06:36:35 UTC
Regarding Directory Server support for RFC:
--- excerpts from conversations with Developments 6/6/13
you will only get the first 9 entries returned in this case. This is due 
to this bug in 389-ds-base where the sizelimit is applied to the overall 
search (not per page):

     https://fedorahosted.org/389/ticket/47347

This was fixed just over a month ago, but it won't be available until 
RHEL 6.5:

     https://bugzilla.redhat.com/show_bug.cgi?id=957864

Official upstream builds containing this fix are not available, but I 
can install a nightly build with the fix to test your code against.

Comment 6 Simeon Pinder 2013-07-29 06:37:26 UTC
Moving this to MODIFIED for testing with next brew build of 3.2.x.

Comment 7 JBoss JIRA Server 2013-07-29 15:51:19 UTC
Larry O'Leary <loleary> made a comment on jira PRODMGT-310

This feature has been accepted and should be available in the 3.2 release.

Comment 8 Larry O'Leary 2013-09-06 14:32:12 UTC
As this is MODIFIED or ON_QA, setting milestone to ER1.

Comment 9 Mike Foley 2013-09-13 19:19:53 UTC
QE verified

https://engineering.redhat.com/trac/jon/ticket/669