Bug 474729

Summary: Unindexed search does not get logged with "notes=U"
Product: [Retired] 389 Reporter: Noriko Hosoi <nhosoi>
Component: Database - Indexes/SearchesAssignee: Noriko Hosoi <nhosoi>
Status: CLOSED CURRENTRELEASE QA Contact: Chandrasekar Kannan <ckannan>
Severity: medium Docs Contact:
Priority: high    
Version: 1.1.3CC: benl, jgalipea, nkinder, rmeggins
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: 8.1 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2009-04-29 23:08:31 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 249650, 493682    
Attachments:
Description Flags
cvs diff ldap/servers/slapd/back-ldbm/filterindex.c
none
csv commit message none

Description Noriko Hosoi 2008-12-05 00:39:34 UTC
Description of problem:
The result of search with the filter which includes unindexed attribute does not have "notes=U" at the end.
[..] conn=22 op=1 SRCH base="dc=example,dc=com" scope=2 filter="(description=*)" attrs=ALL
[..] conn=22 op=1 RESULT err=0 tag=101 nentries=5 etime=0

Comment 1 Noriko Hosoi 2008-12-05 00:46:39 UTC
Created attachment 325766 [details]
cvs diff ldap/servers/slapd/back-ldbm/filterindex.c

Fix Description: In each <index-type> candidates function, check if the attribute is indexed by calling index_read_ext instead of index_read.  The function index_read_ext takes a variable to return whether the attribute is indexed or not.  Once it's determined, set SLAPI_OP_NOTE_UNINDEXED to the pblock, which is used when logging the result.

Comment 2 Noriko Hosoi 2008-12-05 01:09:57 UTC
test cases:
uid has an equality type index.
dn: cn=uid, cn=index, cn=userRoot, cn=ldbm database, cn=plugins, cn=config
nsIndexType: eq

equality filter
ldapsearch -b "dc=example,dc=com" "(uid=tuser0)" dn
[..] conn=4 op=0 SRCH base="dc=example,dc=com" scope=2 filter="(uid=tuser0)" attrs="dn"
[..] conn=4 op=0 RESULT err=0 tag=101 nentries=1 etime=0

presence filter
/ldapsearch -b "dc=example,dc=com" "(uid=*)" dn
[..] conn=5 op=0 SRCH base="dc=example,dc=com" scope=2 filter="(uid=*)" attrs="dn"
[..] conn=5 op=0 RESULT err=0 tag=101 nentries=3 etime=0 notes=U

substring filter
ldapsearch -b "dc=example,dc=com" "(uid=*a*)" dn
[..] conn=6 op=0 SRCH base="dc=example,dc=com" scope=2 filter="(uid=*a*)" attrs="dn"
[..] conn=6 op=0 RESULT err=0 tag=101 nentries=0 etime=0 notes=U

ldapsearch -b "dc=example,dc=com" "(uid=*ab*)" dn
[..] conn=7 op=0 SRCH base="dc=example,dc=com" scope=2 filter="(uid=*ab*)" attrs="dn"
[..] conn=7 op=0 RESULT err=0 tag=101 nentries=0 etime=0 notes=U

ldapsearch -b "dc=example,dc=com" "(uid=*abc*)" dn
[..] conn=8 op=0 SRCH base="dc=example,dc=com" scope=2 filter="(uid=*abc*)" attrs="dn"
[..] conn=8 op=0 RESULT err=0 tag=101 nentries=0 etime=0 notes=U

cn has a presence type, an equality type, and a substring type index.
dn: cn=cn, cn=index, cn=userRoot, cn=ldbm database, cn=plugins, cn=config
nsIndexType: pres
nsIndexType: eq
nsIndexType: sub

When a filter is compositive, if any one of the sub filters is unindexed, it reports "notes=U".
ldapsearch -b "dc=example,dc=com" "(&(uid=*)(cn=*))" dn
[..] conn=9 op=0 SRCH base="dc=example,dc=com" scope=2 filter="(&(uid=*)(cn=*))" attrs="dn"
[..] conn=9 op=0 RESULT err=0 tag=101 nentries=3 etime=0 notes=U

There is a small exception.  If you switch the order of the filter, (cn=*) is evaluated first.  The search returns nentries=8 from my test server.  The value is compared with FILTER_TEST_THRESHOLD (== 10) and when it's smaller than FILTER_TEST_THRESHOLD, the rest of the filter is evaluated directly on the entries returned from the first part of the filter.  It eliminates the chance to check the rest of the filter is indexed or not.
ldapsearch -p 10389 -b "dc=example,dc=com" "(&(cn=*)(uid=*))" dn
[..] conn=2 op=0 SRCH base="dc=example,dc=com" scope=2 filter="(&(cn=*)(uid=*))" attrs="dn"
[..] conn=2 op=0 RESULT err=0 tag=101 nentries=3 etime=0

Comment 3 Nathan Kinder 2008-12-05 01:42:21 UTC
(In reply to comment #2)
> There is a small exception.  If you switch the order of the filter, (cn=*) is
> evaluated first.  The search returns nentries=8 from my test server.  The value
> is compared with FILTER_TEST_THRESHOLD (== 10) and when it's smaller than
> FILTER_TEST_THRESHOLD, the rest of the filter is evaluated directly on the
> entries returned from the first part of the filter.  It eliminates the chance
> to check the rest of the filter is indexed or not.
> ldapsearch -p 10389 -b "dc=example,dc=com" "(&(cn=*)(uid=*))" dn
> [..] conn=2 op=0 SRCH base="dc=example,dc=com" scope=2
> filter="(&(cn=*)(uid=*))" attrs="dn"
> [..] conn=2 op=0 RESULT err=0 tag=101 nentries=3 etime=0

I think this is fine.  The index was indeed used for the search (for finding the 8 candidate entryids for the "uid=*" portion).  I think it is fair to say that this was an "indexed" search.

Comment 4 Noriko Hosoi 2008-12-05 21:05:25 UTC
Created attachment 325908 [details]
csv commit message

Reviewed by Rich and Nathan (Thank you !!!)

Checked in into CVS HEAD.

Comment 5 Jenny Severance 2009-03-17 14:36:57 UTC
fix verfied RHEL 4 DS 8.1

[17/Mar/2009:10:32:19 -0400] conn=5 fd=65 slot=65 connection from 127.0.0.1 to 127.0.0.1
[17/Mar/2009:10:32:19 -0400] conn=5 op=0 BIND dn="cn=Directory Manager" method=128 version=3
[17/Mar/2009:10:32:19 -0400] conn=5 op=0 RESULT err=0 tag=97 nentries=0 etime=0 dn="cn=directory manager"
[17/Mar/2009:10:32:19 -0400] conn=5 op=1 SRCH base="dc=example,dc=com" scope=2 filter="(&(uid=*)(cn=*))" attrs="dn"
[17/Mar/2009:10:32:19 -0400] conn=5 op=1 RESULT err=0 tag=101 nentries=1 etime=0 notes=U
[17/Mar/2009:10:32:19 -0400] conn=5 op=2 UNBIND
[17/Mar/2009:10:32:19 -0400] conn=5 op=2 fd=65 closed - U1
[17/Mar/2009:10:33:44 -0400] conn=6 fd=65 slot=65 connection from 127.0.0.1 to 127.0.0.1
[17/Mar/2009:10:33:44 -0400] conn=6 op=0 BIND dn="cn=Directory Manager" method=128 version=3
[17/Mar/2009:10:33:44 -0400] conn=6 op=0 RESULT err=0 tag=97 nentries=0 etime=0 dn="cn=directory manager"
[17/Mar/2009:10:33:44 -0400] conn=6 op=1 SRCH base="dc=example,dc=com" scope=2 filter="(uid=*)" attrs="dn"
[17/Mar/2009:10:33:44 -0400] conn=6 op=1 RESULT err=0 tag=101 nentries=1 etime=0 notes=U
[17/Mar/2009:10:33:44 -0400] conn=6 op=2 UNBIND
[17/Mar/2009:10:33:44 -0400] conn=6 op=2 fd=65 closed - U1
[17/Mar/2009:10:34:34 -0400] conn=7 fd=65 slot=65 connection from 127.0.0.1 to 127.0.0.1
[17/Mar/2009:10:34:34 -0400] conn=7 op=0 BIND dn="cn=Directory Manager" method=128 version=3
[17/Mar/2009:10:34:34 -0400] conn=7 op=0 RESULT err=0 tag=97 nentries=0 etime=0 dn="cn=directory manager"
[17/Mar/2009:10:34:34 -0400] conn=7 op=1 SRCH base="dc=example,dc=com" scope=2 filter="(uid=*a*)" attrs="dn"
[17/Mar/2009:10:34:34 -0400] conn=7 op=1 RESULT err=0 tag=101 nentries=0 etime=0 notes=U
[17/Mar/2009:10:34:34 -0400] conn=7 op=2 UNBIND
[17/Mar/2009:10:34:34 -0400] conn=7 op=2 fd=65 closed - U1
[17/Mar/2009:10:34:45 -0400] conn=8 fd=65 slot=65 connection from 127.0.0.1 to 127.0.0.1
[17/Mar/2009:10:34:45 -0400] conn=8 op=0 BIND dn="cn=Directory Manager" method=128 version=3
[17/Mar/2009:10:34:45 -0400] conn=8 op=0 RESULT err=0 tag=97 nentries=0 etime=0 dn="cn=directory manager"
[17/Mar/2009:10:34:45 -0400] conn=8 op=1 SRCH base="dc=example,dc=com" scope=2 filter="(uid=*e*)" attrs="dn"
[17/Mar/2009:10:34:45 -0400] conn=8 op=1 RESULT err=0 tag=101 nentries=1 etime=0 notes=U
[17/Mar/2009:10:34:45 -0400] conn=8 op=2 UNBIND
[17/Mar/2009:10:34:45 -0400] conn=8 op=2 fd=65 closed - U1
[17/Mar/2009:10:35:24 -0400] conn=9 fd=65 slot=65 connection from 127.0.0.1 to 127.0.0.1
[17/Mar/2009:10:35:24 -0400] conn=9 op=0 BIND dn="cn=Directory Manager" method=128 version=3
[17/Mar/2009:10:35:24 -0400] conn=9 op=0 RESULT err=0 tag=97 nentries=0 etime=0 dn="cn=directory manager"
[17/Mar/2009:10:35:24 -0400] conn=9 op=1 SRCH base="dc=example,dc=com" scope=2 filter="(uid=*en*)" attrs="dn"
[17/Mar/2009:10:35:24 -0400] conn=9 op=1 RESULT err=0 tag=101 nentries=1 etime=0 notes=U
[17/Mar/2009:10:35:24 -0400] conn=9 op=2 UNBIND
[17/Mar/2009:10:35:24 -0400] conn=9 op=2 fd=65 closed - U1
[17/Mar/2009:10:35:40 -0400] conn=10 fd=65 slot=65 connection from 127.0.0.1 to 127.0.0.1
[17/Mar/2009:10:35:40 -0400] conn=10 op=0 BIND dn="cn=Directory Manager" method=128 version=3
[17/Mar/2009:10:35:40 -0400] conn=10 op=0 RESULT err=0 tag=97 nentries=0 etime=0 dn="cn=directory manager"
[17/Mar/2009:10:35:40 -0400] conn=10 op=1 SRCH base="dc=example,dc=com" scope=2 filter="(uid=*enn*)" attrs="dn"
[17/Mar/2009:10:35:40 -0400] conn=10 op=1 RESULT err=0 tag=101 nentries=1 etime=0 notes=U
[17/Mar/2009:10:35:40 -0400] conn=10 op=2 UNBIND
[17/Mar/2009:10:35:40 -0400] conn=10 op=2 fd=65 closed - U1
[17/Mar/2009:10:35:59 -0400] conn=11 fd=65 slot=65 connection from 127.0.0.1 to 127.0.0.1
[17/Mar/2009:10:35:59 -0400] conn=11 op=0 BIND dn="cn=Directory Manager" method=128 version=3
[17/Mar/2009:10:35:59 -0400] conn=11 op=0 RESULT err=0 tag=97 nentries=0 etime=0 dn="cn=directory manager"
[17/Mar/2009:10:35:59 -0400] conn=11 op=1 SRCH base="dc=example,dc=com" scope=2 filter="(uid=jenny)" attrs="dn"
[17/Mar/2009:10:35:59 -0400] conn=11 op=1 RESULT err=0 tag=101 nentries=1 etime=0
[17/Mar/2009:10:35:59 -0400] conn=11 op=2 UNBIND
[17/Mar/2009:10:35:59 -0400] conn=11 op=2 fd=65 closed - U1
[17/Mar/2009:10:36:26 -0400] conn=12 fd=65 slot=65 connection from 127.0.0.1 to 127.0.0.1
[17/Mar/2009:10:36:26 -0400] conn=12 op=0 BIND dn="cn=Directory Manager" method=128 version=3
[17/Mar/2009:10:36:26 -0400] conn=12 op=0 RESULT err=0 tag=97 nentries=0 etime=0 dn="cn=directory manager"
[17/Mar/2009:10:36:26 -0400] conn=12 op=1 SRCH base="dc=example,dc=com" scope=2 filter="(&(cn=*)(uid=*))" attrs="dn"
[17/Mar/2009:10:36:26 -0400] conn=12 op=1 RESULT err=0 tag=101 nentries=1 etime=0
[17/Mar/2009:10:36:26 -0400] conn=12 op=2 UNBIND
[17/Mar/2009:10:36:26 -0400] conn=12 op=2 fd=65 closed - U1

Comment 6 Chandrasekar Kannan 2009-04-29 23:08:31 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHEA-2009-0455.html