Bug 474729 - Unindexed search does not get logged with "notes=U"
Unindexed search does not get logged with "notes=U"
Status: CLOSED CURRENTRELEASE
Product: 389
Classification: Community
Component: Database - Indexes/Searches (Show other bugs)
1.1.3
All Linux
high Severity medium
: ---
: ---
Assigned To: Noriko Hosoi
Chandrasekar Kannan
:
Depends On:
Blocks: 249650 FDS1.2.0
  Show dependency treegraph
 
Reported: 2008-12-04 19:39 EST by Noriko Hosoi
Modified: 2015-01-04 18:35 EST (History)
4 users (show)

See Also:
Fixed In Version: 8.1
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2009-04-29 19:08:31 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
cvs diff ldap/servers/slapd/back-ldbm/filterindex.c (10.80 KB, patch)
2008-12-04 19:46 EST, Noriko Hosoi
no flags Details | Diff
csv commit message (882 bytes, text/plain)
2008-12-05 16:05 EST, Noriko Hosoi
no flags Details

  None (edit)
Description Noriko Hosoi 2008-12-04 19:39:34 EST
Description of problem:
The result of search with the filter which includes unindexed attribute does not have "notes=U" at the end.
[..] conn=22 op=1 SRCH base="dc=example,dc=com" scope=2 filter="(description=*)" attrs=ALL
[..] conn=22 op=1 RESULT err=0 tag=101 nentries=5 etime=0
Comment 1 Noriko Hosoi 2008-12-04 19:46:39 EST
Created attachment 325766 [details]
cvs diff ldap/servers/slapd/back-ldbm/filterindex.c

Fix Description: In each <index-type> candidates function, check if the attribute is indexed by calling index_read_ext instead of index_read.  The function index_read_ext takes a variable to return whether the attribute is indexed or not.  Once it's determined, set SLAPI_OP_NOTE_UNINDEXED to the pblock, which is used when logging the result.
Comment 2 Noriko Hosoi 2008-12-04 20:09:57 EST
test cases:
uid has an equality type index.
dn: cn=uid, cn=index, cn=userRoot, cn=ldbm database, cn=plugins, cn=config
nsIndexType: eq

equality filter
ldapsearch -b "dc=example,dc=com" "(uid=tuser0)" dn
[..] conn=4 op=0 SRCH base="dc=example,dc=com" scope=2 filter="(uid=tuser0)" attrs="dn"
[..] conn=4 op=0 RESULT err=0 tag=101 nentries=1 etime=0

presence filter
/ldapsearch -b "dc=example,dc=com" "(uid=*)" dn
[..] conn=5 op=0 SRCH base="dc=example,dc=com" scope=2 filter="(uid=*)" attrs="dn"
[..] conn=5 op=0 RESULT err=0 tag=101 nentries=3 etime=0 notes=U

substring filter
ldapsearch -b "dc=example,dc=com" "(uid=*a*)" dn
[..] conn=6 op=0 SRCH base="dc=example,dc=com" scope=2 filter="(uid=*a*)" attrs="dn"
[..] conn=6 op=0 RESULT err=0 tag=101 nentries=0 etime=0 notes=U

ldapsearch -b "dc=example,dc=com" "(uid=*ab*)" dn
[..] conn=7 op=0 SRCH base="dc=example,dc=com" scope=2 filter="(uid=*ab*)" attrs="dn"
[..] conn=7 op=0 RESULT err=0 tag=101 nentries=0 etime=0 notes=U

ldapsearch -b "dc=example,dc=com" "(uid=*abc*)" dn
[..] conn=8 op=0 SRCH base="dc=example,dc=com" scope=2 filter="(uid=*abc*)" attrs="dn"
[..] conn=8 op=0 RESULT err=0 tag=101 nentries=0 etime=0 notes=U

cn has a presence type, an equality type, and a substring type index.
dn: cn=cn, cn=index, cn=userRoot, cn=ldbm database, cn=plugins, cn=config
nsIndexType: pres
nsIndexType: eq
nsIndexType: sub

When a filter is compositive, if any one of the sub filters is unindexed, it reports "notes=U".
ldapsearch -b "dc=example,dc=com" "(&(uid=*)(cn=*))" dn
[..] conn=9 op=0 SRCH base="dc=example,dc=com" scope=2 filter="(&(uid=*)(cn=*))" attrs="dn"
[..] conn=9 op=0 RESULT err=0 tag=101 nentries=3 etime=0 notes=U

There is a small exception.  If you switch the order of the filter, (cn=*) is evaluated first.  The search returns nentries=8 from my test server.  The value is compared with FILTER_TEST_THRESHOLD (== 10) and when it's smaller than FILTER_TEST_THRESHOLD, the rest of the filter is evaluated directly on the entries returned from the first part of the filter.  It eliminates the chance to check the rest of the filter is indexed or not.
ldapsearch -p 10389 -b "dc=example,dc=com" "(&(cn=*)(uid=*))" dn
[..] conn=2 op=0 SRCH base="dc=example,dc=com" scope=2 filter="(&(cn=*)(uid=*))" attrs="dn"
[..] conn=2 op=0 RESULT err=0 tag=101 nentries=3 etime=0
Comment 3 Nathan Kinder 2008-12-04 20:42:21 EST
(In reply to comment #2)
> There is a small exception.  If you switch the order of the filter, (cn=*) is
> evaluated first.  The search returns nentries=8 from my test server.  The value
> is compared with FILTER_TEST_THRESHOLD (== 10) and when it's smaller than
> FILTER_TEST_THRESHOLD, the rest of the filter is evaluated directly on the
> entries returned from the first part of the filter.  It eliminates the chance
> to check the rest of the filter is indexed or not.
> ldapsearch -p 10389 -b "dc=example,dc=com" "(&(cn=*)(uid=*))" dn
> [..] conn=2 op=0 SRCH base="dc=example,dc=com" scope=2
> filter="(&(cn=*)(uid=*))" attrs="dn"
> [..] conn=2 op=0 RESULT err=0 tag=101 nentries=3 etime=0

I think this is fine.  The index was indeed used for the search (for finding the 8 candidate entryids for the "uid=*" portion).  I think it is fair to say that this was an "indexed" search.
Comment 4 Noriko Hosoi 2008-12-05 16:05:25 EST
Created attachment 325908 [details]
csv commit message

Reviewed by Rich and Nathan (Thank you !!!)

Checked in into CVS HEAD.
Comment 5 Jenny Galipeau 2009-03-17 10:36:57 EDT
fix verfied RHEL 4 DS 8.1

[17/Mar/2009:10:32:19 -0400] conn=5 fd=65 slot=65 connection from 127.0.0.1 to 127.0.0.1
[17/Mar/2009:10:32:19 -0400] conn=5 op=0 BIND dn="cn=Directory Manager" method=128 version=3
[17/Mar/2009:10:32:19 -0400] conn=5 op=0 RESULT err=0 tag=97 nentries=0 etime=0 dn="cn=directory manager"
[17/Mar/2009:10:32:19 -0400] conn=5 op=1 SRCH base="dc=example,dc=com" scope=2 filter="(&(uid=*)(cn=*))" attrs="dn"
[17/Mar/2009:10:32:19 -0400] conn=5 op=1 RESULT err=0 tag=101 nentries=1 etime=0 notes=U
[17/Mar/2009:10:32:19 -0400] conn=5 op=2 UNBIND
[17/Mar/2009:10:32:19 -0400] conn=5 op=2 fd=65 closed - U1
[17/Mar/2009:10:33:44 -0400] conn=6 fd=65 slot=65 connection from 127.0.0.1 to 127.0.0.1
[17/Mar/2009:10:33:44 -0400] conn=6 op=0 BIND dn="cn=Directory Manager" method=128 version=3
[17/Mar/2009:10:33:44 -0400] conn=6 op=0 RESULT err=0 tag=97 nentries=0 etime=0 dn="cn=directory manager"
[17/Mar/2009:10:33:44 -0400] conn=6 op=1 SRCH base="dc=example,dc=com" scope=2 filter="(uid=*)" attrs="dn"
[17/Mar/2009:10:33:44 -0400] conn=6 op=1 RESULT err=0 tag=101 nentries=1 etime=0 notes=U
[17/Mar/2009:10:33:44 -0400] conn=6 op=2 UNBIND
[17/Mar/2009:10:33:44 -0400] conn=6 op=2 fd=65 closed - U1
[17/Mar/2009:10:34:34 -0400] conn=7 fd=65 slot=65 connection from 127.0.0.1 to 127.0.0.1
[17/Mar/2009:10:34:34 -0400] conn=7 op=0 BIND dn="cn=Directory Manager" method=128 version=3
[17/Mar/2009:10:34:34 -0400] conn=7 op=0 RESULT err=0 tag=97 nentries=0 etime=0 dn="cn=directory manager"
[17/Mar/2009:10:34:34 -0400] conn=7 op=1 SRCH base="dc=example,dc=com" scope=2 filter="(uid=*a*)" attrs="dn"
[17/Mar/2009:10:34:34 -0400] conn=7 op=1 RESULT err=0 tag=101 nentries=0 etime=0 notes=U
[17/Mar/2009:10:34:34 -0400] conn=7 op=2 UNBIND
[17/Mar/2009:10:34:34 -0400] conn=7 op=2 fd=65 closed - U1
[17/Mar/2009:10:34:45 -0400] conn=8 fd=65 slot=65 connection from 127.0.0.1 to 127.0.0.1
[17/Mar/2009:10:34:45 -0400] conn=8 op=0 BIND dn="cn=Directory Manager" method=128 version=3
[17/Mar/2009:10:34:45 -0400] conn=8 op=0 RESULT err=0 tag=97 nentries=0 etime=0 dn="cn=directory manager"
[17/Mar/2009:10:34:45 -0400] conn=8 op=1 SRCH base="dc=example,dc=com" scope=2 filter="(uid=*e*)" attrs="dn"
[17/Mar/2009:10:34:45 -0400] conn=8 op=1 RESULT err=0 tag=101 nentries=1 etime=0 notes=U
[17/Mar/2009:10:34:45 -0400] conn=8 op=2 UNBIND
[17/Mar/2009:10:34:45 -0400] conn=8 op=2 fd=65 closed - U1
[17/Mar/2009:10:35:24 -0400] conn=9 fd=65 slot=65 connection from 127.0.0.1 to 127.0.0.1
[17/Mar/2009:10:35:24 -0400] conn=9 op=0 BIND dn="cn=Directory Manager" method=128 version=3
[17/Mar/2009:10:35:24 -0400] conn=9 op=0 RESULT err=0 tag=97 nentries=0 etime=0 dn="cn=directory manager"
[17/Mar/2009:10:35:24 -0400] conn=9 op=1 SRCH base="dc=example,dc=com" scope=2 filter="(uid=*en*)" attrs="dn"
[17/Mar/2009:10:35:24 -0400] conn=9 op=1 RESULT err=0 tag=101 nentries=1 etime=0 notes=U
[17/Mar/2009:10:35:24 -0400] conn=9 op=2 UNBIND
[17/Mar/2009:10:35:24 -0400] conn=9 op=2 fd=65 closed - U1
[17/Mar/2009:10:35:40 -0400] conn=10 fd=65 slot=65 connection from 127.0.0.1 to 127.0.0.1
[17/Mar/2009:10:35:40 -0400] conn=10 op=0 BIND dn="cn=Directory Manager" method=128 version=3
[17/Mar/2009:10:35:40 -0400] conn=10 op=0 RESULT err=0 tag=97 nentries=0 etime=0 dn="cn=directory manager"
[17/Mar/2009:10:35:40 -0400] conn=10 op=1 SRCH base="dc=example,dc=com" scope=2 filter="(uid=*enn*)" attrs="dn"
[17/Mar/2009:10:35:40 -0400] conn=10 op=1 RESULT err=0 tag=101 nentries=1 etime=0 notes=U
[17/Mar/2009:10:35:40 -0400] conn=10 op=2 UNBIND
[17/Mar/2009:10:35:40 -0400] conn=10 op=2 fd=65 closed - U1
[17/Mar/2009:10:35:59 -0400] conn=11 fd=65 slot=65 connection from 127.0.0.1 to 127.0.0.1
[17/Mar/2009:10:35:59 -0400] conn=11 op=0 BIND dn="cn=Directory Manager" method=128 version=3
[17/Mar/2009:10:35:59 -0400] conn=11 op=0 RESULT err=0 tag=97 nentries=0 etime=0 dn="cn=directory manager"
[17/Mar/2009:10:35:59 -0400] conn=11 op=1 SRCH base="dc=example,dc=com" scope=2 filter="(uid=jenny)" attrs="dn"
[17/Mar/2009:10:35:59 -0400] conn=11 op=1 RESULT err=0 tag=101 nentries=1 etime=0
[17/Mar/2009:10:35:59 -0400] conn=11 op=2 UNBIND
[17/Mar/2009:10:35:59 -0400] conn=11 op=2 fd=65 closed - U1
[17/Mar/2009:10:36:26 -0400] conn=12 fd=65 slot=65 connection from 127.0.0.1 to 127.0.0.1
[17/Mar/2009:10:36:26 -0400] conn=12 op=0 BIND dn="cn=Directory Manager" method=128 version=3
[17/Mar/2009:10:36:26 -0400] conn=12 op=0 RESULT err=0 tag=97 nentries=0 etime=0 dn="cn=directory manager"
[17/Mar/2009:10:36:26 -0400] conn=12 op=1 SRCH base="dc=example,dc=com" scope=2 filter="(&(cn=*)(uid=*))" attrs="dn"
[17/Mar/2009:10:36:26 -0400] conn=12 op=1 RESULT err=0 tag=101 nentries=1 etime=0
[17/Mar/2009:10:36:26 -0400] conn=12 op=2 UNBIND
[17/Mar/2009:10:36:26 -0400] conn=12 op=2 fd=65 closed - U1
Comment 6 Chandrasekar Kannan 2009-04-29 19:08:31 EDT
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHEA-2009-0455.html

Note You need to log in before you can comment on or make changes to this bug.