Bug 1044133 - Indexed search with filter containing '&' and "!" with attribute subtypes gives wrong result
Summary: Indexed search with filter containing '&' and "!" with attribute subtypes gi...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: 389-ds-base
Version: 7.0
Hardware: Unspecified
OS: Unspecified
low
unspecified
Target Milestone: rc
: ---
Assignee: Rich Megginson
QA Contact: Viktor Ashirov
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2013-12-17 21:12 UTC by Nathan Kinder
Modified: 2015-03-05 09:31 UTC (History)
2 users (show)

Fixed In Version: 389-ds-base-1.3.3.1-1.el7
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-03-05 09:31:38 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2015:0416 normal SHIPPED_LIVE Important: 389-ds-base security, bug fix, and enhancement update 2015-03-05 14:26:33 UTC

Description Nathan Kinder 2013-12-17 21:12:26 UTC
This bug is created as a clone of upstream ticket:
https://fedorahosted.org/389/ticket/47313


Server version: 389DS v1.2.10.26 on CentOS 5.x x86_64

I have not tested it on 1.2.11 and 1.3.0.

How to reproduce:

Add a new multi-value attribute to the schema, say, '''X-Flags'''. Index this attribute on '''presence''' and '''equality''' (in my tests '''no substring index''' was present).

Maybe a test with one of the pre-defined attributes also works (with the same index types)

Create an entry containing this attribute with a subtype, e.g.:

uid=login,dc=example,dc=com
objectClass: top
objectClass: person
objectClass: organizationalPerson
objectClass: inetOrgPerson
objectClass: X-Misc
uid: login
...
X-Flags;en: test

Now make a search using the following filter:
(&(objectClass=inetOrgPerson)(!(X-Flags;fr=test)))

The previously created entry should be returned by this search. However it is not returned. If we eliminate the indexing on X-Flags by changing this search to a substring search (adding "*" to test)
(&(objectClass=inetOrgPerson)(!(X-Flags;fr=test*)))
then the entry is returned. So the problem is obviously in the usage of index. If i disable the indexes on this attribute, all the searches return correct results.



However, simple searches without '&' work just fine, both with and without indexes:
(!(X-Flags;fr=test)) returns the entry correctly.

Comment 2 Viktor Ashirov 2015-01-20 16:28:13 UTC
$ rpm -qa | grep 389
389-ds-base-debuginfo-1.3.3.1-11.el7.x86_64
389-ds-base-libs-1.3.3.1-11.el7.x86_64
389-ds-base-1.3.3.1-11.el7.x86_64

[1] Imported test ldif:
$ ldif2db -n userRoot -i `pwd`/bug-search.ldif
importing data ...
[20/Jan/2015:16:55:18 +0100] - WARNING: Import is running with nsslapd-db-private-import-mem on; No other process is allowed to access the database
[20/Jan/2015:16:55:18 +0100] - check_and_set_import_cache: pagesize: 4096, pages: 471102, procpages: 54346
[20/Jan/2015:16:55:18 +0100] - Import allocates 753760KB import cache.
[20/Jan/2015:16:55:18 +0100] - import userRoot: Beginning import job...
[20/Jan/2015:16:55:18 +0100] - import userRoot: Index buffering enabled with bucket size 100
[20/Jan/2015:16:55:18 +0100] - import userRoot: Processing file "/home/vashirov/bugs/1044133/bug-search.ldif"
[20/Jan/2015:16:55:18 +0100] - import userRoot: Finished scanning file "/home/vashirov/bugs/1044133/bug-search.ldif" (21 entries)
[20/Jan/2015:16:55:19 +0100] - import userRoot: Workers finished; cleaning up...
[20/Jan/2015:16:55:19 +0100] - import userRoot: Workers cleaned up.
[20/Jan/2015:16:55:19 +0100] - import userRoot: Cleaning up producer thread...
[20/Jan/2015:16:55:19 +0100] - import userRoot: Indexing complete.  Post-processing...
[20/Jan/2015:16:55:19 +0100] - import userRoot: Generating numsubordinates (this may take several minutes to complete)...
[20/Jan/2015:16:55:19 +0100] - import userRoot: Generating numSubordinates complete.
[20/Jan/2015:16:55:19 +0100] - import userRoot: Gathering ancestorid non-leaf IDs...
[20/Jan/2015:16:55:19 +0100] - import userRoot: Finished gathering ancestorid non-leaf IDs.
[20/Jan/2015:16:55:19 +0100] - import userRoot: Creating ancestorid index (new idl)...
[20/Jan/2015:16:55:19 +0100] - import userRoot: Created ancestorid index (new idl).
[20/Jan/2015:16:55:19 +0100] - import userRoot: Flushing caches...
[20/Jan/2015:16:55:19 +0100] - import userRoot: Closing files...
[20/Jan/2015:16:55:19 +0100] - All database threads now stopped
[20/Jan/2015:16:55:20 +0100] - import userRoot: Import complete.  Processed 21 entries in 2 seconds. (10.50 entries/sec)

[2] Added X-Flags and X-Misc to 99user.ldif. Then, restarted the server.
dn: cn=schema
#
attributeTypes: ( 2.16.840.1.113730.3.1.123456.1
 NAME 'X-Flags'
 DESC 'Test attribute type'
 SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 X-ORIGIN 'test' )
#
objectClasses: ( 2.16.840.1.113730.3.2.123456.2
  NAME 'X-Misc'
  SUP inetOrgPerson
  STRUCTURAL
  MAY ( X-Flags ) X-ORIGIN 'test' )

$ sudo systemctl start dirsrv.target

[3] Added index for X-Flags:
$ ldapmodify -D "cn=Directory Manager" -w Secret123 << EOF
dn: cn=X-Flags,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
changetype: add
objectClass: top
objectClass: nsIndex
cn: X-Flags
nsSystemIndex: false
nsIndexType: eq
nsIndexType: pres
EOF
adding new entry "cn=X-Flags,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config"

[4] Added an entry:
$ ldapmodify -D "cn=Directory Manager" -w Secret123 << EOF
dn: uid=tuser0,dc=id,dc=polytechnique,dc=edu
changetype: add
objectClass: top
objectClass: person
objectClass: organizationalPerson
objectClass: inetOrgPerson
objectClass: X-Misc
cn: test user0
sn: user0
uid: tuser0
givenName: test
roomNumber: 1000
mail: tuser0@example.com
userPassword: {CLEAR}tuser0
X-Flags;en: test
EOF
adding new entry "uid=tuser0,dc=id,dc=polytechnique,dc=edu"

[5] Make sure the entry is indexed in X-Flags.db:
$ sudo dbscan -f  /var/lib/dirsrv/slapd-rhel7/db/userRoot/X-Flags.db -r
+                                       
	22 
=test                                   
	22 

[6] Search and check whether the entry "uid=consult" appears in the result:
$ ldapsearch -D "cn=Directory Manager" -w Secret123 -b "ou=Comptes Mail,ou=Comptes generiques,ou=Utilisateurs,dc=id,dc=polytechnique,dc=edu" '(&(objectClass=inetOrgPerson)(!(X-Flags;fr=test)))' uid X-Flags
# extended LDIF
#
# LDAPv3
# base <ou=Comptes Mail,ou=Comptes generiques,ou=Utilisateurs,dc=id,dc=polytechnique,dc=edu> with scope subtree
# filter: (&(objectClass=inetOrgPerson)(!(X-Flags;fr=test)))
# requesting: uid X-Flags 
#

# phi, Comptes Mail, Comptes generiques, Utilisateurs, id.polytechnique.edu
dn: uid=phi,ou=Comptes Mail,ou=Comptes generiques,ou=Utilisateurs,dc=id,dc=pol
 ytechnique,dc=edu
uid: phi

# consult, Comptes Mail, Comptes generiques, Utilisateurs, id.polytechnique.e
 du
dn: uid=consult,ou=Comptes Mail,ou=Comptes generiques,ou=Utilisateurs,dc=id,dc
 =polytechnique,dc=edu
uid: consult
X-Flags;en: test

# trex.meca, Comptes Mail, Comptes generiques, Utilisateurs, id.polytechnique
 .edu
dn: uid=trex.meca,ou=Comptes Mail,ou=Comptes generiques,ou=Utilisateurs,dc=id,
 dc=polytechnique,dc=edu
uid: trex.meca

# tuchka, Comptes Mail, Comptes generiques, Utilisateurs, id.polytechnique.ed
 u
dn: uid=tuchka,ou=Comptes Mail,ou=Comptes generiques,ou=Utilisateurs,dc=id,dc=
 polytechnique,dc=edu
uid: tuchka

# search result
search: 2
result: 0 Success

# numResponses: 5
# numEntries: 4


$ ldapsearch -D "cn=Directory Manager" -w Secret123 -b "ou=Comptes Mail,ou=Comptes generiques,ou=Utilisateurs,dc=id,dc=polytechnique,dc=edu" '(&(objectClass=inetOrgPerson)(!(X-Flags;fr=test*)))' uid X-Flags
# extended LDIF
#
# LDAPv3
# base <ou=Comptes Mail,ou=Comptes generiques,ou=Utilisateurs,dc=id,dc=polytechnique,dc=edu> with scope subtree
# filter: (&(objectClass=inetOrgPerson)(!(X-Flags;fr=test*)))
# requesting: uid X-Flags 
#

# phi, Comptes Mail, Comptes generiques, Utilisateurs, id.polytechnique.edu
dn: uid=phi,ou=Comptes Mail,ou=Comptes generiques,ou=Utilisateurs,dc=id,dc=pol
 ytechnique,dc=edu
uid: phi

# consult, Comptes Mail, Comptes generiques, Utilisateurs, id.polytechnique.e
 du
dn: uid=consult,ou=Comptes Mail,ou=Comptes generiques,ou=Utilisateurs,dc=id,dc
 =polytechnique,dc=edu
uid: consult
X-Flags;en: test

# trex.meca, Comptes Mail, Comptes generiques, Utilisateurs, id.polytechnique
 .edu
dn: uid=trex.meca,ou=Comptes Mail,ou=Comptes generiques,ou=Utilisateurs,dc=id,
 dc=polytechnique,dc=edu
uid: trex.meca

# tuchka, Comptes Mail, Comptes generiques, Utilisateurs, id.polytechnique.ed
 u
dn: uid=tuchka,ou=Comptes Mail,ou=Comptes generiques,ou=Utilisateurs,dc=id,dc=
 polytechnique,dc=edu
uid: tuchka

# search result
search: 2
result: 0 Success

# numResponses: 5
# numEntries: 4

Entry appears in both searches.

Marking as VERIFIED.

Comment 4 errata-xmlrpc 2015-03-05 09:31:38 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2015-0416.html


Note You need to log in before you can comment on or make changes to this bug.