Bug 1044133

Summary:	Indexed search with filter containing '&' and "!" with attribute subtypes gives wrong result
Product:	Red Hat Enterprise Linux 7	Reporter:	Nathan Kinder <nkinder>
Component:	389-ds-base	Assignee:	Rich Megginson <rmeggins>
Status:	CLOSED ERRATA	QA Contact:	Viktor Ashirov <vashirov>
Severity:	unspecified	Docs Contact:
Priority:	low
Version:	7.0	CC:	nhosoi, vashirov
Target Milestone:	rc
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	389-ds-base-1.3.3.1-1.el7	Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2015-03-05 09:31:38 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Nathan Kinder 2013-12-17 21:12:26 UTC

This bug is created as a clone of upstream ticket:
https://fedorahosted.org/389/ticket/47313


Server version: 389DS v1.2.10.26 on CentOS 5.x x86_64

I have not tested it on 1.2.11 and 1.3.0.

How to reproduce:

Add a new multi-value attribute to the schema, say, '''X-Flags'''. Index this attribute on '''presence''' and '''equality''' (in my tests '''no substring index''' was present).

Maybe a test with one of the pre-defined attributes also works (with the same index types)

Create an entry containing this attribute with a subtype, e.g.:

uid=login,dc=example,dc=com
objectClass: top
objectClass: person
objectClass: organizationalPerson
objectClass: inetOrgPerson
objectClass: X-Misc
uid: login
...
X-Flags;en: test

Now make a search using the following filter:
(&(objectClass=inetOrgPerson)(!(X-Flags;fr=test)))

The previously created entry should be returned by this search. However it is not returned. If we eliminate the indexing on X-Flags by changing this search to a substring search (adding "*" to test)
(&(objectClass=inetOrgPerson)(!(X-Flags;fr=test*)))
then the entry is returned. So the problem is obviously in the usage of index. If i disable the indexes on this attribute, all the searches return correct results.



However, simple searches without '&' work just fine, both with and without indexes:
(!(X-Flags;fr=test)) returns the entry correctly.

Comment 2 Viktor Ashirov 2015-01-20 16:28:13 UTC

$ rpm -qa | grep 389
389-ds-base-debuginfo-1.3.3.1-11.el7.x86_64
389-ds-base-libs-1.3.3.1-11.el7.x86_64
389-ds-base-1.3.3.1-11.el7.x86_64

[1] Imported test ldif:
$ ldif2db -n userRoot -i `pwd`/bug-search.ldif
importing data ...
[20/Jan/2015:16:55:18 +0100] - WARNING: Import is running with nsslapd-db-private-import-mem on; No other process is allowed to access the database
[20/Jan/2015:16:55:18 +0100] - check_and_set_import_cache: pagesize: 4096, pages: 471102, procpages: 54346
[20/Jan/2015:16:55:18 +0100] - Import allocates 753760KB import cache.
[20/Jan/2015:16:55:18 +0100] - import userRoot: Beginning import job...
[20/Jan/2015:16:55:18 +0100] - import userRoot: Index buffering enabled with bucket size 100
[20/Jan/2015:16:55:18 +0100] - import userRoot: Processing file "/home/vashirov/bugs/1044133/bug-search.ldif"
[20/Jan/2015:16:55:18 +0100] - import userRoot: Finished scanning file "/home/vashirov/bugs/1044133/bug-search.ldif" (21 entries)
[20/Jan/2015:16:55:19 +0100] - import userRoot: Workers finished; cleaning up...
[20/Jan/2015:16:55:19 +0100] - import userRoot: Workers cleaned up.
[20/Jan/2015:16:55:19 +0100] - import userRoot: Cleaning up producer thread...
[20/Jan/2015:16:55:19 +0100] - import userRoot: Indexing complete.  Post-processing...
[20/Jan/2015:16:55:19 +0100] - import userRoot: Generating numsubordinates (this may take several minutes to complete)...
[20/Jan/2015:16:55:19 +0100] - import userRoot: Generating numSubordinates complete.
[20/Jan/2015:16:55:19 +0100] - import userRoot: Gathering ancestorid non-leaf IDs...
[20/Jan/2015:16:55:19 +0100] - import userRoot: Finished gathering ancestorid non-leaf IDs.
[20/Jan/2015:16:55:19 +0100] - import userRoot: Creating ancestorid index (new idl)...
[20/Jan/2015:16:55:19 +0100] - import userRoot: Created ancestorid index (new idl).
[20/Jan/2015:16:55:19 +0100] - import userRoot: Flushing caches...
[20/Jan/2015:16:55:19 +0100] - import userRoot: Closing files...
[20/Jan/2015:16:55:19 +0100] - All database threads now stopped
[20/Jan/2015:16:55:20 +0100] - import userRoot: Import complete.  Processed 21 entries in 2 seconds. (10.50 entries/sec)

[2] Added X-Flags and X-Misc to 99user.ldif. Then, restarted the server.
dn: cn=schema
#
attributeTypes: ( 2.16.840.1.113730.3.1.123456.1
 NAME 'X-Flags'
 DESC 'Test attribute type'
 SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 X-ORIGIN 'test' )
#
objectClasses: ( 2.16.840.1.113730.3.2.123456.2
  NAME 'X-Misc'
  SUP inetOrgPerson
  STRUCTURAL
  MAY ( X-Flags ) X-ORIGIN 'test' )

$ sudo systemctl start dirsrv.target

[3] Added index for X-Flags:
$ ldapmodify -D "cn=Directory Manager" -w Secret123 << EOF
dn: cn=X-Flags,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
changetype: add
objectClass: top
objectClass: nsIndex
cn: X-Flags
nsSystemIndex: false
nsIndexType: eq
nsIndexType: pres
EOF
adding new entry "cn=X-Flags,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config"

[4] Added an entry:
$ ldapmodify -D "cn=Directory Manager" -w Secret123 << EOF
dn: uid=tuser0,dc=id,dc=polytechnique,dc=edu
changetype: add
objectClass: top
objectClass: person
objectClass: organizationalPerson
objectClass: inetOrgPerson
objectClass: X-Misc
cn: test user0
sn: user0
uid: tuser0
givenName: test
roomNumber: 1000
mail: tuser0
userPassword: {CLEAR}tuser0
X-Flags;en: test
EOF
adding new entry "uid=tuser0,dc=id,dc=polytechnique,dc=edu"

[5] Make sure the entry is indexed in X-Flags.db:
$ sudo dbscan -f  /var/lib/dirsrv/slapd-rhel7/db/userRoot/X-Flags.db -r
+                                       
	22 
=test                                   
	22 

[6] Search and check whether the entry "uid=consult" appears in the result:
$ ldapsearch -D "cn=Directory Manager" -w Secret123 -b "ou=Comptes Mail,ou=Comptes generiques,ou=Utilisateurs,dc=id,dc=polytechnique,dc=edu" '(&(objectClass=inetOrgPerson)(!(X-Flags;fr=test)))' uid X-Flags
# extended LDIF
#
# LDAPv3
# base <ou=Comptes Mail,ou=Comptes generiques,ou=Utilisateurs,dc=id,dc=polytechnique,dc=edu> with scope subtree
# filter: (&(objectClass=inetOrgPerson)(!(X-Flags;fr=test)))
# requesting: uid X-Flags 
#

# phi, Comptes Mail, Comptes generiques, Utilisateurs, id.polytechnique.edu
dn: uid=phi,ou=Comptes Mail,ou=Comptes generiques,ou=Utilisateurs,dc=id,dc=pol
 ytechnique,dc=edu
uid: phi

# consult, Comptes Mail, Comptes generiques, Utilisateurs, id.polytechnique.e
 du
dn: uid=consult,ou=Comptes Mail,ou=Comptes generiques,ou=Utilisateurs,dc=id,dc
 =polytechnique,dc=edu
uid: consult
X-Flags;en: test

# trex.meca, Comptes Mail, Comptes generiques, Utilisateurs, id.polytechnique
 .edu
dn: uid=trex.meca,ou=Comptes Mail,ou=Comptes generiques,ou=Utilisateurs,dc=id,
 dc=polytechnique,dc=edu
uid: trex.meca

# tuchka, Comptes Mail, Comptes generiques, Utilisateurs, id.polytechnique.ed
 u
dn: uid=tuchka,ou=Comptes Mail,ou=Comptes generiques,ou=Utilisateurs,dc=id,dc=
 polytechnique,dc=edu
uid: tuchka

# search result
search: 2
result: 0 Success

# numResponses: 5
# numEntries: 4


$ ldapsearch -D "cn=Directory Manager" -w Secret123 -b "ou=Comptes Mail,ou=Comptes generiques,ou=Utilisateurs,dc=id,dc=polytechnique,dc=edu" '(&(objectClass=inetOrgPerson)(!(X-Flags;fr=test*)))' uid X-Flags
# extended LDIF
#
# LDAPv3
# base <ou=Comptes Mail,ou=Comptes generiques,ou=Utilisateurs,dc=id,dc=polytechnique,dc=edu> with scope subtree
# filter: (&(objectClass=inetOrgPerson)(!(X-Flags;fr=test*)))
# requesting: uid X-Flags 
#

# phi, Comptes Mail, Comptes generiques, Utilisateurs, id.polytechnique.edu
dn: uid=phi,ou=Comptes Mail,ou=Comptes generiques,ou=Utilisateurs,dc=id,dc=pol
 ytechnique,dc=edu
uid: phi

# consult, Comptes Mail, Comptes generiques, Utilisateurs, id.polytechnique.e
 du
dn: uid=consult,ou=Comptes Mail,ou=Comptes generiques,ou=Utilisateurs,dc=id,dc
 =polytechnique,dc=edu
uid: consult
X-Flags;en: test

# trex.meca, Comptes Mail, Comptes generiques, Utilisateurs, id.polytechnique
 .edu
dn: uid=trex.meca,ou=Comptes Mail,ou=Comptes generiques,ou=Utilisateurs,dc=id,
 dc=polytechnique,dc=edu
uid: trex.meca

# tuchka, Comptes Mail, Comptes generiques, Utilisateurs, id.polytechnique.ed
 u
dn: uid=tuchka,ou=Comptes Mail,ou=Comptes generiques,ou=Utilisateurs,dc=id,dc=
 polytechnique,dc=edu
uid: tuchka

# search result
search: 2
result: 0 Success

# numResponses: 5
# numEntries: 4

Entry appears in both searches.

Marking as VERIFIED.

Comment 4 errata-xmlrpc 2015-03-05 09:31:38 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2015-0416.html