Bug 559315 - Searching some attributes are now case sensitive when they were previously case-insensitive
Summary: Searching some attributes are now case sensitive when they were previously ca...
Status: CLOSED CURRENTRELEASE
Alias: None
Product: 389
Classification: Retired
Component: Schema
Version: 1.3.0
Hardware: All
OS: Linux
low
high
Target Milestone: ---
Assignee: Rich Megginson
QA Contact: Viktor Ashirov
URL:
Whiteboard:
Keywords:
: 559323 609321 (view as bug list)
Depends On:
Blocks: 389_1.2.6 639035
TreeView+ depends on / blocked
 
Reported: 2010-01-27 18:00 UTC by Terry Soucy
Modified: 2015-12-07 16:50 UTC (History)
4 users (show)

(edit)
Clone Of:
(edit)
Last Closed: 2015-12-07 16:50:21 UTC


Attachments (Terms of Use)
python ldap schema lookup script (2.41 KB, text/plain)
2010-01-29 18:51 UTC, Rich Megginson
no flags Details
patch1 - do not use syntax plugins directly (68.24 KB, patch)
2010-02-16 23:00 UTC, Rich Megginson
nhosoi: review+
Details | Diff
patch2- wrap new style matching rule plugins (17.45 KB, patch)
2010-02-16 23:00 UTC, Rich Megginson
nhosoi: review+
Details | Diff
patch3 - change extensible filter code to use new matching rule plugins (18.58 KB, patch)
2010-02-16 23:01 UTC, Rich Megginson
nhosoi: review+
Details | Diff
patch4- change syntax plugins to register new matching rules (95.00 KB, patch)
2010-02-16 23:02 UTC, Rich Megginson
nhosoi: review+
Details | Diff
patch5 - crash looking up compat syntax; numeric string syntax using integer; make octet string ordering work correctly (5.08 KB, patch)
2010-02-23 21:09 UTC, Rich Megginson
nhosoi: review+
Details | Diff

Description Terry Soucy 2010-01-27 18:00:52 UTC
Description of problem:

Upgraded to 389 DS 1.2.5, and now certain mail attributes 

Version-Release number of selected component (if applicable):
389 DS version 1.2.5

How reproducible:
Reproducible every time

Steps to Reproduce:
1. Verify that mail is lowercase on ldap record
2. ldapsearch -x -W -H ldap://ldap -D "uid=Authentication,dc=unb,dc=ca" -b "ou=people,dc=unb,dc=ca" -s sub "(mail=tsoucy@unb.CA)" mail

  
Actual results:
Search fail:
# extended LDIF
#
# LDAPv3
# base <ou=people,dc=unb,dc=ca> with scope subtree
# filter: (mail=tsoucy@unb.CA)
# requesting: mail 
#

# search result
search: 2
result: 0 Success

# numResponses: 1


Expected results:
# extended LDIF
#
# LDAPv3
# base <ou=people,dc=unb,dc=ca> with scope subtree
# filter: (mail=tsoucy@UNB.CA)
# requesting: mail 
#

# cdb9c1b03391630d1f3e655758fbc00203d340aa, people, unb.ca
dn: unbCaId=cdb9c1b03391630d1f3e655758fbc00203d340aa,ou=people,dc=unb,dc=ca
mail: tsoucy@unb.ca


Additional info:

Possibly an issue with the recently added syntax

Comment 1 Terry Soucy 2010-01-27 18:13:29 UTC
An addition to this.  If we revert back to the schema that was backed up during the upgrade, the search performs as expected.  The backed up schema files are ...

00core.ldif
01common.ldif
05rfc2247.ldif
28pilot.ldif
30ns-common.ldif
50ns-directory.ldif
60mozilla.ldif

Terry Soucy

Comment 2 Noriko Hosoi 2010-01-27 18:20:52 UTC
Reassigning to Rich since he's investigating this problem now...

Comment 3 Rich Megginson 2010-01-27 18:31:46 UTC
problem attributes so far:

mail
sambadomainname

Comment 4 Rich Megginson 2010-01-27 18:32:05 UTC
*** Bug 559323 has been marked as a duplicate of this bug. ***

Comment 5 Terry Soucy 2010-01-27 18:40:38 UTC
Correction, it does NOT work with the old schema in place.

Comment 6 Rich Megginson 2010-01-28 18:03:30 UTC
(In reply to comment #5)
> Correction, it does NOT work with the old schema in place.    

Are you sure you're using the old mail attribute definition?  The new one is in 05rfc4524.ldif - the old one is in a different file.

Comment 7 Rich Megginson 2010-01-28 23:17:48 UTC
Are you sure about sambadomainname?  The definition of sambaDomainName in the schema shipped with 389 (60samba3.ldif) hasn't changed in quite some time.

Comment 8 Anthony Messina 2010-01-29 04:11:33 UTC
Yes, I am sure about sambadomainname,but only on one of my i686 servers:

LOWERCASE search:
ldapsearch -x -D uid=samabroot,ou=People,dc=example,dc=com -W sambadomainname=example.com sambadomainname
Enter LDAP Password:
# extended LDIF
#
# LDAPv3
# base <dc=example,dc=com> (default) with scope subtree
# filter: sambadomainname=example.com
# requesting: ALL
#

# search result
search: 2
result: 0 Success

# numResponses: 1


UPPERCASE search:
[root@elburn ~]# ldapsearch -x -D uid=sambaroot,ou=People,dc=example,dc=com -W sambadomainname=EXAMPLE.COM sambadomainname
Enter LDAP Password:
# extended LDIF
#
# LDAPv3
# base <dc=example,dc=com> (default) with scope subtree
# filter: sambadomainname=EXAMPLE.COM
# requesting: ALL
#

# example.com, Domains, example.com
dn: sambadomainname=example.com,ou=Domains,dc=example,dc=com
sambadomainname: EXAMPLE.COM

# search result
search: 2
result: 0 Success

# numResponses: 2
# numEntries: 1

I do only see it on this server, however, which is a consumer of a single master.  I have another i686 that is also a consumer of the same single master.  The "other" i686 and the x86_64 master do not have this issue.

The one with the problem is my newest server.  Could this be a configuration option that I may have missed during setup or something?

Comment 9 Rich Megginson 2010-01-29 15:41:09 UTC
What samba schema are you using?  e.g.
cd /etc/dirsrv/slapd-yourinstance/schema
grep -i sambadomainname *

The definition in 389 1.2.0 and later is this:
attributeTypes: (
  1.3.6.1.4.1.7165.2.1.38
  NAME 'sambaDomainName'
  DESC 'Windows NT domain to which the user belongs'
  EQUALITY caseIgnoreMatch
  SYNTAX 1.3.6.1.4.1.1466.115.121.1.15{128}
  )

This should be case insensitive now, and should have been case insensitive for several releases.

Comment 10 Rich Megginson 2010-01-29 18:51:10 UTC
Created attachment 387625 [details]
python ldap schema lookup script

I used the attached script to dump the attribute types from the schema in the latest server.
The following are the list of attributes which have syntax CES but have a case ignore matching rule:
( 1.2.840.113556.1.4.478 NAME 'calCalURI' DESC 'RFC2739: URI of entire default calendar' EQUALITY caseIgnoreIA5Match SUBSTR caseIgnoreIA5SubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.26 )
( 1.2.840.113556.1.4.479 NAME 'calFBURL' DESC 'RFC2739: URI to the users default freebusy data' EQUALITY caseIgnoreIA5Match SUBSTR caseIgnoreIA5SubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.26 )
( 0.9.2342.19200300.100.1.25 NAME 'dc' EQUALITY caseIgnoreIA5Match SUBSTR caseIgnoreIA5SubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.26 SINGLE-VALUE )
( 0.9.2342.19200300.100.1.3 NAME 'mail' EQUALITY caseIgnoreIA5Match SUBSTR caseIgnoreIA5SubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.26{256} )
( 5.3.6.1.1.1.1.1 NAME 'accessTo' DESC 'Access to which servers user is allowed' EQUALITY caseIgnoreIA5Match SUBSTR caseIgnoreIA5SubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.26 )
( 5.3.6.1.1.1.1.0 NAME 'trustModel' DESC 'Access scheme' EQUALITY caseIgnoreIA5Match SUBSTR caseIgnoreIA5SubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.26 SINGLE-VALUE )
( 0.9.2342.19200300.100.1.37 NAME 'associatedDomain' EQUALITY caseIgnoreIA5Match SUBSTR caseIgnoreIA5SubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.26 )
( 1.3.6.1.4.1.13769.2.2 NAME ( 'mozillaSecondEmail' 'xmozillasecondemail' ) EQUALITY caseIgnoreIA5Match SUBSTR caseIgnoreIA5SubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.26{256} SINGLE-VALUE )
( 1.2.840.113556.1.4.485 NAME 'calOtherCalAdrURIs' DESC 'RFC2739: multi-value URI to other request destinations' EQUALITY caseIgnoreIA5Match SUBSTR caseIgnoreIA5SubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.26 )
( 1.2.840.113556.1.4.484 NAME 'calOtherCAPURIs' DESC 'RFC2739: multi-value URI to other calendars' EQUALITY caseIgnoreIA5Match SUBSTR caseIgnoreIA5SubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.26 )
( 1.2.840.113556.1.4.481 NAME 'calCalAdrURI' DESC 'RFC2739: URI for event equests destination' EQUALITY caseIgnoreIA5Match SUBSTR caseIgnoreIA5SubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.26 )
( 1.2.840.113556.1.4.480 NAME 'calCAPURI' DESC 'RFC2739: URI used to communicate with the users calendar' EQUALITY caseIgnoreIA5Match SUBSTR caseIgnoreIA5SubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.26 )
( 1.2.840.113556.1.4.483 NAME 'calOtherFBURLs' DESC 'RFC2739: multi-value URI for other free/busy data' EQUALITY caseIgnoreIA5Match SUBSTR caseIgnoreIA5SubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.26 )
( 1.2.840.113556.1.4.482 NAME 'calOtherCalURIs' DESC 'RFC2739: multi-value URI for snapshots of other calendars' EQUALITY caseIgnoreIA5Match SUBSTR caseIgnoreIA5SubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.26 )

The following are the list of attributes which have syntax CIS but have a case exact matching rule:
( 1.3.6.1.4.1.42.2.27.4.1.11 NAME 'javaReferenceAddress' DESC 'Addresses associated with a JNDI Reference' EQUALITY caseExactMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 )
( 1.3.6.1.4.1.250.1.57 NAME 'labeledURI' EQUALITY caseExactMatch SUBSTR caseExactSubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 )
( 1.3.6.1.4.1.42.2.27.4.1.6 NAME 'javaClassName' DESC 'Fully qualified name of distinguished Java class or interface' EQUALITY caseExactMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 SINGLE-VALUE )
( 1.3.6.1.4.1.42.2.27.4.1.13 NAME 'javaClassNames' DESC 'Fully qualified Java class or interface name' EQUALITY caseExactMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 )
( 1.3.6.1.4.1.42.2.27.4.1.10 NAME 'javaFactory' DESC 'Fully qualified Java class name of a JNDI object factory' EQUALITY caseExactMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 SINGLE-VALUE )
( 1.3.6.1.1.1.1.33 NAME 'automountInformation' DESC 'Information used by the autofs automounter' EQUALITY caseExactIA5Match SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 SINGLE-VALUE )

Comment 11 Anthony Messina 2010-01-30 16:31:16 UTC
On the two servers I have where it works, I have:

grep -i sambadomainname *
60samba.ldif:attributeTypes: ( 1.3.6.1.4.1.7165.2.1.38 NAME 'sambaDomainName' DESC 'Windows NT domain to which the user belongs' EQUALITY caseIgnoreMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15{128} )
....
99user.ldif:attributetypes: ( 1.3.6.1.4.1.7165.2.1.38 NAME 'sambaDomainName' DESC 'Windows NT domain to which the user belongs' EQUALITY caseIgnoreMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15{128} X-ORIGIN 'user defined' )


On the server I have where it DOES NOT work, I have:

grep -i sambadomainname *
<nothing returned>

even though I did create the 60samba.ldif file (from the /usr/share/dirsrv/data directory). That was the process for my other servers.  It now appears as though it's the 60samba3.ldif file that defines the sambaDomainName attr rather than the 60samba.ldif.

Comment 12 Anthony Messina 2010-01-30 21:43:34 UTC
Ok, I've figured out my problem for sambadomainname.  On my original 389 DS master, when installed about a year ago, it was the 60samba.ldif file that contained sambadomainname.  When binging in a copy of the file /usr/share/dirsrv/data/60samba.ldif into my directory instance schema directory (/etc/dirsrv/slapd-ds/schema) everything loaded and was available.

On my newest (~2 months old) 389 DS consumer, the sambadomainname is in 60samba3.ldif.  I did not realize that these files would be changing and without a look, performed the same proceure I did for the first (copying /usr/share/dirsrv/data/60samba.ldif to the per-instance schema directory), started the consumer up and everything seemed find and would replicate, etc.

It seems that what I need to do is scrap the definitions in the 60samba.ldif file and replace them with the 60samba3.ldif file.

I have read through http://www.redhat.com/docs/manuals/dir-server/8.1/admin/Extending_the_Directory_Schema.html in an effort to figure out how to do this, but I am not sure that I can, since on the 389 DS master, all of the samba* attributes were pulled into the 99user.ldif file.

How can I simply remove all of the 60samba.ldif stuff and restart with the 60samba3.ldif stuff?

Comment 13 Anthony Messina 2010-01-30 22:55:50 UTC
What I did to fix the above was to remove the 60samba.ldif file from the instance specific schema dir, copy in the 60samba3.ldif file, and remove all lines that were in either 60samba.ldif or 60samba3.ldif from the 99user.ldif file.

This seemed to work.  I now do not have any issue querying for sambadomainname, regardless of case.

Comment 14 Terry Soucy 2010-02-01 15:33:07 UTC
I ran the above python script and the result for the mail attribute is the same ...

From Rich's output from the default schema ...

( 0.9.2342.19200300.100.1.3 NAME 'mail' EQUALITY caseIgnoreIA5Match SUBSTR caseIgnoreIA5SubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.26{256} )

From the output when I run the script against my own schema ...

( 0.9.2342.19200300.100.1.3 NAME 'mail' EQUALITY caseIgnoreIA5Match SUBSTR caseIgnoreIA5SubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.26{256} )

This is the same for all three systems in our cluster (single-master rep)

We are using the mail attribute definition from the 05rfc4524.ldif file

I don't know what else to do to help track this down.  We are currently looking at processing an ldif to change everyones mail attribute to include an uppercase domain so that we can work around this issue.

Terry

Comment 15 Rich Megginson 2010-02-01 15:40:07 UTC
You may just want to change the 'mail' attribute to SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 instead of .26.  That should make it case insensitive.

Comment 16 Terry Soucy 2010-02-01 17:23:00 UTC
That fixed the issue.  Will this be changed in the schema file (05rfc4524.ldif) in future releases?

Comment 17 Rich Megginson 2010-02-01 17:26:01 UTC
We're still trying to figure out the best approach to take.  We'd rather not revert the schema, but that looks to be the best short term option.

Comment 18 Rich Megginson 2010-02-16 23:00:22 UTC
Created attachment 394656 [details]
patch1 - do not use syntax plugins directly

Comment 19 Rich Megginson 2010-02-16 23:00:55 UTC
Created attachment 394657 [details]
patch2- wrap new style matching rule plugins

Comment 20 Rich Megginson 2010-02-16 23:01:34 UTC
Created attachment 394659 [details]
patch3 - change extensible filter code to use new matching rule plugins

Comment 21 Rich Megginson 2010-02-16 23:02:00 UTC
Created attachment 394660 [details]
patch4- change syntax plugins to register new matching rules

Comment 22 Rich Megginson 2010-02-17 22:06:17 UTC
To ssh://git.fedorahosted.org/git/389/ds.git
   363be33..ecf93e6  master -> master

commit ecf93e699b04d45fdfa07b12094adaab0233c47a
Author: Rich Megginson <rmeggins@redhat.com>
Date:   Tue Feb 16 15:56:59 2010 -0700

    change syntax plugins to register required matching rule plugins

commit 6adbad044ef95411882ec546281a0df6d0816673
Author: Rich Megginson <rmeggins@redhat.com>
Date:   Wed Feb 10 14:49:03 2010 -0700

    change extensible filter code to use new syntax function style mr funcs

commit ca6e6538a65bc03f7b8e1c521b5d0ba6d7b82a9e
Author: Rich Megginson <rmeggins@redhat.com>
Date:   Wed Feb 10 09:16:28 2010 -0700

    wrap new style matching rule plugins for use in old style indexing code

commit 834c706f04e53bb3ca95caa31c6e1166ad79210e
Author: Rich Megginson <rmeggins@redhat.com>
Date:   Mon Feb 8 08:57:52 2010 -0700

    Do not use syntax plugins directly for filters, indexing

commit 3e5e21c68afc5ff38d0d843fafaddd145e4d38f5
Author: Rich Megginson <rmeggins@redhat.com>
Date:   Mon Feb 8 08:53:44 2010 -0700

    bump version to 1.2.6.a2

Comment 23 Rich Megginson 2010-02-23 21:09:32 UTC
Created attachment 395827 [details]
patch5 - crash looking up compat syntax; numeric string syntax using integer; make octet string ordering work correctly

Comment 24 Rich Megginson 2010-02-23 21:35:45 UTC
To ssh://git.fedorahosted.org/git/389/ds.git
   c26ba79..c0fd017  master -> master

commit c0fd0171fed64b026bc80bad872b6641a0c4d86f
Author: Rich Megginson <rmeggins@redhat.com>
Date:   Mon Feb 22 13:59:01 2010 -0700

    Reviewed by: nhosoi (Thanks!)
    Branch: HEAD
    Fix Description: slapi_matchingrule_is_compat() was not checking for NULL; t
he matching rule syntax plugin was registering with the INTEGER syntax oid; the 
bin_filter_ava() function needs to be ordering aware to implement the octetStrin
gOrderingMatch; in default_mr_filter_create(), make sure the requested matching 
rule is provided by the given plugin
    Platforms tested: RHEL5 x86_64
    Flag Day: no
    Doc impact: no

Comment 25 Rich Megginson 2010-03-09 16:50:31 UTC
To ssh://git.fedorahosted.org/git/389/ds.git
   b8ff06d..2db1f5a  master -> master

commit 2db1f5a13b7198de00b2b14232110ab42fc361ac
Author: Rich Megginson <rmeggins@redhat.com>
Date:   Mon Mar 8 20:53:49 2010 -0700
    Reviewed by: nhosoi (Thanks!)
    Fix Description:
    1) The 60qmail.ldif schema we ship used integerMatch and IA5 syntax
    because we used not to support numericString syntax and matching rules -
    these have been changed to use the standard qmail definitions
    2) Allow IA5String syntax to use caseExactSubstringsMatch - this is requir
    by krbPrincipalName

Comment 26 Rich Megginson 2010-06-30 00:05:29 UTC
*** Bug 609321 has been marked as a duplicate of this bug. ***

Comment 29 Amita Sharma 2011-07-29 05:37:51 UTC
Thanks Rich,
here are the results of Filter :
filter startup 	100% (1/1) 	  	 
filter run 	100% (267/267) 	  	 
filter cleanup 	100% (2/2)

Hence marking VERIFIED.


Note You need to log in before you can comment on or make changes to this bug.