Bug 639289 - Adding a new CN entry with UpperCase UTF-8 Character
Adding a new CN entry with UpperCase UTF-8 Character
Status: CLOSED CURRENTRELEASE
Product: 389
Classification: Community
Component: Database - Import/Export (Show other bugs)
1.2.6
i686 Linux
medium Severity medium
: ---
: ---
Assigned To: Noriko Hosoi
Viktor Ashirov
:
Depends On:
Blocks: 389_1.2.7 639035
  Show dependency treegraph
 
Reported: 2010-10-01 07:49 EDT by Ozmen Emre Demirkol
Modified: 2015-12-07 12:13 EST (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2015-12-07 12:13:14 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
git patch file (master) (1.92 KB, patch)
2010-10-04 20:30 EDT, Noriko Hosoi
nkinder: review+
rmeggins: review+
Details | Diff
Screenshot.png (85.24 KB, image/png)
2011-06-16 08:08 EDT, Amita Sharma
no flags Details

  None (edit)
Description Ozmen Emre Demirkol 2010-10-01 07:49:05 EDT
adding a new CN if i use UPPERCASE UTF-8 character server respond "does not such object".
if i use lowercase there is no problem. just for UPPERCASE UTF-8


i found this problem after fc12 t fc13 upgrade.

But, i made a clean install and migrate all old directory from ldif file.
The problem still continue. 

This is interesting. Because my ldif file has lots of UTF-8 CNs. while i was importing them didn't see any problems.


log example;

[01/Oct/2010:11:24:29 +0300] conn=47903 op=1 SRCH base="CN=H\c3\9cSEY\c4\b0N CANTA\c5\9e,ou=TMO_GNMUD_BILGII_SLEM_DAIRE_BSK,o=TARIM_VE_KOYISLERI_BAKANLIGI,o=KAMU_KURUMLARI,o=NES,o=SERTIFIKA,C=TR" scope=0 filter="(cn=H\c3\9cSEY\c4\b0N CANTA\c5\9e)" attrs=ALL
[01/Oct/2010:11:24:29 +0300] conn=47903 op=1 RESULT err=32 tag=101 nentries=0 etime=0
[01/Oct/2010:11:24:29 +0300] conn=47903 op=2 SRCH base="CN=H\c3\9cSEY\c4\b0N CANTA\c5\9e-1344529107,ou=TMO_GNMUD_BILGII_SLEM_DAIRE_BSK,o=TARIM_VE_KOYISLERI_BAKANLIGI,o=KAMU_KURUMLARI,o=NES,o=SERTIFIKA,C=TR" scope=0 filter="(cn=H\c3\9cSEY\c4\b0N CANTA\c5\9e-1344529107)" attrs=ALL
[01/Oct/2010:11:24:29 +0300] conn=47903 op=2 RESULT err=32 tag=101 nentries=0 etime=0
Comment 1 Ozmen Emre Demirkol 2010-10-01 07:52:14 EDT
Sorry, the example is not hole log

[01/Oct/2010:11:24:29 +0300] conn=47903 op=1 SRCH base="CN=H\c3\9cSEY\c4\b0N CANTA\c5\9e,ou=TMO_GNMUD_BILGII_SLEM_DAIRE_BSK,o=TARIM_VE_KOYISLERI_BAKANLIGI,o=KAMU_KURUMLARI,o=NES,o=SERTIFIKA,C=TR" scope=0 filter="(cn=H\c3\9cSEY\c4\b0N CANTA\c5\9e)" attrs=ALL
[01/Oct/2010:11:24:29 +0300] conn=47903 op=1 RESULT err=32 tag=101 nentries=0 etime=0
[01/Oct/2010:11:24:29 +0300] conn=47903 op=2 SRCH base="CN=H\c3\9cSEY\c4\b0N CANTA\c5\9e-1344529107,ou=TMO_GNMUD_BILGII_SLEM_DAIRE_BSK,o=TARIM_VE_KOYISLERI_BAKANLIGI,o=KAMU_KURUMLARI,o=NES,o=SERTIFIKA,C=TR" scope=0 filter="(cn=H\c3\9cSEY\c4\b0N CANTA\c5\9e-1344529107)" attrs=ALL
[01/Oct/2010:11:24:29 +0300] conn=47903 op=2 RESULT err=32 tag=101 nentries=0 etime=0
[01/Oct/2010:11:24:29 +0300] conn=47904 fd=68 slot=68 connection from xxx.xxx.xxx.xxx to yyy.yyy.yyy.yyy
[01/Oct/2010:11:24:29 +0300] conn=47904 op=0 BIND dn="uid=dizin1,dc=kamusm,dc=gov,dc=tr" method=128 version=3
[01/Oct/2010:11:24:29 +0300] conn=47904 op=0 RESULT err=0 tag=97 nentries=0 etime=0 dn="uid=dizin1,dc=kamusm,dc=gov,dc=tr"
[01/Oct/2010:11:24:29 +0300] conn=47904 op=1 ADD dn="CN=H\c3\9cSEY\c4\b0N CANTA\c5\9e-1344529107,ou=TMO_GNMUD_BILGII_SLEM_DAIRE_BSK,o=TARIM_VE_KOYISLERI_BAKANLIGI,o=KAMU_KURUMLARI,o=NES,o=SERTIFIKA,C=TR"
[01/Oct/2010:11:24:29 +0300] conn=47904 op=1 RESULT err=32 tag=105 nentries=0 etime=0
Comment 2 Noriko Hosoi 2010-10-04 20:30:50 EDT
Created attachment 451562 [details]
git patch file (master)

Description:
There was a bug in the utf8 uppe2Lower table:
Character İ (LATIN CAPITAL LETTER I WITH DOT ABOVE) did not map
to the corresponding LATIN SMALL LETTER DOTLESS I (2 bytes) but
to ascii 'i' (1 byte).  The shortened DN tailed with a garbage
character and the entry was treated as an orphan entry which does
not belong to any suffix.

This patch fixes the mapping table mismatch as well as adds a code
to dn_ignore_case_to_end to force to NULL terminate the converted
string.

Files:
 ldap/servers/slapd/dn.c
 ldap/servers/slapd/utf8compare.c
Comment 4 Ozmen Emre Demirkol 2010-10-05 03:16:18 EDT
Thank you so much. it's working.
I hope i'll be included next release.
Comment 5 Noriko Hosoi 2010-10-05 12:35:18 EDT
Reviewed and tested by Rich and Ozmen (Thank you!).

Pushed to master.

$ git push
Counting objects: 13, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (7/7), done.
Writing objects: 100% (7/7), 999 bytes, done.
Total 7 (delta 5), reused 0 (delta 0)
To ssh://git.fedorahosted.org/git/389/ds.git
   5f1cf3b..76c3339  master -> master

$ git merge 639289
Updating 5f1cf3b..76c3339
Fast-forward
 ldap/servers/slapd/dn.c          |    3 +++
 ldap/servers/slapd/utf8compare.c |    2 +-
 2 files changed, 4 insertions(+), 1 deletions(-)

This fix is going to be included in 389 v1.2.7.
Comment 6 Noriko Hosoi 2010-10-05 12:42:49 EDT
Reviewed by Nathan, as well (Thank you!!).
Comment 7 Jenny Galipeau 2011-06-06 15:05:27 EDT
can you please add steps to reproduce and verify thanks
Comment 8 Noriko Hosoi 2011-06-06 17:38:27 EDT
Create an entry including the character in dn:
  Character İ (LATIN CAPITAL LETTER I WITH DOT ABOVE)
e.g., 
dn: cn=İ, dc=example,dc=com
cn: İ
...

Add the entry, then search the entry with the filter
"(cn=İ)"
as well as
"(cn=ı)" <== corresponding LATIN SMALL LETTER DOTLESS I (2 bytes)

The search should return the entry.

Also, search with the filter "(cn=i)" <== ascii 'i'.

This search should NOT return the entry.

If the above searches work as expected, this bug is verified.
Comment 9 Amita Sharma 2011-06-16 08:08:14 EDT
Created attachment 505029 [details]
Screenshot.png

I have added the entry from ds console, PFA.
Then, I tried to search it :

1. This is not as expected...
[root@rhel61 ~]# ldapsearch -x -h localhost -p 389 -D 'cn=directory manager' -w Secret123 -b "uid=ı,dc=example,dc=com"
# extended LDIF
#
# LDAPv3
# base <uid=ı,dc=example,dc=com> with scope subtree
# filter: (objectclass=*)
# requesting: ALL
#

# \C4\B0, example.com
dn:: dWlkPcSwLGRjPWV4YW1wbGUsZGM9Y29t
uid:: xLA=
givenName:: xLA=
objectClass: top
objectClass: person
objectClass: organizationalPerson
objectClass: inetorgperson
sn:: xLA=
cn:: xLAgxLA=

# search result
search: 2
result: 0 Success

# numResponses: 2
# numEntries: 1

2. This is as expected
[root@rhel61 ~]# ldapsearch -x -h localhost -p 389 -D 'cn=directory manager' -w Secret123 -b "uid=i,dc=example,dc=com"
# extended LDIF
#
# LDAPv3
# base <uid=i,dc=example,dc=com> with scope subtree
# filter: (objectclass=*)
# requesting: ALL
#

# search result
search: 2
result: 32 No such object
matchedDN: dc=example,dc=com

# numResponses: 1

- Also when I added it with ldapadd :
ldapadd -x -h localhost -p 389 -D "cn=Directory Manager" -w Secret123  << EOF
> dn: cn=İ, dc=example,dc=com
> cn: İ
> sn: ams
> givenname: ams
> objectclass: top
> objectclass: person
> objectclass: organizationalPerson
> objectclass: inetOrgPerson
> uid: ams
> mail: ams@example.com
> userpassword: amsamsams
> EOF
adding new entry "cn=İ, dc=example,dc=com"

[root@rhel61 ~]# ldapsearch -x -h localhost -p 389 -D 'cn=directory manager' -w Secret123 -b "cn=İ,dc=example,dc=com"
# extended LDIF
#
# LDAPv3
# base <cn=İ,dc=example,dc=com> with scope subtree
# filter: (objectclass=*)
# requesting: ALL
#

# \C4\B0, example.com
dn:: Y249xLAsZGM9ZXhhbXBsZSxkYz1jb20=
cn:: xLA=
sn: ams
givenName: ams
objectClass: top
objectClass: person
objectClass: organizationalPerson
objectClass: inetOrgPerson
uid: ams
mail: ams@example.com
userPassword:: e1NTSEF9UXhHNnh5TDZnNndQU1BteCtYTTdvTUJIM2NicWlQTHF4SEVFelE9PQ=
 =

# search result
search: 2
result: 0 Success

# numResponses: 2
# numEntries: 1

Not as expected ?
Comment 10 Rich Megginson 2011-06-16 11:05:55 EDT
(In reply to comment #9)
> Created attachment 505029 [details]
> Screenshot.png
> 
> I have added the entry from ds console, PFA.
> Then, I tried to search it :
> 
> 1. This is not as expected...
> [root@rhel61 ~]# ldapsearch -x -h localhost -p 389 -D 'cn=directory manager' -w
> Secret123 -b "uid=ı,dc=example,dc=com"
> # extended LDIF
> #
> # LDAPv3
> # base <uid=ı,dc=example,dc=com> with scope subtree
> # filter: (objectclass=*)
> # requesting: ALL
> #
> 
> # \C4\B0, example.com
> dn:: dWlkPcSwLGRjPWV4YW1wbGUsZGM9Y29t
> uid:: xLA=
> givenName:: xLA=
> objectClass: top
> objectClass: person
> objectClass: organizationalPerson
> objectClass: inetorgperson
> sn:: xLA=
> cn:: xLAgxLA=
> 
> # search result
> search: 2
> result: 0 Success
> 
> # numResponses: 2
> # numEntries: 1

This is as expected.  One of the "features" of ldapsearch is that it will base64 encode non-printable values, such as strings which contain 8-bit characters.
python
>>> import base64
>>> base64.b64decode('dWlkPcSwLGRjPWV4YW1wbGUsZGM9Y29t')
'uid=\xc4\xb0,dc=example,dc=com'
>>> base64.b64decode('xLA=')
'\xc4\xb0'

I believe \xc4\xb0 is the utf-8 encoding of ı

> 
> 2. This is as expected
> [root@rhel61 ~]# ldapsearch -x -h localhost -p 389 -D 'cn=directory manager' -w
> Secret123 -b "uid=i,dc=example,dc=com"
> # extended LDIF
> #
> # LDAPv3
> # base <uid=i,dc=example,dc=com> with scope subtree
> # filter: (objectclass=*)
> # requesting: ALL
> #
> 
> # search result
> search: 2
> result: 32 No such object
> matchedDN: dc=example,dc=com
> 
> # numResponses: 1
> 
> - Also when I added it with ldapadd :
> ldapadd -x -h localhost -p 389 -D "cn=Directory Manager" -w Secret123  << EOF
> > dn: cn=İ, dc=example,dc=com
> > cn: İ
> > sn: ams
> > givenname: ams
> > objectclass: top
> > objectclass: person
> > objectclass: organizationalPerson
> > objectclass: inetOrgPerson
> > uid: ams
> > mail: ams@example.com
> > userpassword: amsamsams
> > EOF
> adding new entry "cn=İ, dc=example,dc=com"
> 
> [root@rhel61 ~]# ldapsearch -x -h localhost -p 389 -D 'cn=directory manager' -w
> Secret123 -b "cn=İ,dc=example,dc=com"
> # extended LDIF
> #
> # LDAPv3
> # base <cn=İ,dc=example,dc=com> with scope subtree
> # filter: (objectclass=*)
> # requesting: ALL
> #
> 
> # \C4\B0, example.com
> dn:: Y249xLAsZGM9ZXhhbXBsZSxkYz1jb20=
> cn:: xLA=
> sn: ams
> givenName: ams
> objectClass: top
> objectClass: person
> objectClass: organizationalPerson
> objectClass: inetOrgPerson
> uid: ams
> mail: ams@example.com
> userPassword:: e1NTSEF9UXhHNnh5TDZnNndQU1BteCtYTTdvTUJIM2NicWlQTHF4SEVFelE9PQ=
>  =
> 
> # search result
> search: 2
> result: 0 Success
> 
> # numResponses: 2
> # numEntries: 1
> 
> Not as expected ?

Yes, as expected.
Comment 11 Amita Sharma 2011-06-17 02:43:33 EDT
ok, that said I am marking the bug as VERIFIED.

Note You need to log in before you can comment on or make changes to this bug.