Red Hat Bugzilla – Bug 639289
Adding a new CN entry with UpperCase UTF-8 Character
Last modified: 2015-12-07 12:13:14 EST
adding a new CN if i use UPPERCASE UTF-8 character server respond "does not such object". if i use lowercase there is no problem. just for UPPERCASE UTF-8 i found this problem after fc12 t fc13 upgrade. But, i made a clean install and migrate all old directory from ldif file. The problem still continue. This is interesting. Because my ldif file has lots of UTF-8 CNs. while i was importing them didn't see any problems. log example; [01/Oct/2010:11:24:29 +0300] conn=47903 op=1 SRCH base="CN=H\c3\9cSEY\c4\b0N CANTA\c5\9e,ou=TMO_GNMUD_BILGII_SLEM_DAIRE_BSK,o=TARIM_VE_KOYISLERI_BAKANLIGI,o=KAMU_KURUMLARI,o=NES,o=SERTIFIKA,C=TR" scope=0 filter="(cn=H\c3\9cSEY\c4\b0N CANTA\c5\9e)" attrs=ALL [01/Oct/2010:11:24:29 +0300] conn=47903 op=1 RESULT err=32 tag=101 nentries=0 etime=0 [01/Oct/2010:11:24:29 +0300] conn=47903 op=2 SRCH base="CN=H\c3\9cSEY\c4\b0N CANTA\c5\9e-1344529107,ou=TMO_GNMUD_BILGII_SLEM_DAIRE_BSK,o=TARIM_VE_KOYISLERI_BAKANLIGI,o=KAMU_KURUMLARI,o=NES,o=SERTIFIKA,C=TR" scope=0 filter="(cn=H\c3\9cSEY\c4\b0N CANTA\c5\9e-1344529107)" attrs=ALL [01/Oct/2010:11:24:29 +0300] conn=47903 op=2 RESULT err=32 tag=101 nentries=0 etime=0
Sorry, the example is not hole log [01/Oct/2010:11:24:29 +0300] conn=47903 op=1 SRCH base="CN=H\c3\9cSEY\c4\b0N CANTA\c5\9e,ou=TMO_GNMUD_BILGII_SLEM_DAIRE_BSK,o=TARIM_VE_KOYISLERI_BAKANLIGI,o=KAMU_KURUMLARI,o=NES,o=SERTIFIKA,C=TR" scope=0 filter="(cn=H\c3\9cSEY\c4\b0N CANTA\c5\9e)" attrs=ALL [01/Oct/2010:11:24:29 +0300] conn=47903 op=1 RESULT err=32 tag=101 nentries=0 etime=0 [01/Oct/2010:11:24:29 +0300] conn=47903 op=2 SRCH base="CN=H\c3\9cSEY\c4\b0N CANTA\c5\9e-1344529107,ou=TMO_GNMUD_BILGII_SLEM_DAIRE_BSK,o=TARIM_VE_KOYISLERI_BAKANLIGI,o=KAMU_KURUMLARI,o=NES,o=SERTIFIKA,C=TR" scope=0 filter="(cn=H\c3\9cSEY\c4\b0N CANTA\c5\9e-1344529107)" attrs=ALL [01/Oct/2010:11:24:29 +0300] conn=47903 op=2 RESULT err=32 tag=101 nentries=0 etime=0 [01/Oct/2010:11:24:29 +0300] conn=47904 fd=68 slot=68 connection from xxx.xxx.xxx.xxx to yyy.yyy.yyy.yyy [01/Oct/2010:11:24:29 +0300] conn=47904 op=0 BIND dn="uid=dizin1,dc=kamusm,dc=gov,dc=tr" method=128 version=3 [01/Oct/2010:11:24:29 +0300] conn=47904 op=0 RESULT err=0 tag=97 nentries=0 etime=0 dn="uid=dizin1,dc=kamusm,dc=gov,dc=tr" [01/Oct/2010:11:24:29 +0300] conn=47904 op=1 ADD dn="CN=H\c3\9cSEY\c4\b0N CANTA\c5\9e-1344529107,ou=TMO_GNMUD_BILGII_SLEM_DAIRE_BSK,o=TARIM_VE_KOYISLERI_BAKANLIGI,o=KAMU_KURUMLARI,o=NES,o=SERTIFIKA,C=TR" [01/Oct/2010:11:24:29 +0300] conn=47904 op=1 RESULT err=32 tag=105 nentries=0 etime=0
Created attachment 451562 [details] git patch file (master) Description: There was a bug in the utf8 uppe2Lower table: Character İ (LATIN CAPITAL LETTER I WITH DOT ABOVE) did not map to the corresponding LATIN SMALL LETTER DOTLESS I (2 bytes) but to ascii 'i' (1 byte). The shortened DN tailed with a garbage character and the entry was treated as an orphan entry which does not belong to any suffix. This patch fixes the mapping table mismatch as well as adds a code to dn_ignore_case_to_end to force to NULL terminate the converted string. Files: ldap/servers/slapd/dn.c ldap/servers/slapd/utf8compare.c
Thank you so much. it's working. I hope i'll be included next release.
Reviewed and tested by Rich and Ozmen (Thank you!). Pushed to master. $ git push Counting objects: 13, done. Delta compression using up to 4 threads. Compressing objects: 100% (7/7), done. Writing objects: 100% (7/7), 999 bytes, done. Total 7 (delta 5), reused 0 (delta 0) To ssh://git.fedorahosted.org/git/389/ds.git 5f1cf3b..76c3339 master -> master $ git merge 639289 Updating 5f1cf3b..76c3339 Fast-forward ldap/servers/slapd/dn.c | 3 +++ ldap/servers/slapd/utf8compare.c | 2 +- 2 files changed, 4 insertions(+), 1 deletions(-) This fix is going to be included in 389 v1.2.7.
Reviewed by Nathan, as well (Thank you!!).
can you please add steps to reproduce and verify thanks
Create an entry including the character in dn: Character İ (LATIN CAPITAL LETTER I WITH DOT ABOVE) e.g., dn: cn=İ, dc=example,dc=com cn: İ ... Add the entry, then search the entry with the filter "(cn=İ)" as well as "(cn=ı)" <== corresponding LATIN SMALL LETTER DOTLESS I (2 bytes) The search should return the entry. Also, search with the filter "(cn=i)" <== ascii 'i'. This search should NOT return the entry. If the above searches work as expected, this bug is verified.
Created attachment 505029 [details] Screenshot.png I have added the entry from ds console, PFA. Then, I tried to search it : 1. This is not as expected... [root@rhel61 ~]# ldapsearch -x -h localhost -p 389 -D 'cn=directory manager' -w Secret123 -b "uid=ı,dc=example,dc=com" # extended LDIF # # LDAPv3 # base <uid=ı,dc=example,dc=com> with scope subtree # filter: (objectclass=*) # requesting: ALL # # \C4\B0, example.com dn:: dWlkPcSwLGRjPWV4YW1wbGUsZGM9Y29t uid:: xLA= givenName:: xLA= objectClass: top objectClass: person objectClass: organizationalPerson objectClass: inetorgperson sn:: xLA= cn:: xLAgxLA= # search result search: 2 result: 0 Success # numResponses: 2 # numEntries: 1 2. This is as expected [root@rhel61 ~]# ldapsearch -x -h localhost -p 389 -D 'cn=directory manager' -w Secret123 -b "uid=i,dc=example,dc=com" # extended LDIF # # LDAPv3 # base <uid=i,dc=example,dc=com> with scope subtree # filter: (objectclass=*) # requesting: ALL # # search result search: 2 result: 32 No such object matchedDN: dc=example,dc=com # numResponses: 1 - Also when I added it with ldapadd : ldapadd -x -h localhost -p 389 -D "cn=Directory Manager" -w Secret123 << EOF > dn: cn=İ, dc=example,dc=com > cn: İ > sn: ams > givenname: ams > objectclass: top > objectclass: person > objectclass: organizationalPerson > objectclass: inetOrgPerson > uid: ams > mail: ams@example.com > userpassword: amsamsams > EOF adding new entry "cn=İ, dc=example,dc=com" [root@rhel61 ~]# ldapsearch -x -h localhost -p 389 -D 'cn=directory manager' -w Secret123 -b "cn=İ,dc=example,dc=com" # extended LDIF # # LDAPv3 # base <cn=İ,dc=example,dc=com> with scope subtree # filter: (objectclass=*) # requesting: ALL # # \C4\B0, example.com dn:: Y249xLAsZGM9ZXhhbXBsZSxkYz1jb20= cn:: xLA= sn: ams givenName: ams objectClass: top objectClass: person objectClass: organizationalPerson objectClass: inetOrgPerson uid: ams mail: ams@example.com userPassword:: e1NTSEF9UXhHNnh5TDZnNndQU1BteCtYTTdvTUJIM2NicWlQTHF4SEVFelE9PQ= = # search result search: 2 result: 0 Success # numResponses: 2 # numEntries: 1 Not as expected ?
(In reply to comment #9) > Created attachment 505029 [details] > Screenshot.png > > I have added the entry from ds console, PFA. > Then, I tried to search it : > > 1. This is not as expected... > [root@rhel61 ~]# ldapsearch -x -h localhost -p 389 -D 'cn=directory manager' -w > Secret123 -b "uid=ı,dc=example,dc=com" > # extended LDIF > # > # LDAPv3 > # base <uid=ı,dc=example,dc=com> with scope subtree > # filter: (objectclass=*) > # requesting: ALL > # > > # \C4\B0, example.com > dn:: dWlkPcSwLGRjPWV4YW1wbGUsZGM9Y29t > uid:: xLA= > givenName:: xLA= > objectClass: top > objectClass: person > objectClass: organizationalPerson > objectClass: inetorgperson > sn:: xLA= > cn:: xLAgxLA= > > # search result > search: 2 > result: 0 Success > > # numResponses: 2 > # numEntries: 1 This is as expected. One of the "features" of ldapsearch is that it will base64 encode non-printable values, such as strings which contain 8-bit characters. python >>> import base64 >>> base64.b64decode('dWlkPcSwLGRjPWV4YW1wbGUsZGM9Y29t') 'uid=\xc4\xb0,dc=example,dc=com' >>> base64.b64decode('xLA=') '\xc4\xb0' I believe \xc4\xb0 is the utf-8 encoding of ı > > 2. This is as expected > [root@rhel61 ~]# ldapsearch -x -h localhost -p 389 -D 'cn=directory manager' -w > Secret123 -b "uid=i,dc=example,dc=com" > # extended LDIF > # > # LDAPv3 > # base <uid=i,dc=example,dc=com> with scope subtree > # filter: (objectclass=*) > # requesting: ALL > # > > # search result > search: 2 > result: 32 No such object > matchedDN: dc=example,dc=com > > # numResponses: 1 > > - Also when I added it with ldapadd : > ldapadd -x -h localhost -p 389 -D "cn=Directory Manager" -w Secret123 << EOF > > dn: cn=İ, dc=example,dc=com > > cn: İ > > sn: ams > > givenname: ams > > objectclass: top > > objectclass: person > > objectclass: organizationalPerson > > objectclass: inetOrgPerson > > uid: ams > > mail: ams@example.com > > userpassword: amsamsams > > EOF > adding new entry "cn=İ, dc=example,dc=com" > > [root@rhel61 ~]# ldapsearch -x -h localhost -p 389 -D 'cn=directory manager' -w > Secret123 -b "cn=İ,dc=example,dc=com" > # extended LDIF > # > # LDAPv3 > # base <cn=İ,dc=example,dc=com> with scope subtree > # filter: (objectclass=*) > # requesting: ALL > # > > # \C4\B0, example.com > dn:: Y249xLAsZGM9ZXhhbXBsZSxkYz1jb20= > cn:: xLA= > sn: ams > givenName: ams > objectClass: top > objectClass: person > objectClass: organizationalPerson > objectClass: inetOrgPerson > uid: ams > mail: ams@example.com > userPassword:: e1NTSEF9UXhHNnh5TDZnNndQU1BteCtYTTdvTUJIM2NicWlQTHF4SEVFelE9PQ= > = > > # search result > search: 2 > result: 0 Success > > # numResponses: 2 > # numEntries: 1 > > Not as expected ? Yes, as expected.
ok, that said I am marking the bug as VERIFIED.