Bug 1860008

Summary: On ADD replication URP issue internal searches with filter containing unescaped chars
Product: Red Hat Enterprise Linux 7 Reporter: Têko Mihinto <tmihinto>
Component: 389-ds-baseAssignee: thierry bordaz <tbordaz>
Status: CLOSED ERRATA QA Contact: RHDS QE <ds-qe-bugs>
Severity: medium Docs Contact:
Priority: high    
Version: 7.8CC: bsmejkal, ekasprzy, mreynolds, msauton, pasik, spichugi, tbordaz, vashirov
Target Milestone: rcKeywords: TestCaseProvided, Triaged
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: 389-ds-base-1.3.10.2-7.1.el7_9 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-11-10 13:14:10 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1898850, 1902042    

Description Têko Mihinto 2020-07-23 14:17:16 UTC
Description of problem:
A search with a filter that targets a specific value is being considered as a substring search.

Version-Release number of selected component (if applicable):
# rpm -qa | grep 389-ds-base-1
389-ds-base-1.3.10.1-5.el7.x86_64
#
# cat /etc/redhat-release
Red Hat Enterprise Linux Server release 7.8 Beta (Maipo)
#

How reproducible:
Always.

Steps to Reproduce:
1. Create an entry that has a value which contains an asterisk.
For instance:
# ldapmodify -x -D "cn=Directory Manager" -W -h localhost -p 389 -a
Enter LDAP Password:
dn: uid=asterisk_*_in_value,ou=People,dc=example,dc=com
objectClass: top
objectClass: person
objectClass: inetOrgPerson
cn: asterisk_cn
sn: asterisk_sn

adding new entry "uid=asterisk_*_in_value,ou=People,dc=example,dc=com"
#

2. Run a search using the exact value:
# ldapsearch -xLLL -D "cn=Directory Manager" -W -h localhost -p 389 -b "dc=example,dc=com" "uid=asterisk_*_in_value" 1.1
Enter LDAP Password:
dn: uid=asterisk_*_in_value,ou=People,dc=example,dc=com
#


3. One would expect an indexed search thanks to the "uid" equality index, but the access log shows an unindexed search:
=============================================
[23/Jul/2020:15:51:07.773604190 +0200] conn=19 fd=64 slot=64 connection from ::1 to ::1
[23/Jul/2020:15:51:07.774704485 +0200] conn=19 op=0 BIND dn="cn=Directory Manager" method=128 version=3
[23/Jul/2020:15:51:07.775101095 +0200] conn=19 op=0 RESULT err=0 tag=97 nentries=0 etime=0.001096163 dn="cn=directory manager"
[23/Jul/2020:15:51:07.775749868 +0200] conn=19 op=1 SRCH base="dc=example,dc=com" scope=2 filter="(uid=asterisk_*_in_value)" attrs="1.1"
[23/Jul/2020:15:51:07.776787523 +0200] conn=19 op=1 RESULT err=0 tag=101 nentries=1 etime=0.001453590 notes=U
[23/Jul/2020:15:51:07.778618986 +0200] conn=19 op=2 UNBIND
[23/Jul/2020:15:51:07.778652169 +0200] conn=19 op=2 fd=64 closed - U1
=============================================

Actual results:
The search is not indexed.

Expected results:
The search should be indexed since uid has the equality index type and the number of candidates is lower than the IDL scan limit.
=============================================
dn: cn=uid,cn=index,cn=example,cn=ldbm database,cn=plugins,cn=config
objectClass: top
objectClass: nsIndex
cn: uid
nsSystemIndex: false
nsIndexType: eq
creatorsName: cn=ldbm database,cn=plugins,cn=config
modifiersName: cn=ldbm database,cn=plugins,cn=config
...
=============================================

# ldapsearch -xLLL -D "cn=Directory Manager" -W -h localhost -p 389 -b "dc=example,dc=com" "uid=*" 1.1 | grep -c ^dn:
Enter LDAP Password:
7
#

Additional info:

The "*" character doesn't seem to be part of the characters to escape in an LDAP DN:
    https://www.ietf.org/rfc/rfc4514.txt
========================================
  Other characters may be escaped.

   Each octet of the character to be escaped is replaced by a backslash
   and two hex digits, which form a single octet in the code of the
   character.  Alternatively, if and only if the character to be escaped
   is one of

      ' ', '"', '#', '+', ',', ';', '<', '=', '>', or '\'
      (U+0020, U+0022, U+0023, U+002B, U+002C, U+003B,
       U+003C, U+003D, U+003E, U+005C, respectively)

   it can be prefixed by a backslash ('\' U+005C).
========================================

Comment 6 thierry bordaz 2020-08-19 15:15:39 UTC
A RDN can contain ASTERISK without the need to escape it ( https://www.ietf.org/rfc/rfc4514.txt). So 'uid=asterisk_*_in_value,ou=People,dc=example,dc=com' is a valid DN

A ldapclient search filter should escape ASTERISK it its filter (https://tools.ietf.org/html/rfc4515#section-3). Only 'subtstring' item can contain ASTERISK. So it is normal that item "uid=asterisk_*_in_value" is treated as a subtring (any) filter.
If a '*' is present is an assertion it should be escaped

========================================

   For AssertionValues that contain UTF-8 character data, each octet of
   the character to be escaped is replaced by a backslash and two hex
   digits, which form a single octet in the code of the character.  For
   example, the filter checking whether the "cn" attribute contained a
   value with the character "*" anywhere in it would be represented as
   "(cn=*\2a*)".

...

The following examples illustrate the use of the escaping mechanism.

        (o=Parens R Us \28for all your parenthetical needs\29)
        (cn=*\2A*)
        (filename=C:\5cMyFile)
        (bin=\00\00\00\04)
        (sn=Lu\c4\8di\c4\87)
        (1.3.6.1.4.1.1466.0=\04\02\48\69)

   The first example shows the use of the escaping mechanism to
   represent parenthesis characters.  The second shows how to represent
   a "*" in an assertion value, preventing it from being interpreted as
   a substring indicator.  The third illustrates the escaping of the
   backslash character.
========================================


Is it possible for the ldapclient to change the filter it requests and transforms '*' into '\2a'. ?

Comment 8 thierry bordaz 2020-08-20 08:43:01 UTC
The ldapclient is 'replication plugin' itself !!!

The case identified internal searches with filter "(&(objectclass=nstombstone)(nscpentrydn=<entryDN))" (during ADD). So this is a bug in replication plugin that uses filter assertion without appropriate escaping. Changing the BZ title to reflect that.

Comment 10 thierry bordaz 2020-08-20 16:30:20 UTC
upstream ticket https://pagure.io/389-ds-base/issue/51244

Comment 14 thierry bordaz 2020-10-02 10:05:43 UTC
Fix pushed upstream => POST

Comment 20 bsmejkal 2020-10-26 13:16:18 UTC
============================================================================================================ test session starts =============================================================================================================
platform linux -- Python 3.6.8, pytest-6.1.1, py-1.9.0, pluggy-0.13.1 -- /usr/bin/python3
cachedir: .pytest_cache
metadata: {'Python': '3.6.8', 'Platform': 'Linux-3.10.0-1160.6.1.el7.x86_64-x86_64-with-redhat-7.9-Maipo', 'Packages': {'pytest': '6.1.1', 'py': '1.9.0', 'pluggy': '0.13.1'}, 'Plugins': {'metadata': '1.10.0', 'html': '2.1.1'}}
389-ds-base: 1.3.10.2-7.1.el7_9
nss: 3.53.1-3.el7_9
nspr: 4.25.0-2.el7_9
openldap: 2.4.44-22.el7
cyrus-sasl: 2.1.26-23.el7
FIPS: disabled
rootdir: /workspace/ds/dirsrvtests, configfile: pytest.ini
plugins: metadata-1.10.0, html-2.1.1
collected 15 items / 14 deselected / 1 selected                                                                                                                                                                                              

tests/suites/replication/acceptance_test.py::test_urp_trigger_substring_search PASSED                                                                                                                                                  [100%]

========================================================================================== 1 passed, 14 deselected in 61.24s ================================================================================================================

Marking as VERIFIED.

Comment 24 errata-xmlrpc 2020-11-10 13:14:10 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (389-ds-base bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:5041