Bug 1523183

Summary: search with CoS attribute is getting slower after modifying/adding CosTemplate
Product: Red Hat Enterprise Linux 7 Reporter: Hiroko Miura <hmiura>
Component: 389-ds-baseAssignee: mreynolds
Status: CLOSED ERRATA QA Contact: Viktor Ashirov <vashirov>
Severity: urgent Docs Contact: Marc Muehlfeld <mmuehlfe>
Priority: urgent    
Version: 7.4CC: atolani, gparente, knakai, mreynolds, msauton, nkinder, rmeggins, thirayam, tkubota
Target Milestone: rcKeywords: ZStream
Target Release: ---   
Hardware: All   
OS: All   
Whiteboard:
Fixed In Version: 389-ds-base-1.3.7.5-12 Doc Type: Bug Fix
Doc Text:
Using a large number of CoS templates no longer slow down the virtual attribute processing time Due to a bug, using a large number of Class of Service (CoS) templates in Directory Server increased the virtual attribute processing time. This update improves the structure of the CoS storage. As a result, using a large number of CoS templates no longer increases the virtual attribute processing time.
Story Points: ---
Clone Of:
: 1526928 (view as bug list) Environment:
Last Closed: 2018-04-10 14:23:50 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1526928    
Attachments:
Description Flags
sample ldif to reproduce the issue
none
Resolve mm issue with vattr none

Description Hiroko Miura 2017-12-07 11:27:54 UTC
Created attachment 1364283 [details]
sample ldif to reproduce the issue

Description of problem:

When using Indirect Cos like the following,

dn: cn=cosDefinition,dc=test,dc=com
objectClass: top
objectClass: ldapsubentry
objectClass: cossuperdefinition
objectClass: cosIndirectDefinition
cosAttribute: ou merge-schemes
cosAttribute: description merge-schemes
cosAttribute: postalCode merge-schemes
cn: cosDefinition
cosIndirectSpecifier: seeAlso

search takes longer than before after cosTemplate entries are added/modified.


Version-Release number of selected component (if applicable):

389-ds-base-1.3.6.1-24.el7_4.x86_64 


How reproducible:

This can be reproduced with attached sample LDIF.


Steps to Reproduce:
1. create suffix dc=test,dc=com and import test.ldif 
   (or adjust suffix accordingly)

2. run the following search

ldapsearch -D "cn=directory manager" -w password -b dc=test,dc=com uid=user1

  => this return the result immediately

3. add new cosTemplcate by the following command

ldapmodify -D "cn=directory manager" -w password  -a -c -f ou10000.ldif

   stop ldapmodify after adding about 500 entries by CTRL+C


4. run the same search

ldapsearch -D "cn=directory manager" -w password -b dc=test,dc=com uid=user1

  => this will take a time than step 2

5. add new cosTemplcate by the following command again

ldapmodify -D "cn=directory manager" -w password  -a -c -f ou10000.ldif

   stop ldapmodify after adding around 500 entries by CTRL+C

6. run the same search

ldapsearch -D "cn=directory manager" -w password -b dc=test,dc=com uid=user1

  => this will take a time than step 2 and 4

7. add additional Cos Identifier attribute

  ldapmodify -D "cn=directory manager" -w password -a -f addCosIdentifier.ldif
  

8. run the same search

ldapsearch -D "cn=directory manager" -w password -b dc=test,dc=com uid=user1

  => this will take too time than before (Cos works as expected though)


9. restart server and do the same search

  => this returns entry immediately


Actual results:
search takes longer time than before

Expected results:
search returns result soon

Additional info:
This happen after applying 389-ds-base-1.3.6.1-24.el7_4.x86_64.
But this does not happen with 389-ds-base-1.3.6.1-23.el7_4.x86_64

Comment 9 wibrown@redhat.com 2017-12-11 09:32:29 UTC
Upstream ticket:
https://pagure.io/389-ds-base/issue/49495

Comment 16 wibrown@redhat.com 2017-12-11 16:59:50 UTC
Created attachment 1366205 [details]
Resolve mm issue with vattr

This patch resolves the issue. This has been resolved upstream on affected branches.

Comment 18 Marc Sauton 2017-12-14 00:53:37 UTC
patch was confirmed good, making this bz 1523183 a RHEL-7.4.z candidate for the sf customer case number 01987672
to be reviewed tomorrow with dev and qe.

Comment 23 Viktor Ashirov 2018-01-30 13:04:30 UTC
Build tested:
389-ds-base-1.3.7.5-14.el7.x86_64


CoS test suite from acceptance tests passes 100% without a crash.

CoS scale stress test passes:
===================================================================== test session starts =====================================================================
platform linux -- Python 3.6.3, pytest-3.3.2, py-1.5.2, pluggy-0.6.0 -- /opt/rh/rh-python36/root/usr/bin/python3
cachedir: .cache
metadata: {'Python': '3.6.3', 'Platform': 'Linux-3.10.0-837.el7.x86_64-x86_64-with-redhat-7.5-Maipo', 'Packages': {'pytest': '3.3.2', 'py': '1.5.2', 'pluggy': '0.6.0'}, 'Plugins': {'metadata': '1.5.1', 'html': '1.16.1'}}
389-ds-base: 1.3.7.5-14.el7
nss: 3.34.0-4.el7
nspr: 4.17.0-1.el7
openldap: 2.4.44-12.el7
svrcore: 4.1.3-2.el7

rootdir: /export/tests, inifile:
plugins: metadata-1.5.1, html-1.16.1
collected 1 item                                                                                                                                              

stress/cos/cos_scale_template_test.py::test_indirect_template_scale PASSED                                                                              [100%]

------------------------------------------------ generated xml file: /mnt/tests/rhds/tests/upstream/report.xml ------------------------------------------------
----------------------------------------------- generated html file: /mnt/tests/rhds/tests/upstream/report.html -----------------------------------------------
================================================================== 1 passed in 62.78 seconds ==================================================================

Comment 27 errata-xmlrpc 2018-04-10 14:23:50 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:0811