Bug 1388135 - [RFE] limit the retro changelog to dns subtree.
Summary: [RFE] limit the retro changelog to dns subtree.
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: ipa
Version: 7.4
Hardware: Unspecified
OS: Unspecified
high
unspecified
Target Milestone: rc
: ---
Assignee: IPA Maintainers
QA Contact: ipa-qe
URL:
Whiteboard:
Depends On:
Blocks: 1420851
TreeView+ depends on / blocked
 
Reported: 2016-10-24 14:47 UTC by German Parente
Modified: 2019-12-16 07:12 UTC (History)
11 users (show)

Fixed In Version: ipa-4.5.4-4.el7
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-04-10 16:40:25 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2018:0918 None None None 2018-04-10 16:41:35 UTC

Description German Parente 2016-10-24 14:47:07 UTC
Description of problem:

In Ipa, we are using content syncronization plugin to let DNS component be aware of changes under "cn=dns"

We had a bug already fixed to exclude ipaca backend from retro changelog:

bz 1346321

this has helped to prevent some deadlocks.

But there's still some improvement here. In ipa we have a high number of updates in some situations like provisioning users in environments with large static groups or/and compat plugin enabled.

Each time an update is done to the db, a retro changelog operation is logged.

I think we will improve performance a lot and have less issues (lots of issues with updates in the retro changelog are seen at customer side) if we could restrict, in IPA, the retro changelog operations to dns subtree only.

We don't have many dns updates.

I am asking here whether an enhancement is possible to restrict by DNS subtree.

Instead of excluding "o=ipaca", the change will be similar but this time we need an include suffix clause:


nsslapd-include-suffix: "cn=dns,...."

Comment 7 Petr Vobornik 2016-11-25 13:11:30 UTC
Upstream ticket:
https://fedorahosted.org/freeipa/ticket/6515

Comment 11 Tomas Krizek 2017-10-26 10:41:38 UTC
Fixed upstream
master:
https://pagure.io/freeipa/c/624b34ab2b16d6de58ffb1a5c18210ee0c24e57b

Comment 16 thierry bordaz 2017-12-19 13:59:16 UTC
There are two aspects to test in that BZ:

- performance: make sure that the throughput of the updates improves in 7.5 vs 7.4.
  The improvements is due to retroCL logging updates only if the updates are under 'cn=dns,...'.
  So you may target updates outside of this subtree (for example, user-add/mod/del, host-add/mod/del, group-add/mod/del... ) and should notice higher throughput in 7.5.
  If updates target 'cn=dns' entries, I would expect lower improvements. Indeed IPA commands are doing updates in several subtrees so even if some of them land into 'cn=dns,..' so others will be outside

- functional test: this is to verify that the amount of records into retroCL is less in 7.5 than in 7.4
  You may tune logging to log retroCL updates (ADD under 'cn=changelog).
ldapmodify -D "cn=directory manager'...
dn: cn=Retro Changelog Plugin,cn=plugins,cn=config
changetype: modify
replace: nsslapd-logAccess
nsslapd-logAccess: on
-
dn: cn=config
replace: nsslapd-accesslog-level
nsslapd-accesslog-level: 260
   Then count in access logs how many internal ADDs like:
[19/Dec/2017:14:43:38.279145287 +0100] conn=Internal op=-1 ADD dn="changenumber=19,cn=changelog"
[19/Dec/2017:14:43:38.286007231 +0100] conn=Internal op=-1 RESULT err=0 tag=48 nentries=0 etime=0.0006866293

   An other option is to count the record number in changelog
ldapsearch -D DM -b "cn=changelog" "(objectclass=*)" dn | wc -l

Comment 23 Nikhil Dehadrai 2018-01-05 10:14:29 UTC
389-ds-base-1.3.7.5-11.el7.x86_64
ipa-server-4.5.4-7.el7.x86_64


Verified this bug in two ways 

A) Functional test against RHEL 74z and RHEL 75, with following observations:

IPA master RHEL7.4.z:
--------------------------
1) Setup IPA-master.
2) Create approximately 5K users entries to it.
3) Create approximately 5k host entries to it.
4) Once complete check the output of following commands

[root@nd74zipa ~]# rpm -q ipa-server
ipa-server-4.5.0-22.el7_4.x86_64

[root@nd74zipa ~]# ldapsearch -D "cn=directory manager" -w Secret123 -b "cn=changelog" "(objectclass=*)" dn | wc -l
75038

[root@nd74zipa ~]# grep -c 'conn=Internal' /var/log/dirsrv/slapd-TESTRELM-TEST/access
5100


IPA master RHEL7.5:
--------------------------
1) Setup IPA-master.
2) Create approximately 5K users entries to it.
3) Create approximately 5k host entries to it.
4) Once complete check the output of following commands

[root@ndipa ~]# rpm -q ipa-server
ipa-server-4.5.4-7.el7.x86_64


[root@ndipa ~]# ldapsearch -D "cn=directory manager" -w Secret123 -b "cn=changelog" "(objectclass=*)" dn | wc -l
32

[root@ndipa ~]# grep -c 'conn=Internal' /var/log/dirsrv/slapd-TESTRELM-TEST/access
104


B) basic performance test for WRITE operations, with following observations:

Noticed that for WRITE operations there is slight improvement.

System under test:
RHEL 74z and RHEL 7.5

Setup:
--------
RAM: 2GB, SWAP : 2GB
Setup to add upto 10K users, such that when IPA  is populated with 5Kusers add and time 1 user on top of it on, similarly followed the same after 10k user setup:

Non-Tuned (sec):
----------------
74z :: user-add  :: 5k ::  1.244
75  :: user-add  :: 5k ::  1.098 (-0.146 improvement)

74z :: user-add  :: 10k ::  1.25
75  :: user-add  :: 10k ::  1.154 (-0.096 improvement)

The time taken by the script to populate entire data of 10K users  on:
RHEL 74z : 28minutes 10s
RHEL 75  : 14minutes 39s

Thus based on above observations and confirmations in comment#18 and Comment#22, marking the status of this bugzilla to "VERIFIED"

Comment 26 errata-xmlrpc 2018-04-10 16:40:25 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:0918


Note You need to log in before you can comment on or make changes to this bug.