RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1388135 - [RFE] limit the retro changelog to dns subtree.
Summary: [RFE] limit the retro changelog to dns subtree.
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: ipa
Version: 7.4
Hardware: Unspecified
OS: Unspecified
high
unspecified
Target Milestone: rc
: ---
Assignee: IPA Maintainers
QA Contact: ipa-qe
URL:
Whiteboard:
Depends On:
Blocks: 1420851
TreeView+ depends on / blocked
 
Reported: 2016-10-24 14:47 UTC by German Parente
Modified: 2021-09-09 11:58 UTC (History)
11 users (show)

Fixed In Version: ipa-4.5.4-4.el7
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-04-10 16:40:25 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2018:0918 0 None None None 2018-04-10 16:41:35 UTC

Description German Parente 2016-10-24 14:47:07 UTC 
Description of problem:

In Ipa, we are using content syncronization plugin to let DNS component be aware of changes under "cn=dns"

We had a bug already fixed to exclude ipaca backend from retro changelog:

bz 1346321

this has helped to prevent some deadlocks.

But there's still some improvement here. In ipa we have a high number of updates in some situations like provisioning users in environments with large static groups or/and compat plugin enabled.

Each time an update is done to the db, a retro changelog operation is logged.

I think we will improve performance a lot and have less issues (lots of issues with updates in the retro changelog are seen at customer side) if we could restrict, in IPA, the retro changelog operations to dns subtree only.

We don't have many dns updates.

I am asking here whether an enhancement is possible to restrict by DNS subtree.

Instead of excluding "o=ipaca", the change will be similar but this time we need an include suffix clause:


nsslapd-include-suffix: "cn=dns,...."
Comment 7 Petr Vobornik 2016-11-25 13:11:30 UTC 
Upstream ticket:
https://fedorahosted.org/freeipa/ticket/6515
Comment 11 Tomas Krizek 2017-10-26 10:41:38 UTC 
Fixed upstream
master:
https://pagure.io/freeipa/c/624b34ab2b16d6de58ffb1a5c18210ee0c24e57b
Comment 12 Tomas Krizek 2017-10-27 08:43:17 UTC 
Fixed upstream
ipa-4-5:
https://pagure.io/freeipa/c/d03d338f990586b9fc0d8a623ecaa7c9fa4db665
Fixed upstream
ipa-4-6:
https://pagure.io/freeipa/c/68a7e478d4b44a96a6b2684e60203f622e719dd9
Comment 16 thierry bordaz 2017-12-19 13:59:16 UTC 
There are two aspects to test in that BZ:

- performance: make sure that the throughput of the updates improves in 7.5 vs 7.4.
  The improvements is due to retroCL logging updates only if the updates are under 'cn=dns,...'.
  So you may target updates outside of this subtree (for example, user-add/mod/del, host-add/mod/del, group-add/mod/del... ) and should notice higher throughput in 7.5.
  If updates target 'cn=dns' entries, I would expect lower improvements. Indeed IPA commands are doing updates in several subtrees so even if some of them land into 'cn=dns,..' so others will be outside

- functional test: this is to verify that the amount of records into retroCL is less in 7.5 than in 7.4
  You may tune logging to log retroCL updates (ADD under 'cn=changelog).
ldapmodify -D "cn=directory manager'...
dn: cn=Retro Changelog Plugin,cn=plugins,cn=config
changetype: modify
replace: nsslapd-logAccess
nsslapd-logAccess: on
-
dn: cn=config
replace: nsslapd-accesslog-level
nsslapd-accesslog-level: 260
   Then count in access logs how many internal ADDs like:
[19/Dec/2017:14:43:38.279145287 +0100] conn=Internal op=-1 ADD dn="changenumber=19,cn=changelog"
[19/Dec/2017:14:43:38.286007231 +0100] conn=Internal op=-1 RESULT err=0 tag=48 nentries=0 etime=0.0006866293

   An other option is to count the record number in changelog
ldapsearch -D DM -b "cn=changelog" "(objectclass=*)" dn | wc -l
Comment 23 Nikhil Dehadrai 2018-01-05 10:14:29 UTC 
389-ds-base-1.3.7.5-11.el7.x86_64
ipa-server-4.5.4-7.el7.x86_64


Verified this bug in two ways 

A) Functional test against RHEL 74z and RHEL 75, with following observations:

IPA master RHEL7.4.z:
--------------------------
1) Setup IPA-master.
2) Create approximately 5K users entries to it.
3) Create approximately 5k host entries to it.
4) Once complete check the output of following commands

[root@nd74zipa ~]# rpm -q ipa-server
ipa-server-4.5.0-22.el7_4.x86_64

[root@nd74zipa ~]# ldapsearch -D "cn=directory manager" -w Secret123 -b "cn=changelog" "(objectclass=*)" dn | wc -l
75038

[root@nd74zipa ~]# grep -c 'conn=Internal' /var/log/dirsrv/slapd-TESTRELM-TEST/access
5100


IPA master RHEL7.5:
--------------------------
1) Setup IPA-master.
2) Create approximately 5K users entries to it.
3) Create approximately 5k host entries to it.
4) Once complete check the output of following commands

[root@ndipa ~]# rpm -q ipa-server
ipa-server-4.5.4-7.el7.x86_64


[root@ndipa ~]# ldapsearch -D "cn=directory manager" -w Secret123 -b "cn=changelog" "(objectclass=*)" dn | wc -l
32

[root@ndipa ~]# grep -c 'conn=Internal' /var/log/dirsrv/slapd-TESTRELM-TEST/access
104


B) basic performance test for WRITE operations, with following observations:

Noticed that for WRITE operations there is slight improvement.

System under test:
RHEL 74z and RHEL 7.5

Setup:
--------
RAM: 2GB, SWAP : 2GB
Setup to add upto 10K users, such that when IPA  is populated with 5Kusers add and time 1 user on top of it on, similarly followed the same after 10k user setup:

Non-Tuned (sec):
----------------
74z :: user-add  :: 5k ::  1.244
75  :: user-add  :: 5k ::  1.098 (-0.146 improvement)

74z :: user-add  :: 10k ::  1.25
75  :: user-add  :: 10k ::  1.154 (-0.096 improvement)

The time taken by the script to populate entire data of 10K users  on:
RHEL 74z : 28minutes 10s
RHEL 75  : 14minutes 39s

Thus based on above observations and confirmations in comment#18 and Comment#22, marking the status of this bugzilla to "VERIFIED"
Comment 26 errata-xmlrpc 2018-04-10 16:40:25 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:0918
Note You need to log in before you can comment on or make changes to this bug.