Hide Forgot
Description of problem: In Ipa, we are using content syncronization plugin to let DNS component be aware of changes under "cn=dns" We had a bug already fixed to exclude ipaca backend from retro changelog: bz 1346321 this has helped to prevent some deadlocks. But there's still some improvement here. In ipa we have a high number of updates in some situations like provisioning users in environments with large static groups or/and compat plugin enabled. Each time an update is done to the db, a retro changelog operation is logged. I think we will improve performance a lot and have less issues (lots of issues with updates in the retro changelog are seen at customer side) if we could restrict, in IPA, the retro changelog operations to dns subtree only. We don't have many dns updates. I am asking here whether an enhancement is possible to restrict by DNS subtree. Instead of excluding "o=ipaca", the change will be similar but this time we need an include suffix clause: nsslapd-include-suffix: "cn=dns,...."
Upstream ticket: https://fedorahosted.org/freeipa/ticket/6515
Fixed upstream master: https://pagure.io/freeipa/c/624b34ab2b16d6de58ffb1a5c18210ee0c24e57b
Fixed upstream ipa-4-5: https://pagure.io/freeipa/c/d03d338f990586b9fc0d8a623ecaa7c9fa4db665 Fixed upstream ipa-4-6: https://pagure.io/freeipa/c/68a7e478d4b44a96a6b2684e60203f622e719dd9
There are two aspects to test in that BZ: - performance: make sure that the throughput of the updates improves in 7.5 vs 7.4. The improvements is due to retroCL logging updates only if the updates are under 'cn=dns,...'. So you may target updates outside of this subtree (for example, user-add/mod/del, host-add/mod/del, group-add/mod/del... ) and should notice higher throughput in 7.5. If updates target 'cn=dns' entries, I would expect lower improvements. Indeed IPA commands are doing updates in several subtrees so even if some of them land into 'cn=dns,..' so others will be outside - functional test: this is to verify that the amount of records into retroCL is less in 7.5 than in 7.4 You may tune logging to log retroCL updates (ADD under 'cn=changelog). ldapmodify -D "cn=directory manager'... dn: cn=Retro Changelog Plugin,cn=plugins,cn=config changetype: modify replace: nsslapd-logAccess nsslapd-logAccess: on - dn: cn=config replace: nsslapd-accesslog-level nsslapd-accesslog-level: 260 Then count in access logs how many internal ADDs like: [19/Dec/2017:14:43:38.279145287 +0100] conn=Internal op=-1 ADD dn="changenumber=19,cn=changelog" [19/Dec/2017:14:43:38.286007231 +0100] conn=Internal op=-1 RESULT err=0 tag=48 nentries=0 etime=0.0006866293 An other option is to count the record number in changelog ldapsearch -D DM -b "cn=changelog" "(objectclass=*)" dn | wc -l
389-ds-base-1.3.7.5-11.el7.x86_64 ipa-server-4.5.4-7.el7.x86_64 Verified this bug in two ways A) Functional test against RHEL 74z and RHEL 75, with following observations: IPA master RHEL7.4.z: -------------------------- 1) Setup IPA-master. 2) Create approximately 5K users entries to it. 3) Create approximately 5k host entries to it. 4) Once complete check the output of following commands [root@nd74zipa ~]# rpm -q ipa-server ipa-server-4.5.0-22.el7_4.x86_64 [root@nd74zipa ~]# ldapsearch -D "cn=directory manager" -w Secret123 -b "cn=changelog" "(objectclass=*)" dn | wc -l 75038 [root@nd74zipa ~]# grep -c 'conn=Internal' /var/log/dirsrv/slapd-TESTRELM-TEST/access 5100 IPA master RHEL7.5: -------------------------- 1) Setup IPA-master. 2) Create approximately 5K users entries to it. 3) Create approximately 5k host entries to it. 4) Once complete check the output of following commands [root@ndipa ~]# rpm -q ipa-server ipa-server-4.5.4-7.el7.x86_64 [root@ndipa ~]# ldapsearch -D "cn=directory manager" -w Secret123 -b "cn=changelog" "(objectclass=*)" dn | wc -l 32 [root@ndipa ~]# grep -c 'conn=Internal' /var/log/dirsrv/slapd-TESTRELM-TEST/access 104 B) basic performance test for WRITE operations, with following observations: Noticed that for WRITE operations there is slight improvement. System under test: RHEL 74z and RHEL 7.5 Setup: -------- RAM: 2GB, SWAP : 2GB Setup to add upto 10K users, such that when IPA is populated with 5Kusers add and time 1 user on top of it on, similarly followed the same after 10k user setup: Non-Tuned (sec): ---------------- 74z :: user-add :: 5k :: 1.244 75 :: user-add :: 5k :: 1.098 (-0.146 improvement) 74z :: user-add :: 10k :: 1.25 75 :: user-add :: 10k :: 1.154 (-0.096 improvement) The time taken by the script to populate entire data of 10K users on: RHEL 74z : 28minutes 10s RHEL 75 : 14minutes 39s Thus based on above observations and confirmations in comment#18 and Comment#22, marking the status of this bugzilla to "VERIFIED"
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2018:0918