Bug 1229764

Summary: pki cert-find could be time consuming
Product: Red Hat Enterprise Linux 7 Reporter: German Parente <gparente>
Component: pki-coreAssignee: Matthew Harmsen <mharmsen>
Status: CLOSED ERRATA QA Contact: Asha Akkiangady <aakkiang>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 7.1CC: alee, cfu, cheimes, edewata, ftweedal, gagriogi, gkapoor, jmagne, mharmsen, nkinder, rpattath
Target Milestone: rc   
Target Release: 7.2   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: pki-core-10.2.5-3.el7 Doc Type: Bug Fix
Doc Text:
* The "ipa cert-find" and "pki cert-find" commands previously took an unexpectedly long time to complete when a large number certificates were present in the database. With this update, the default LDAP filter generated by the commands has been changed to "(certStatus=*)" to make the query run by the commands match an existing virtual list view (VLV) index. This prevents the described "ipa cert-find" and "pki cert-find" performance problems.
 Story Points: ---
Clone Of: Environment:
Last Closed: 2015-11-19 09:22:52 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description German Parente 2015-06-09 15:04:44 UTC 
Description of problem:

"ipa cert-find" and "pki cert-find" commands with no additional parameters could be very time consuming if there are too many certificates.

The problem is that the query the commands are doing to the dirsrv instance is not matching any exisiting vlvindex. So, the query is unindexed.


Version-Release number of selected component (if applicable):

pki-tools-10.1.2-7.el7.x86_64


How reproducible: always

Steps to Reproduce:

do "pki cert-find" and see the access logs. We should find something like:

[09/Jun/2015:16:05:09 +0200] conn=7 op=2 BIND dn="" method=sasl version=3 mech=GSSAPI
[09/Jun/2015:16:05:09 +0200] conn=7 op=2 RESULT err=0 tag=97 nentries=0 etime=0 dn="uid=admin,cn=users,cn=accounts,dc=example,dc=org"
[09/Jun/2015:16:05:09 +0200] conn=7 op=3 SRCH base="cn=ipaconfig,cn=etc,dc=example,dc=org" scope=0 filter="(objectClass=*)" attrs=ALL
[09/Jun/2015:16:05:09 +0200] conn=7 op=3 RESULT err=0 tag=101 nentries=1 etime=0
[09/Jun/2015:16:05:09 +0200] conn=7 op=4 SRCH base="cn=masters,cn=ipa,cn=etc,dc=example,dc=org" scope=2 filter="(&(objectClass=ipaConfigObject)(cn=CA))" attrs=ALL
[09/Jun/2015:16:05:09 +0200] conn=7 op=4 RESULT err=0 tag=101 nentries=2 etime=0
[09/Jun/2015:16:05:09 +0200] conn=3 op=7 SRCH base="ou=certificateRepository,ou=ca,o=ipaca" scope=1 filter="(&(serialno=*))" attrs=ALL
[09/Jun/2015:16:05:09 +0200] conn=3 op=7 SORT serialno (14)
[09/Jun/2015:16:05:09 +0200] conn=3 op=7 VLV 0:10:A 14:14 (0)
[09/Jun/2015:16:05:09 +0200] conn=3 op=7 RESULT err=0 tag=101 nentries=1 etime=0 notes=U
[09/Jun/2015:16:05:09 +0200] conn=3 op=8 SRCH base="ou=certificateRepository,ou=ca,o=ipaca" scope=1 filter="(&(serialno=*))" attrs=ALL
[09/Jun/2015:16:05:09 +0200] conn=3 op=8 SORT serialno (14)
[09/Jun/2015:16:05:09 +0200] conn=3 op=8 VLV 0:10:0:14 1:14 (0)
[09/Jun/2015:16:05:09 +0200] conn=3 op=8 RESULT err=0 tag=101 nentries=11 etime=0 notes=U
[09/Jun/2015:16:05:10 +0200] conn=3 op=9 SRCH base="ou=certificateRepository,ou=ca,o=ipaca" scope=1 filter="(&(serialno=*))" attrs=ALL
[09/Jun/2015:16:05:10 +0200] conn=3 op=9 SORT serialno (14)
[09/Jun/2015:16:05:10 +0200] conn=3 op=9 VLV 0:10:10:14 11:14 (0)
[09/Jun/2015:16:05:10 +0200] conn=3 op=9 RESULT err=0 tag=101 nentries=4 etime=1 notes=U
[09/Jun/2015:16:05:10 +0200] conn=7 op=5 UNBIND
[09/Jun/2015:16:05:10 +0200] conn=7 op=5 fd=92 closed - U1


all the searches showing "notes=U"

The query 

base="ou=certificateRepository,ou=ca,o=ipaca" scope=1 filter="(&(serialno=*))"

is not matching any vlvindex.


Additional info:

Workaround is to add a new vlvindex like this:

---------------------------------------------------------
dn: cn=allCertsSerial-pki-tomcat,cn=ipaca,cn=ldbm database,cn=plugins,cn=config
objectClass: top
objectClass: vlvSearch
vlvFilter: (&(serialno=*))
vlvScope: 1
vlvBase: ou=certificateRepository,ou=ca,o=ipaca
cn: allCertsSerial-pki-tomcat
numSubordinates: 1

dn: cn=allCertsSerial-pki-tomcatIndex,cn=allCertsSerial-pki-tomcat,cn=ipaca,cn=ldbm database,cn=plugins,cn=config
vlvUses: 0
vlvEnabled: 0
vlvSort: serialno
objectClass: top
objectClass: vlvIndex
cn: allCertsSerial-pki-tomcatIndex
-------------------------------------------------------------

+ apply it:


/var/lib/dirsrv/scripts-EXAMPLE-ORG/vlvindex -n ipaca -T  allCertsSerial-pki-tomcatIndex
Comment 3 Matthew Harmsen 2015-07-07 04:38:01 UTC 
Per PKI TRAC Ticket #1449 - pki cert-find could be time consuming: add VLV index for new installations, Endi Dewata addressed this issue:

Fixed in master: ac5447a8e0bac5112882be700a17a9274e322adc

This was additionally addressed in the patch to RHEL 7.2 pki-core.
Comment 4 gagriogi 2015-07-09 07:48:19 UTC 
Is it possible to get it on 7.1 as well since 7.2 won't be out for a few more months?
Comment 6 Endi Sukma Dewata 2015-08-28 16:25:22 UTC 
Steps to verify the fix:
1. Install CA
2. Execute pki cert-find
3. Check DS access log

The cert-find command should generate an LDAP search operation with filter "(certstatus=*)" instead of "(&(serialno=*))". The new filter will match an existing VLV, so it will fix the performance issue with a large set of data.
Comment 7 Roshni 2015-08-28 19:41:50 UTC 
[root@cloud-qe-7 pki-tomcat]# rpm -qi pki-ca
Name        : pki-ca
Version     : 10.2.6
Release     : 8.el7pki
Architecture: noarch
Install Date: Fri 28 Aug 2015 03:33:27 PM EDT
Group       : System Environment/Daemons
Size        : 2416291
License     : GPLv2
Signature   : (none)
Source RPM  : pki-core-10.2.6-8.el7pki.src.rpm
Build Date  : Tue 25 Aug 2015 01:18:45 AM EDT
Build Host  : x86-025.build.eng.bos.redhat.com
Relocations : (not relocatable)
Packager    : Red Hat, Inc. <http://bugzilla.redhat.com/bugzilla>
Vendor      : Red Hat, Inc.
URL         : http://pki.fedoraproject.org/
Summary     : Certificate System - Certificate Authority

Verification steps as in comment 6 :

DS access log had the following

[28/Aug/2015:15:35:57 -0400] conn=218 op=14 SRCH base="ou=certificateRepository,ou=ca,dc=pki-ca" scope=1 filter="(certStatus=*)" attrs=ALL
[28/Aug/2015:15:35:57 -0400] conn=218 op=14 SORT serialno (6)
[28/Aug/2015:15:35:57 -0400] conn=218 op=14 VLV 0:20:A 6:6 (0)
[28/Aug/2015:15:35:57 -0400] conn=218 op=14 RESULT err=0 tag=101 nentries=1 etime=0 notes=U
[28/Aug/2015:15:35:57 -0400] conn=218 op=15 SRCH base="ou=certificateRepository,ou=ca,dc=pki-ca" scope=1 filter="(certStatus=*)" attrs=ALL
[28/Aug/2015:15:35:57 -0400] conn=218 op=15 SORT serialno (6)
[28/Aug/2015:15:35:57 -0400] conn=218 op=15 VLV 0:20:0:6 1:6 (0)
[28/Aug/2015:15:35:57 -0400] conn=218 op=15 RESULT err=0 tag=101 nentries=6 etime=0 notes=U


Reproduced the issue using the following build pki-ca-10.2.5-2.el7.noarch.rpm and notice the following in DS access log

[28/Aug/2015:15:31:24 -0400] conn=154 op=24 SRCH base="ou=certificateRepository,ou=ca,dc=pki-ca" scope=1 filter="(&(serialno=*))" attrs=ALL
[28/Aug/2015:15:31:24 -0400] conn=154 op=24 SORT serialno (6)
[28/Aug/2015:15:31:24 -0400] conn=154 op=24 VLV 0:10:A 6:6 (0)
[28/Aug/2015:15:31:24 -0400] conn=154 op=24 RESULT err=0 tag=101 nentries=1 etime=0 notes=U
[28/Aug/2015:15:31:24 -0400] conn=154 op=25 SRCH base="ou=certificateRepository,ou=ca,dc=pki-ca" scope=1 filter="(&(serialno=*))" attrs=ALL
[28/Aug/2015:15:31:24 -0400] conn=154 op=25 SORT serialno (6)
[28/Aug/2015:15:31:24 -0400] conn=154 op=25 VLV 0:10:0:6 1:6 (0)
[28/Aug/2015:15:31:24 -0400] conn=154 op=25 RESULT err=0 tag=101 nentries=6 etime=0 notes=U
[28/Aug/2015:15:32:17 -0400] conn=166 op=10 SRCH base="ou=sessions,ou=Security Domain,dc=pki-ca" scope=2 filter="(objectClass=securityDomainSessionEntry)" attrs="cn"
[28/Aug/2015:15:32:17 -0400] conn=166 op=10 RESULT err=32 tag=101 nentries=0 etime=0
Comment 8 errata-xmlrpc 2015-11-19 09:22:52 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-2276.html