Bug 1695685

Summary: IPA Web UI is slow to display user details page.
Product: Red Hat Enterprise Linux 8 Reporter: Gaurav Swami <gswami>
Component: ipaAssignee: IPA Maintainers <ipa-maint>
Status: ASSIGNED --- QA Contact: ipa-qe <ipa-qe>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: ---CC: cheimes, frenaud, ftrivino, ksiddiqu, ndehadra, pasik, pvoborni, rcritten, ssidhaye, sumenon, tscherf
Target Milestone: rcKeywords: Desktop, Triaged
Target Release: ---Flags: frenaud: needinfo? (sumenon)
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Gaurav Swami 2019-04-03 15:25:35 UTC
Description of problem:

When customer searches for a user via IPA WebUI, it takes really long time to display the user's details. The same user can be queries via CLI, within few seconds.

The ipa cert-find command also takes time to complete, also sometime it fails with error `Unable to communicate with CMS (500)`.

----------------------------------
[root@srvlx40235 ~]# time ipa cert-find --users=p017079 --all
----------------------
0 certificates matched
----------------------
----------------------------
Number of entries returned 0
----------------------------

real    1m6.283s
user    0m0.529s
sys     0m0.095s
[root@srvlx40235 ~]#
----------------------------------


The customer got 125053 entries under `ou=certificateRepository,ou=ca,o=ipaca` when user details displayed on IPA web-UI.

It looks for every single certificates in the database.

----
[29/Mar/2019:13:26:41.424312347 +0100] conn=265 op=2411 SRCH base="ou=certificateRepository,ou=ca,o=ipaca" scope=1 filter="(certStatus=*)" attrs=ALL
[29/Mar/2019:13:26:41.447810348 +0100] conn=265 op=2411 SORT serialno 
[29/Mar/2019:13:26:41.447824289 +0100] conn=265 op=2411 VLV 0:2147483647:99990:125049 99991:125049 (0)
[29/Mar/2019:13:26:47.409452926 +0100] conn=265 op=2410 RESULT err=4 tag=101 nentries=10000 etime=15.1550448031
[29/Mar/2019:13:26:47.720526716 +0100] conn=265 op=2412 SRCH base="ou=certificateRepository,ou=ca,o=ipaca" scope=1 filter="(certStatus=*)" attrs=ALL
[29/Mar/2019:13:26:47.813523986 +0100] conn=265 op=2412 SORT serialno 
[29/Mar/2019:13:26:47.813541377 +0100] conn=265 op=2412 VLV 0:2147483647:49995:125049 49996:125049 (0)
[29/Mar/2019:13:26:56.972922273 +0100] conn=265 op=2411 RESULT err=4 tag=101 nentries=10000 etime=15.0548812925
[29/Mar/2019:13:26:57.490204761 +0100] conn=265 op=2413 SRCH base="ou=certificateRepository,ou=ca,o=ipaca" scope=1 filter="(certStatus=*)" attrs=ALL
[29/Mar/2019:13:26:57.547431167 +0100] conn=265 op=2413 SORT serialno 
[29/Mar/2019:13:26:57.547445102 +0100] conn=265 op=2413 VLV 0:2147483647:109989:125049 109990:125049 (0)
[29/Mar/2019:13:26:59.977039120 +0100] conn=265 op=2414 SRCH base="ou=certificateRepository,ou=ca,o=ipaca" scope=0 filter="(|(objectClass=*)(objectClass=ldapsubentry))" attrs="description"
[29/Mar/2019:13:26:59.977385581 +0100] conn=265 op=2414 RESULT err=0 tag=101 nentries=1 etime=0.0001801085
[29/Mar/2019:13:27:03.623850364 +0100] conn=265 op=2412 RESULT err=4 tag=101 nentries=10000 etime=15.1095023500
[29/Mar/2019:13:27:03.774106766 +0100] conn=265 op=2415 SRCH base="ou=certificateRepository,ou=ca,o=ipaca" scope=1 filter="(certStatus=*)" attrs=ALL
[29/Mar/2019:13:27:03.949381244 +0100] conn=265 op=2415 SORT serialno 
[29/Mar/2019:13:27:03.949395594 +0100] conn=265 op=2415 VLV 0:2147483647:59994:125049 59995:125049 (0)
[29/Mar/2019:13:27:13.444654998 +0100] conn=265 op=-1 fd=81 closed error 104 (Connection reset by peer) - TCP connection reset by peer.
---------

Version-Release number of selected component (if applicable):


ipa-server-4.6.4-10.el7_6.3.x86_64


Additional info:

Comment 3 Christian Heimes 2019-04-03 15:57:51 UTC
Upstream ticket:
https://pagure.io/freeipa/issue/7901

Comment 4 Christian Heimes 2019-04-03 15:58:22 UTC
I implemented an optimization for service and host certificates in RHBZ#1669012. The optimization doesn't apply to user certificates. I think it should be possible to optimize the user cert case, too.

Fraser, do you remember if all user certificates have "cn=$username"?

Comment 5 Rob Crittenden 2019-04-03 17:09:51 UTC
I'm not sure it is safe to make assumptions on what the subject will contain. It could be keyed on uid, e-mail and/or cn, or some other available attribute (like principal).

Comment 6 Christian Heimes 2019-04-03 17:13:14 UTC
The cert_request API endpoint enforces CN == username, https://github.com/freeipa/freeipa/blob/350954589774499d99bf87cb5631c664bb0707c4/ipaserver/plugins/cert.py#L747-L753 . Are there any ways to work around this restrictions?

Comment 7 Rob Crittenden 2019-04-04 12:16:14 UTC
Sorry, you're right. I was thinking more broadly about the all the searches that happen for a certificate. Given that we require cn=<uid> it is safe to query the CA on this IMHO.

Comment 8 Christian Heimes 2019-04-09 07:14:40 UTC
Fixed upstream
master:
https://pagure.io/freeipa/c/8a5dc1b375db94c4e722fa725f48eb16d032f1aa

Comment 14 Sumedh Sidhaye 2019-12-18 09:07:42 UTC
    Test data:
    About 30k users
    About 15k certs
     
    Test Machine memory
    [root@master ~]# free -mh
                  total        used        free      shared  buff/cache   available
    Mem:           3.7G        2.8G        143M         51M        756M        562M
    Swap:          2.0G        763M        1.3G
    [root@master ~]#
     
    CLI queries
     
    [root@master ~]# time ipa cert-find --subject=test103707 --all
    ipa: WARNING: Search result has been truncated: Configured size limit exceeded
    ---------------------
    1 certificate matched
    ---------------------
      Issuing CA: ipa
      Certificate: MIIE8DCCA9igAwIBAgICL3UwDQYJKoZIhvcNAQELBQAwOTEXMBUGA1UECgwOVEVTVFJFQUxNLlRFU1QxHjAcBgNVBAMMFUNlcnRpZmljYXRlIEF1dGhvcml0eTAeFw0xOTEyMTcxMTUzMTdaFw0yMTEyMTcxMTUzMTdaMDwxFzAVBgNVBAoMDlRFU1RSRUFMTS5URVNUMSEwHwYDVQQDDBh0ZXN0MTAzNzA3LnRlc3RyZWxtLnRlc3QwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDME/AB1omh41fAasyDsSUGiB/5dXaOPVNI6LfL7Ui+36iDT+a/DPc9Unr+oKE+uMKG4NJilRo5EXdsUGJHZVmUOVyiVAHyLUOjNkZmTBzeiw5JFERRNCEQREqqdV4sFufucr0qYn53aY2+oE61oW0Q9bBCi2Q4Sn+BWJkMQT7Qajb57xheIzrJaqKdPLazH2P+IEDPxoyBM9PXKuXGvWqQdrpbACgGlfDUEV9bymPfFzHpOgTPVxoPKI8l0fgnH9aQEQkceHjdDxP0MYSb1nfbGUMqCqVqt/DRlIF7Fql6Bv0QdnFCkSkblwXw0cXujlpK4LyxxJUmCYqwN01Y+zTpAgMBAAGjggH9MIIB+TAfBgNVHSMEGDAWgBTNVHjlypVlKsjcWUCGUssE1G7MTjBABggrBgEFBQcBAQQ0MDIwMAYIKwYBBQUHMAGGJGh0dHA6Ly9pcGEtY2EudGVzdHJlYWxtLnRlc3QvY2Evb2NzcDAOBgNVHQ8BAf8EBAMCBPAwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMCMHkGA1UdHwRyMHAwbqA2oDSGMmh0dHA6Ly9pcGEtY2EudGVzdHJlYWxtLnRlc3QvaXBhL2NybC9NYXN0ZXJDUkwuYmluojSkMjAwMQ4wDAYDVQQKDAVpcGFjYTEeMBwGA1UEAwwVQ2VydGlmaWNhdGUgQXV0aG9yaXR5MB0GA1UdDgQWBBRCrFYJVADwkjUF9WcrV8NyFKxodjCBygYDVR0RBIHCMIG/ghh0ZXN0MTAzNzA3LnRlc3RyZWxtLnRlc3SgSQYKKwYBBAGCNxQCA6A7DDl0ZXN0c2VydmljZTEwMzcwNy90ZXN0MTAzNzA3LnRlc3RyZWxtLnRlc3RAVEVTVFJFQUxNLlRFU1SgWAYGKwYBBQICoE4wTKAQGw5URVNUUkVBTE0uVEVTVKE4MDagAwIBAaEvMC0bEXRlc3RzZXJ2aWNlMTAzNzA3Gxh0ZXN0MTAzNzA3LnRlc3RyZWxtLnRlc3QwDQYJKoZIhvcNAQELBQADggEBAGoYnC+6u3uS9B+ajkUXdMJpOT+uWw1b1e5aUzgXFXsWRyMcOhK8ULgOikJrNllVk2HojSp/hbsS+6zqVLTG5WSsaaDlWpLJeesr7AL+GkooOpQViPHzUtImSslyQOclHj27kkVAckmxWKcUtQjMNzSo431kCpIp3TggEr0fQ7Uwk9P5ZGIn7tgAwXyelzILjy36oKYed4BKFG3AuaNMYTNjWTBhUVSu1qmPMTBeLEzmZpqD3sB2sfRpBDQHB6noXCV0tSD4EyneIHwv8zAoOtZ8yVC3Ds/WbdAL2ymgQAJSes9XNSmrpUCZfMpeQBK3PQaLuQM/USLNm0QAvpIUJS4=
      Certificate chain: MIIE8DCCA9igAwIBAgICL3UwDQYJKoZIhvcNAQELBQAwOTEXMBUGA1UECgwOVEVTVFJFQUxNLlRFU1QxHjAcBgNVBAMMFUNlcnRpZmljYXRlIEF1dGhvcml0eTAeFw0xOTEyMTcxMTUzMTdaFw0yMTEyMTcxMTUzMTdaMDwxFzAVBgNVBAoMDlRFU1RSRUFMTS5URVNUMSEwHwYDVQQDDBh0ZXN0MTAzNzA3LnRlc3RyZWxtLnRlc3QwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDME/AB1omh41fAasyDsSUGiB/5dXaOPVNI6LfL7Ui+36iDT+a/DPc9Unr+oKE+uMKG4NJilRo5EXdsUGJHZVmUOVyiVAHyLUOjNkZmTBzeiw5JFERRNCEQREqqdV4sFufucr0qYn53aY2+oE61oW0Q9bBCi2Q4Sn+BWJkMQT7Qajb57xheIzrJaqKdPLazH2P+IEDPxoyBM9PXKuXGvWqQdrpbACgGlfDUEV9bymPfFzHpOgTPVxoPKI8l0fgnH9aQEQkceHjdDxP0MYSb1nfbGUMqCqVqt/DRlIF7Fql6Bv0QdnFCkSkblwXw0cXujlpK4LyxxJUmCYqwN01Y+zTpAgMBAAGjggH9MIIB+TAfBgNVHSMEGDAWgBTNVHjlypVlKsjcWUCGUssE1G7MTjBABggrBgEFBQcBAQQ0MDIwMAYIKwYBBQUHMAGGJGh0dHA6Ly9pcGEtY2EudGVzdHJlYWxtLnRlc3QvY2Evb2NzcDAOBgNVHQ8BAf8EBAMCBPAwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMCMHkGA1UdHwRyMHAwbqA2oDSGMmh0dHA6Ly9pcGEtY2EudGVzdHJlYWxtLnRlc3QvaXBhL2NybC9NYXN0ZXJDUkwuYmluojSkMjAwMQ4wDAYDVQQKDAVpcGFjYTEeMBwGA1UEAwwVQ2VydGlmaWNhdGUgQXV0aG9yaXR5MB0GA1UdDgQWBBRCrFYJVADwkjUF9WcrV8NyFKxodjCBygYDVR0RBIHCMIG/ghh0ZXN0MTAzNzA3LnRlc3RyZWxtLnRlc3SgSQYKKwYBBAGCNxQCA6A7DDl0ZXN0c2VydmljZTEwMzcwNy90ZXN0MTAzNzA3LnRlc3RyZWxtLnRlc3RAVEVTVFJFQUxNLlRFU1SgWAYGKwYBBQICoE4wTKAQGw5URVNUUkVBTE0uVEVTVKE4MDagAwIBAaEvMC0bEXRlc3RzZXJ2aWNlMTAzNzA3Gxh0ZXN0MTAzNzA3LnRlc3RyZWxtLnRlc3QwDQYJKoZIhvcNAQELBQADggEBAGoYnC+6u3uS9B+ajkUXdMJpOT+uWw1b1e5aUzgXFXsWRyMcOhK8ULgOikJrNllVk2HojSp/hbsS+6zqVLTG5WSsaaDlWpLJeesr7AL+GkooOpQViPHzUtImSslyQOclHj27kkVAckmxWKcUtQjMNzSo431kCpIp3TggEr0fQ7Uwk9P5ZGIn7tgAwXyelzILjy36oKYed4BKFG3AuaNMYTNjWTBhUVSu1qmPMTBeLEzmZpqD3sB2sfRpBDQHB6noXCV0tSD4EyneIHwv8zAoOtZ8yVC3Ds/WbdAL2ymgQAJSes9XNSmrpUCZfMpeQBK3PQaLuQM/USLNm0QAvpIUJS4=,
                         MIIDlDCCAnygAwIBAgIBATANBgkqhkiG9w0BAQsFADA5MRcwFQYDVQQKDA5URVNUUkVBTE0uVEVTVDEeMBwGA1UEAwwVQ2VydGlmaWNhdGUgQXV0aG9yaXR5MB4XDTE5MTIwOTA1NTIyMloXDTM5MTIwOTA1NTIyMlowOTEXMBUGA1UECgwOVEVTVFJFQUxNLlRFU1QxHjAcBgNVBAMMFUNlcnRpZmljYXRlIEF1dGhvcml0eTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAKld8aPzbfnNZ6L3uNnj6A/WPY3R8WItoIs7smKLFF6NV1RMrw7jov9WEvakJthirn88ViXDSk7zmKq5G5SEx/uCoFcQ47OcBwGzjv3oRWLGSPMOam2TfaTfLnkDdDIHu33FeKKPDk1cTnyTc2iECZXY7UjlO/aMGp1xuxR8uP/qd0083LlIeHaCjDsjwmbF7GPnl+5bD8yt09s2SHpZOsyopWgPIMzJZ8NNJ/BhlTG93RHpXF1xNd2qZ+Nou+4lDD0BeeDjYUoXkg2Wp66tRF5GrKZz2SfIGEEcZYWf8WkBI756Yd6JBUrOD+RujRHBagoQqcJiSkiUmnlFP7YRFsECAwEAAaOBpjCBozAfBgNVHSMEGDAWgBTNVHjlypVlKsjcWUCGUssE1G7MTjAPBgNVHRMBAf8EBTADAQH/MA4GA1UdDwEB/wQEAwIBxjAdBgNVHQ4EFgQUzVR45cqVZSrI3FlAhlLLBNRuzE4wQAYIKwYBBQUHAQEENDAyMDAGCCsGAQUFBzABhiRodHRwOi8vaXBhLWNhLnRlc3RyZWFsbS50ZXN0L2NhL29jc3AwDQYJKoZIhvcNAQELBQADggEBAIIg54wFzMaY8faeSSl1EgSC4dhcEvtRN650iIveBWb/+fe7wFGZL6+vx0IPvVp3z+mzzKBj+fZ2dx5gX2ik+bC0ey4l4Xv7B4cuf8NbRNkWdJZRQuqugFuKp6JL/oCTgq2N7rQ+WMkrQV1Ct9In6E8kY48sEtPzeWmlG0CU004bGSnkfqNNaoQ4b3OhnMi6RCivuTAEg5yqPnJemcyR3SPlgLgQXv3/VG1me9Y2aHarrBnGazAX7XfG5sIgy/Ctk9psBBItEmsbAF/WZptvKXPpVDLhxeni+AhQZwjD5zVhOQDqUo4XuDtcGoGU3TNdI122ANABVU+WsO4Ft2f9EAE=
      Subject: CN=test103707.testrelm.test,O=TESTREALM.TEST
      Subject DNS name: test103707.testrelm.test
      Subject UPN: testservice103707/test103707.testrelm.test
      Subject Kerberos principal name: testservice103707/test103707.testrelm.test
      Subject Other Name: 1.3.6.1.4.1.311.20.2.3:DDl0ZXN0c2VydmljZTEwMzcwNy90ZXN0MTAzNzA3LnRlc3RyZWxtLnRlc3RAVEVTVFJFQUxNLlRFU1Q=,
                          1.3.6.1.5.2.2:MEygEBsOVEVTVFJFQUxNLlRFU1ShODA2oAMCAQGhLzAtGxF0ZXN0c2VydmljZTEwMzcwNxsYdGVzdDEwMzcwNy50ZXN0cmVsbS50ZXN0
      Issuer: CN=Certificate Authority,O=TESTREALM.TEST
      Not Before: Tue Dec 17 11:53:17 2019 UTC
      Not After: Fri Dec 17 11:53:17 2021 UTC
      Fingerprint (SHA1): b7:a5:20:51:12:1c:0e:36:99:93:39:fe:ec:1a:b4:93:d7:9b:87:3a
      Fingerprint (SHA256): 20:75:ba:43:8d:cb:fc:3c:95:0c:c4:9e:ad:22:db:cc:af:f1:77:fe:e9:4c:86:05:7e:18:32:ed:6b:ce:88:b6
      Serial number: 12149
      Serial number (hex): 0x2F75
      Status: VALID
      Revoked: False
    ----------------------------
    Number of entries returned 1
    ----------------------------
     
    real    0m23.673s
    user    0m0.500s
    sys     0m0.063s
    [root@master ~]#
     
     
    [root@master ~]# time ipa cert-find --users=test103707 --all
    ----------------------
    0 certificates matched
    ----------------------
    ----------------------------
    Number of entries returned 0
    ----------------------------
     
    real    0m0.884s
    user    0m0.448s
    sys     0m0.073s
    [root@master ~]#
     
     
     
    [root@master ~]# time ipa user-find --login user2121
    --------------
    1 user matched
    --------------
      User login: user2121
      First name: user2121
      Last name: user2121
      Home directory: /other-home/user2121
      Login shell: /bin/zsh
      Principal name: user2121
      Principal alias: user2121
      Email address: user2121
      UID: 405002123
      GID: 405002123
      SSH public key fingerprint: SHA256:cStA9o5TRSARbeketEOooMUMSWRSsArIAXloBZ4vNsE public key test (ssh-rsa)
      Account disabled: False
    ----------------------------
    Number of entries returned 1
    ----------------------------
     
    real    0m0.685s
    user    0m0.516s
    sys     0m0.077s
    [root@master ~]#
     
     
     
    WEB UI:
     
    1. trying to find a user
    https://master.testrealm.test/ipa/ui/#/e/user/search//filter=user21212
     
    almost instant
     
     
    2. Accessing the Certificates tab, it takes a lot of time to load the data
    about ~15-20 seconds
     
     
     
    3. Querying for a particular user https://master.testrealm.test/ipa/ui/#/e/cert/search//search_option=subject&filter=test103707
     
    The above took 27-28 seconds


Based on above observations, marking the bug in ASSIGNED state since querying is taking significant time and this needs further investigation.

Comment 19 Sumedh Sidhaye 2019-12-18 09:43:01 UTC
Build used for testing:

[root@master ~]# rpm -qa ipa-*
ipa-common-4.6.6-11.el7.noarch
ipa-client-common-4.6.6-11.el7.noarch
ipa-client-4.6.6-11.el7.x86_64
ipa-server-4.6.6-11.el7.x86_64
ipa-server-trust-ad-4.6.6-11.el7.x86_64
ipa-server-common-4.6.6-11.el7.noarch
ipa-server-dns-4.6.6-11.el7.noarch

Comment 24 Florence Blanc-Renaud 2020-02-14 15:13:19 UTC
Thank you taking your time and submitting this request for Red Hat Enterprise Linux 7. Unfortunately, this bug cannot be kept even as a stretch goal and was postponed to RHEL8.