Bug 1298288 - [RFE] Improve performance in large environments.
Summary: [RFE] Improve performance in large environments.
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: ipa
Version: 7.2
Hardware: Unspecified
OS: Unspecified
high
medium
Target Milestone: rc
: ---
Assignee: IPA Maintainers
QA Contact: Namita Soman
Aneta Šteflová Petrová
URL:
Whiteboard:
Keywords: FutureFeature
: 1360810 (view as bug list)
Depends On: 1138797 1196958 1301901
Blocks: 1313485 1292074 1296125 1351239
TreeView+ depends on / blocked
 
Reported: 2016-01-13 16:22 UTC by Petr Vobornik
Modified: 2016-11-04 05:49 UTC (History)
5 users (show)

(edit)
Server performance has improved in many areas

Some operations in Identity Management run much faster now. For example, this enhancement enables better scalability in large deployments exceeding 50,000 users and hosts. Most notably, the improvements include:
* Faster adding of users and hosts
* Faster Kerberos authentication for all commands
* Faster execution of the "ipa user-find" and "ipa host-find" commands

For information on how to reduce the time required for provisioning of a large number of entries, see https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html-single/Linux_Domain_Identity_Authentication_and_Policy_Guide/index.html#performance-tuning

Note that to make the find operations faster, the "ipa *-find" commands no longer show membership by default. To display the membership, add the "--all" option to "ipa *-find" or, alternatively, use the "ipa *-show" commands.
Clone Of:
: 1351239 (view as bug list)
(edit)
Last Closed: 2016-11-04 05:49:12 UTC


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2016:2404 normal SHIPPED_LIVE ipa bug fix and enhancement update 2016-11-03 13:56:18 UTC

Description Petr Vobornik 2016-01-13 16:22:05 UTC
At the moment, IPA performance is not great in large environments. E.g. user-add becomes the slower the more users exist or ipa-extom-plugin can exhaust DS worker threads if DS server is flooded with "IPA trusted domain ID mapper" extop operations.

More use cases and testing details will be added and specified in more details to determine the scope of this RFE.

Comment 3 Petr Vobornik 2016-01-29 14:23:08 UTC
Upstream ticket:
https://fedorahosted.org/freeipa/ticket/5597

Comment 4 Petr Vobornik 2016-01-29 14:23:15 UTC
Upstream ticket:
https://fedorahosted.org/freeipa/ticket/5599

Comment 5 Petr Vobornik 2016-02-19 14:49:30 UTC
Upstream ticket:
https://fedorahosted.org/freeipa/ticket/5448

Comment 6 Petr Vobornik 2016-04-14 14:14:38 UTC
Upstream ticket:
https://fedorahosted.org/freeipa/ticket/5788

Comment 7 Petr Vobornik 2016-04-14 14:31:58 UTC
Upstream ticket:
https://fedorahosted.org/freeipa/ticket/5802

Comment 8 Petr Vobornik 2016-05-06 15:35:40 UTC
Upstream ticket:
https://fedorahosted.org/freeipa/ticket/5866

Comment 9 Petr Vobornik 2016-06-02 12:52:34 UTC
Upstream ticket:
https://fedorahosted.org/freeipa/ticket/5914

Comment 10 Petr Vobornik 2016-06-06 14:29:28 UTC
https://fedorahosted.org/freeipa/ticket/5802 was unlinked. It requires bigger changes in member of plugin and therefore it is out of scope of 4.4 release.

Comment 11 Martin Kosek 2016-06-10 12:57:02 UTC
5597 was closed as invalid.

Comment 13 Petr Vobornik 2016-07-01 13:21:34 UTC
Ticket https://fedorahosted.org/freeipa/ticket/5788 won't be addressed in scope of this bz.

Comment 17 Martin Bašti 2016-07-27 13:20:05 UTC
Upstream ticket:
https://fedorahosted.org/freeipa/ticket/6098

Comment 18 Martin Bašti 2016-07-27 13:23:32 UTC
Certificate issues ticket #6098 were found.

Should be this moved back to assigned? (Nothing prevents QA to test other parts (users, hosts, provisioning))

Comment 19 Martin Bašti 2016-07-27 13:36:55 UTC
Upstream ticket:
https://fedorahosted.org/freeipa/ticket/6100

Comment 20 Martin Bašti 2016-07-27 14:23:48 UTC
Upstream ticket:
https://fedorahosted.org/freeipa/ticket/6118

Comment 21 Martin Bašti 2016-07-27 14:45:42 UTC
LGTM, thank you

Comment 22 Martin Bašti 2016-07-27 14:55:46 UTC
*** Bug 1360810 has been marked as a duplicate of this bug. ***

Comment 23 Martin Bašti 2016-07-27 15:17:24 UTC
Several bug found upstream, moving to assigned

Comment 24 Martin Bašti 2016-07-27 17:27:19 UTC
Fixed slow user-add (regression caused by kerberos aliases feature in 4.4) upstream
master:
https://fedorahosted.org/freeipa/changeset/807702c986976ade8005ec344fcd827f70b2ba2f

Comment 27 Nikhil Dehadrai 2016-09-23 13:42:07 UTC
IPA-server version: ipa-server-4.4.0-12.el7.x86_64

ENVIRONMENT:
------------
1. VM
2. RAM: 4GB RAM
3. Processor: 8
4. Tuned as per the details mentioned at:
https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7-Beta/html/Linux_Domain_Identity_Authentication_and_Policy_Guide/bulk-provisioning.html

5. Data population script:
https://github.com/freeipa/freeipa-tools/blob/master/create-test-data.py

Steps:
1. Setup IPA on RHEL 7.3.
2. Save the create-test-data.py to the machine and create ldif file using this python script.
 
# python create-test-data.py > my.ldif

Tune the machine as per the details.
3. Once done, capture the details at regular intervals for respective commands such that the each command is run 10 times and the avg time is taken into account.

Observations:
Using the setup and steps above following are the observations:
 
---------------------------------------------------------------------                  
        Task            7.3     7.2     Diff (7.3 – 7.2) (seconds)
---------------------------------------------------------------------
        User-add        5.03    9.62    -4.59
        User-find       4.35    8.71    -4.36
        User-show       3.06    8.8     -5.74
        Host-add        59.73   76.38   -16.65
        Host-find       61.13   80.23   -19.1
        Host-show       50.99   79.1    -28.11
        Group-add       3.34    5.65    -2.31
        Group-find      3.72    4.97    -1.25
        Group-show      2.91    3.95    -1.04
        Hostgroup-add   3.69    4.63    -0.94
        Hostgroup-find  4.57    5.62    -1.05
        Hostgroup-show  2.6     5.11    -2.51
        Sudorule-add    3.49    3.62    -0.13
        Sudorule-find   6.56    7.23    -0.67
        Sudorule-show   2.58    5.26    -2.68
        Hbacrule-add    3.34    3.4     -0.06
        Hbacrule-find   7.02    9.13    -2.11
        Hbacrule-show   2.58    5.63    -3.05

Thus, it is noticed that there is a significant improvement in IPA 7.3 command execution. Thus marking the status of bug to "VERIFIED".

Comment 29 errata-xmlrpc 2016-11-04 05:49:12 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-2404.html


Note You need to log in before you can comment on or make changes to this bug.