RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 904119 - Significant decrease in migration performance
Summary: Significant decrease in migration performance
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: ipa
Version: 6.4
Hardware: Unspecified
OS: Unspecified
urgent
high
Target Milestone: rc
: ---
Assignee: Martin Kosek
QA Contact: Namita Soman
URL:
Whiteboard:
Depends On:
Blocks: 908308 910554
TreeView+ depends on / blocked
 
Reported: 2013-01-25 14:00 UTC by Jenny Severance
Modified: 2013-11-21 20:48 UTC (History)
5 users (show)

Fixed In Version: ipa-3.0.0-26.el6
Doc Type: Bug Fix
Doc Text:
During migration, users were added to the default users group for each user added. Adding users to a group which had thousands consumed time and caused a significant performance slowdown. With this patch, users are now added in batches of 100, which provides a considerable performance boost over the previous method.
Clone Of:
: 910554 (view as bug list)
Environment:
Last Closed: 2013-11-21 20:48:56 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2013:1651 0 normal SHIPPED_LIVE ipa bug fix and enhancement update 2013-11-21 00:39:40 UTC

Description Jenny Severance 2013-01-25 14:00:12 UTC 
Description of problem:
Migration scenario is 10,000 users and 12 groups

RHEL 6.4

with compat plugin enabled

10 hours 14 minutes

:: [   LOG    ] :: slapd pid : 19259
:: [   LOG    ] :: Before migration free memory : 14951256064
:: [   LOG    ] :: Before migration slapd VmRSS : 28620kB
:: [   LOG    ] :: Before migration slapd VmHWM : 29096kB
:: [   LOG    ] :: ======================= Migration started: Thu Jan 17 05:13:32 EST 2013 ========================
:: [   LOG    ] :: EXECUTING: time -p echo mysecret | ipa migrate-ds --with-compat ldap://directoryserver.testrelm.com:389
:: [   PASS   ] :: Migration completed successfully.
:: [   LOG    ] :: ======================= Migration finished: Thu Jan 17 15:28:02 EST 2013 ========================
:: [   LOG    ] :: slapd pid : 19259
:: [   LOG    ] :: After migration free memory : 12555595776
:: [   LOG    ] :: After migration slapd VmRSS : 1183860kB
:: [   LOG    ] :: After migration slapd VmHWM : 1188992kB

with compat plugin disabled

18 hours 39 minutes

:: [   LOG    ] :: slapd pid : 19443
:: [   LOG    ] :: Before migration free memory : 15077851136
:: [   LOG    ] :: Before migration slapd VmRSS : 24744kB
:: [   LOG    ] :: Before migration slapd VmHWM : 24744kB
:: [   LOG    ] :: ======================= Migration started: Sat Jan 19 01:43:02 EST 2013 ========================
:: [   LOG    ] :: EXECUTING: echo mysecret | ipa migrate-ds ldap://directoryserver.testrelm.com:389
:: [   PASS   ] :: Migration completed successfully.
:: [   LOG    ] :: ======================= Migration finished: Sat Jan 19 20:22:05 EST 2013 ========================
:: [   LOG    ] :: slapd pid : 19443
:: [   LOG    ] :: After migration free memory : 12738027520
:: [   LOG    ] :: After migration slapd VmRSS : 1212460kB
:: [   LOG    ] :: After migration slapd VmHWM : 1217564kB



RHEL 6.3

with compat plugin enabled

7 hours 17 minutes

:: [   LOG    ] :: slapd pid : 21414
:: [   LOG    ] :: Before migration free memory : 14818889728
:: [   LOG    ] :: Before migration slapd VmRSS : 38200kB
:: [   LOG    ] :: Before migration slapd VmHWM : 41652kB
:: [   LOG    ] :: ======================= Migration started: Mon Apr  2 11:12:05 EDT 2012 ========================
:: [   LOG    ] :: EXECUTING: time -p echo mysecret | ipa migrate-ds --with-compat ldap://directoryserver.testrelm.com:389
:: [   PASS   ] :: Migration completed successfully.
:: [   LOG    ] :: ======================= Migration finished: Mon Apr  2 18:29:56 EDT 2012 ========================
:: [   LOG    ] :: slapd pid : 21414
:: [   LOG    ] :: After migration free memory : 13182754816
:: [   LOG    ] :: After migration slapd VmRSS : 772656kB
:: [   LOG    ] :: After migration slapd VmHWM : 772656kB


with compat plugin disabled

4 hours 36 minutes

:: [   LOG    ] :: slapd pid : 21409
:: [   LOG    ] :: Before migration free memory : 132102623232
:: [   LOG    ] :: Before migration slapd VmRSS : 25168kB
:: [   LOG    ] :: Before migration slapd VmHWM : 25168kB
:: [   LOG    ] :: ======================= Migration started: Mon Apr  9 13:26:12 EDT 2012 ========================
:: [   LOG    ] :: EXECUTING: echo mysecret | ipa migrate-ds ldap://directoryserver.testrelm.com:389
:: [   PASS   ] :: Migration completed successfully.
:: [   LOG    ] :: ======================= Migration finished: Mon Apr  9 18:02:18 EDT 2012 ========================
:: [   LOG    ] :: slapd pid : 21409
:: [   LOG    ] :: After migration free memory : 130853560320
:: [   LOG    ] :: After migration slapd VmRSS : 331900kB
:: [   LOG    ] :: After migration slapd VmHWM : 332432kB 

Version-Release number of selected component (if applicable):
ipa-server-3.0.0-22.el6.x86_64
389-ds-base-1.2.11.15-10.el6.x86_64

How reproducible:


Steps to Reproduce:
1.
2.
3.
  
Actual results:


Expected results:


Additional info:
Comment 2 RHEL Program Management 2013-01-29 06:49:24 UTC 
This request was not resolved in time for the current release.
Red Hat invites you to ask your support representative to
propose this request, if still desired, for consideration in
the next release of Red Hat Enterprise Linux.
Comment 3 Dmitri Pal 2013-01-31 22:10:22 UTC 
Upstream ticket:
https://fedorahosted.org/freeipa/ticket/3386
Comment 5 Martin Kosek 2013-02-05 16:20:06 UTC 
Fixed upstream:
master: https://fedorahosted.org/freeipa/changeset/53c94361d6d119a68a645ac3e5f702c31afab831
ipa-3-1: https://fedorahosted.org/freeipa/changeset/c796bead5ed9ed52df91b09290028548289440cf
ipa-3-0: https://fedorahosted.org/freeipa/changeset/793aa1168d0c60af8c07efd12337b4e7240f5124
Comment 8 Martin Kosek 2013-02-08 15:28:13 UTC 
Regression fixed. migrate-ds command could crash with Internal Error if there was no group to migrate and --continue option was used.

master: https://fedorahosted.org/freeipa/changeset/08b84befbc40faca21ffe44dcc69aaf498a2f9f2
ipa-3-1: https://fedorahosted.org/freeipa/changeset/aa798f3ca0a08e457e7316b7017db5647d1df550
ipa-3-0: https://fedorahosted.org/freeipa/changeset/f0566b3856320334119968b81b6ea31a25bf8ea6
Comment 9 Namita Soman 2013-02-12 19:00:11 UTC 
This bug also needs to be cloned for inclusion in RHEL 7.0
Comment 11 Namita Soman 2013-09-12 02:35:12 UTC 
Verified using ipa-server-3.0.0-35

with compat plugin enabled

2 hours 44 minutes

:: [ 13:25:05 ] ::  slapd pid : 20288
:: [ 13:25:05 ] ::  Before migration free memory : 3498647552
:: [ 13:25:05 ] ::  Before migration slapd VmRSS : 30772kB
:: [ 13:25:05 ] ::  Before migration slapd VmHWM : 30872kB
:: [ 13:25:05 ] ::  ======================= Migration started: Mon Sep  9 13:25:05 EDT 2013 ========================
:: [ 13:25:05 ] ::  EXECUTING: time -p echo Secret123 | ipa migrate-ds --with-compat ldap://ipaqa64vmh.testrelm.com:389
:: [   PASS   ] :: Migration completed successfully. 
:: [ 16:09:32 ] ::  ======================= Migration finished: Mon Sep  9 16:09:32 EDT 2013 ========================
:: [ 16:09:32 ] ::  slapd pid : 20288
:: [ 16:09:32 ] ::  After migration free memory : 1738301440
:: [ 16:09:32 ] ::  After migration slapd VmRSS : 1184088kB
:: [ 16:09:32 ] ::  After migration slapd VmHWM : 1188280kB

with compat plugin disabled

2 hours 36 minutes

:: [ 10:53:24 ] ::  slapd pid : 18548
:: [ 10:53:24 ] ::  Before migration free memory : 15083929600
:: [ 10:53:24 ] ::  Before migration slapd VmRSS : 28908kB
:: [ 10:53:24 ] ::  Before migration slapd VmHWM : 28908kB
:: [ 10:53:24 ] ::  ======================= Migration started: Wed Sep 11 10:53:24 EDT 2013 ========================
:: [ 10:53:24 ] ::  EXECUTING: echo Secret123 | ipa migrate-ds ldap://cloud-qe-1-vm-1.testrelm.com:389
:: [   PASS   ] :: Migration completed successfully. 
:: [ 12:29:08 ] ::  ======================= Migration finished: Wed Sep 11 12:29:08 EDT 2013 ========================
:: [ 12:29:08 ] ::  slapd pid : 18548
:: [ 12:29:08 ] ::  After migration free memory : 13061668864
:: [ 12:29:08 ] ::  After migration slapd VmRSS : 1055668kB
:: [ 12:29:08 ] ::  After migration slapd VmHWM : 1061276kB
Comment 13 errata-xmlrpc 2013-11-21 20:48:56 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2013-1651.html
Note You need to log in before you can comment on or make changes to this bug.