Bug 1350962 - db:migrate on large database takes long time and large amount of memory
Summary: db:migrate on large database takes long time and large amount of memory
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat CloudForms Management Engine
Classification: Red Hat
Component: Performance
Version: 5.4.0
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: GA
: 5.5.5
Assignee: Jason Frey
QA Contact: Alex Krzos
URL:
Whiteboard:
Depends On: 1337159
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-06-28 20:42 UTC by John Prause
Modified: 2016-07-26 15:36 UTC (History)
11 users (show)

Fixed In Version: 5.5.5.2
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 1337159
Environment:
Last Closed: 2016-07-26 15:36:10 UTC
Category: ---
Cloudforms Team: ---
Target Upstream Version:


Attachments (Terms of Use)
Files from migration on 5552 (86.20 KB, application/zip)
2016-07-19 12:10 UTC, Alex Krzos
no flags Details


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2016:1488 normal SHIPPED_LIVE CFME 5.5.5 bug fixes and enhancement update 2016-07-26 19:28:27 UTC

Comment 2 Alex Krzos 2016-07-19 12:08:18 UTC
Testing 5.5.5.2 I see some reduction in timing but the most noticeable savings is in memory utilization:

Peak appliance used memory during rake db:migrate
5.5.4.1 - 11.3GiB
5.5.5.2 - 2.7GiB

db:migrate Timing:
5.5.4.1 - 6hr 45m
5.5.5.2 - 5hr 43m

Database still bloats on event_streams:

event_streams - 17GB


vmdb_production=# SELECT pg_size_pretty(pg_database_size('vmdb_production'));
 pg_size_pretty
----------------
 37 GB
(1 row)

vmdb_production=# VACUUM FULL event_streams;
VACUUM
vmdb_production=# SELECT pg_size_pretty(pg_database_size('vmdb_production'));
 pg_size_pretty
----------------
 26 GB
(1 row)


FixSerializedReportsForRailsFour decreases in timing:
5.5.4.1 - 21999.6597s
5.5.5.2 - 19057.7571s

As 

RenameEmsEventTableToEventStream increases in timing:
5.5.4.1 - 876.8407s
5.5.5.2 - 1194.6601s

Though net affect on db:migrate is lower by an hour in this testbed.

Comment 4 Alex Krzos 2016-07-19 12:10:35 UTC
Created attachment 1181613 [details]
Files from migration on 5552

Comment 6 errata-xmlrpc 2016-07-26 15:36:10 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-1488.html


Note You need to log in before you can comment on or make changes to this bug.