Bug 1295741 - [RFE] Update mongodb to 3.X
[RFE] Update mongodb to 3.X
Status: ON_QA
Product: Red Hat Satellite 6
Classification: Red Hat
Component: Performance (Show other bugs)
6.1.4
Unspecified Unspecified
medium Severity high (vote)
: Unspecified
: --
Assigned To: Chris Roberts
https://projects.theforeman.org/issue...
: FutureFeature, Triaged
: 1439083 (view as bug list)
Depends On:
Blocks: 1122832 1545876 1546813
  Show dependency treegraph
 
Reported: 2016-01-05 06:35 EST by Pradeep Kumar Surisetty
Modified: 2018-07-15 17:29 EDT (History)
11 users (show)

See Also:
Fixed In Version:
Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
Foreman Issue Tracker 23030 None None None 2018-03-27 10:47 EDT
Foreman Issue Tracker 23272 None None None 2018-04-13 13:28 EDT
Platform MBU Aha 72 None None None 2018-01-22 06:21 EST

  None (edit)
Description Pradeep Kumar Surisetty 2016-01-05 06:35:46 EST
Description of problem:

we use mongodb 2.4 in latest satelitte-6.1
mongodb-3.0 provides better scalability and performance improvements.
Its high time to update mongodb

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:
Comment 1 Chris Duryee 2016-03-17 12:47:42 EDT
Mongo 2.4.9 will have a CPU spike during syncs.

For example:

* create 10 products, each with 10 yum repos. However, make the repos identical. You can accomplish this by creating a zoo repo and then symlinking zoo1-10 to zoo, so the URLs are different. When testing this, I used 32 pkgs in the repo.

* sync each repo

result: high mongo CPU usage, and this warning from pulp:

pulp: pulp_rpm.plugins.importers.yum.purge:INFO: Purging duplicate NEVRA can take significantly longer in versions of mongodb lower than 2.6. Consider upgrading mongodb if cleaning duplicate packages takes an unreasonably long time.

Note that the sync time remains constant over time, and there's no add'l memory consumption.
Comment 2 Chris Duryee 2016-03-17 13:15:42 EDT
On 6.2, syncing a small repo that is defined 100x:

# time hammer --username admin --password changeme repository synchronize --organization perf-org-1 --product perf-product-10 --
name perf-repo-10
[..................................................................................................................................................] [100%]
No new packages.


real    0m49.286s


after upgrading to mongodb 2.6.11:

# time hammer --username admin --password changeme repository synchronize --organization perf-org-1 --product perf-product-10
 --name perf-repo-10
[..................................................................................................................................................] [100%]
No new packages.


real    0m12.354s


note: i re-ran createrepo on the repo between each run to make sure the repomd.xml timestamp was getting updated.
Comment 3 Pradeep Kumar Surisetty 2016-03-23 00:23:45 EDT
Chris

You tried concurrent sync? 

--Pradeep
Comment 4 Chris Duryee 2016-03-23 08:44:50 EDT
This was not concurrent, it was just one sync. I think there is a code change in Pulp 2.8 that makes sync significantly faster, but it uses newer (2.6+) versions of mongodb.

I did not try with Mongo 3, just 2.6.
Comment 7 Bryan Kearney 2016-07-08 16:48:46 EDT
Per 6.3 planning, moving out non acked bugs to the backlog
Comment 9 Bryan Kearney 2017-04-06 16:22:43 EDT
*** Bug 1439083 has been marked as a duplicate of this bug. ***
Comment 11 Peter Vreman 2017-08-07 09:18:12 EDT
As information of a satellite in operation how big mongodb filesystems can grow over time:

/dev/mapper/vg_li_lc_1017_app1-lv_mongodb   150G  130G   21G  87% /var/lib/mongodb
Comment 12 Peter Vreman 2017-12-18 11:14:31 EST
Current mongodb size on by Sat6 instance that handles frequent Content (View) changes:

[crash/LI] root@li-lc-1017:~# df -h /var/lib/mongodb
Filesystem                                 Size  Used Avail Use% Mounted on
/dev/mapper/vg_li_lc_1017_app1-lv_mongodb  200G  185G   16G  93% /var/lib/mongodb

That is about 10-15G per month growth after i posted comment 11 above.

Note You need to log in before you can comment on or make changes to this bug.