Description of problem: we use mongodb 2.4 in latest satelitte-6.1 mongodb-3.0 provides better scalability and performance improvements. Its high time to update mongodb Version-Release number of selected component (if applicable): How reproducible: Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Additional info:
Mongo 2.4.9 will have a CPU spike during syncs. For example: * create 10 products, each with 10 yum repos. However, make the repos identical. You can accomplish this by creating a zoo repo and then symlinking zoo1-10 to zoo, so the URLs are different. When testing this, I used 32 pkgs in the repo. * sync each repo result: high mongo CPU usage, and this warning from pulp: pulp: pulp_rpm.plugins.importers.yum.purge:INFO: Purging duplicate NEVRA can take significantly longer in versions of mongodb lower than 2.6. Consider upgrading mongodb if cleaning duplicate packages takes an unreasonably long time. Note that the sync time remains constant over time, and there's no add'l memory consumption.
On 6.2, syncing a small repo that is defined 100x: # time hammer --username admin --password changeme repository synchronize --organization perf-org-1 --product perf-product-10 -- name perf-repo-10 [..................................................................................................................................................] [100%] No new packages. real 0m49.286s after upgrading to mongodb 2.6.11: # time hammer --username admin --password changeme repository synchronize --organization perf-org-1 --product perf-product-10 --name perf-repo-10 [..................................................................................................................................................] [100%] No new packages. real 0m12.354s note: i re-ran createrepo on the repo between each run to make sure the repomd.xml timestamp was getting updated.
Chris You tried concurrent sync? --Pradeep
This was not concurrent, it was just one sync. I think there is a code change in Pulp 2.8 that makes sync significantly faster, but it uses newer (2.6+) versions of mongodb. I did not try with Mongo 3, just 2.6.
Per 6.3 planning, moving out non acked bugs to the backlog
*** Bug 1439083 has been marked as a duplicate of this bug. ***
As information of a satellite in operation how big mongodb filesystems can grow over time: /dev/mapper/vg_li_lc_1017_app1-lv_mongodb 150G 130G 21G 87% /var/lib/mongodb
Current mongodb size on by Sat6 instance that handles frequent Content (View) changes: [crash/LI] root@li-lc-1017:~# df -h /var/lib/mongodb Filesystem Size Used Avail Use% Mounted on /dev/mapper/vg_li_lc_1017_app1-lv_mongodb 200G 185G 16G 93% /var/lib/mongodb That is about 10-15G per month growth after i posted comment 11 above.
Verified in Satellite 6.4 # mongo MongoDB shell version v3.4.9 connecting to: mongodb://127.0.0.1:27017 MongoDB server version: 3.4.9 Server has startup warnings: 2018-08-22T18:08:22.599+0200 I CONTROL [initandlisten] 2018-08-22T18:08:22.599+0200 I CONTROL [initandlisten] ** WARNING: You are running on a NUMA machine. 2018-08-22T18:08:22.599+0200 I CONTROL [initandlisten] ** We suggest launching mongod like this to avoid performance problems: 2018-08-22T18:08:22.599+0200 I CONTROL [initandlisten] ** numactl --interleave=all mongod [other options] 2018-08-22T18:08:22.599+0200 I CONTROL [initandlisten] 2018-08-22T18:08:22.599+0200 I CONTROL [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always'. 2018-08-22T18:08:22.599+0200 I CONTROL [initandlisten] ** We suggest setting it to 'never' 2018-08-22T18:08:22.599+0200 I CONTROL [initandlisten] 2018-08-22T18:08:22.599+0200 I CONTROL [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/defrag is 'always'. 2018-08-22T18:08:22.599+0200 I CONTROL [initandlisten] ** We suggest setting it to 'never' 2018-08-22T18:08:22.599+0200 I CONTROL [initandlisten] > db.version() 3.4.9
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2018:2927