Description of problem: Optimize package profile task processing time Version-Release number of selected component (if applicable): Satellite 6.2.11 How reproducible: Always Steps to Reproduce: 1. It was found that on a heavily loaded Satellite box, the package profile task processing time can be longer which leads to consequent host registration failures. Actual results: package profile update tasks take a long time to run Expected results: package profile update task processing should be optimized to run faster. Additional info:
Created redmine issue http://projects.theforeman.org/issues/20682 from this bug
Some initial changes for this are in the following upstream PR: https://github.com/Katello/katello/pull/6912 Note: this PR does not currently address cleaning up the duplicates that may exist within the katello_installed_packages table. Those duplicates would have been caused by having entries added for the same package on N different hosts.
Upstream bug assigned to bbuckingham
Created attachment 1318342 [details] rhel6_hotfix
Created attachment 1318343 [details] rhel7_hotfix
=== HOTFIX INSTRUCTIONS FOR SATELLITE 6.2.11 ONLY === 1. Download attached file tfm-rubygem-katello-hotfix-rhel6.tar.xz or tfm-rubygem-katello-hotfix-rhel7.tar.xz depending on your Satellite host OS version 2. Stop service on Satellite katello-service stop 3. Extract the tarball and install the packages in Satellite 4. Upgrade Satellite (hotfix has table index creation) satellite-installer --upgrade 5. [optional] To clean duplicate installed packages: foreman-rake katello:clean_installed_packages
Moving this bug to POST for triage into Satellite 6 since the upstream issue http://projects.theforeman.org/issues/20682 has been resolved.
Created attachment 1323848 [details] el7 hotfix
Created attachment 1323849 [details] el6 hotfix
Added new hotfix that adds an additional feature: After running the "foreman-rake clean_installed_packages" script, a new setting will be flipped to true. The "bulk_query_installed_packages" setting. this will flip the queries to using a more efficient bulk query. Same instructions as before.
Created attachment 1324512 [details] el7 hotfix
Created attachment 1324513 [details] el6 hotfix
New hotfixes, uploaded. Simply update from the existing ones using yum
*** Bug 1483136 has been marked as a duplicate of this bug. ***
Created attachment 1343800 [details] Sat 6.2.12 el7 hotfix
Created attachment 1343801 [details] Sat 6.2.12 el6 hotfix
=== HOTFIX INSTRUCTIONS FOR SATELLITE 6.2.12 ONLY === 1. Download attached file sat6.2.12-el7-hotfix.tgz or sat6.2.12-el6-hotfix.tgz depending on your Satellite host OS version 2. Stop service on Satellite katello-service stop 3. Extract the tarball and install the packages in Satellite 4. Upgrade Satellite (hotfix has table index creation) or optionally if you do wish to not run the full upgrade procedure you may run: foreman-rake db:migrate otherwise run: satellite-installer --upgrade 5. [optional] To clean duplicate installed packages: foreman-rake katello:clean_installed_packages This last step 5 will reduce the # of rows in your database table and improve Satellite performance.
Update to step (4) if you do the optional route of db:migrate: """ 4. Upgrade Satellite (hotfix has table index creation) or optionally if you do wish to not run the full upgrade procedure you may run: katello-service start foreman-rake db:migrate otherwise run: satellite-installer --upgrade """
Question Should the hotfix be applied only to the Satellite server? No need to apply it to the Capsules? Thanks
(In reply to Ilan Green from comment #44) > Question > Should the hotfix be applied only to the Satellite server? > No need to apply it to the Capsules? > > Thanks Only to Satellite, since the patched RPMs are katello ones and there is no katello on Capsules.
Moving this bug to POST for triage into Satellite 6 since the upstream issue http://projects.theforeman.org/issues/21691 has been resolved.
Verified in Satellite 6.2.13 Snap 1. Several customer db upgrades have successfully been completed to 6.2.13, with moderate (no more than 1 hour) increases in upgrade time. Subsequent upgrades did not show the same time increase. Outside of the upgrades, I'm seeing consistently short package profile processing time. Typical times range from around 1 to 4 seconds. See attached verification screenshot for specific processing times. Stressing the satellite pushed the processing time up to around 30 seconds. See second attached screenshot.
Created attachment 1366979 [details] verification screenshot 1
Created attachment 1366980 [details] verification screenshot 2
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2017:3491