Description of problem: Publishing and promoting content-views are growing in time required to publish/promote as more content-views are created. Timing can be reduced by restarting foreman-proxy however not to the degree of a clean system. Version-Release number of selected component (if applicable): Satellite 6.0.3 RHEL 6.5 2.6.32-431.20.3 katello-1.5.0-26.el6sat.noarch foreman-1.6.0.21-1.el6sat.noarch candlepin-0.9.19-1.el6_5.noarch pulp-server-2.4.0-0.23.beta.el6sat.noarch puppet-3.6.2-1.el6sat.noarch How reproducible: Always Steps to Reproduce: 1. Deploy Sat6 Server 2. Upload Manifest 3. Sync Content 4. Create, add repo, publish, promote more than 20 Content Views 5. Review timings required to publish/promote each content-view Actual results: Here is 20 Content-View Publish Timings: time hammer content-view publish --id $cvid 136.263 146.127 143.102 150.898 149.587 154.589 154.205 161.326 165.865 175.199 180.863 189.981 191.544 197.365 200.213 216.728 216.512 231.184 236.033 245.959 Expected results: Content-view publishing should not grow as more Additional info: As a work around, restarting foreman-proxy reduces the timing but it continues to grow quickly there after.
Since this issue was entered in Red Hat Bugzilla, the release flag has been set to ? to ensure that it is properly evaluated for this release.
This also affects re-publishing: The following shows the timing for a publish on the same content-view 10 times, then restarting foreman-proxy and followed by 10 more publishes on the same content-view. 128.119 126.899 142.159 153.075 167.625 179.12 188.724 197.218 216.234 228.535 # service foreman-proxy restart 130.982 144.965 143.197 156.973 167.467 180.09 187.695 202.582 214.01 223.046
This also affects publishing of a content-view that only contains puppet-modules: 96.252 93.866 108.077 123.208 138.402 154.435 184.113 180.13 233.713 229.495 330.35 337.455 # service foreman-proxy restart 111.616 118.396 168.445 179.871 225.195 208.824 251.559 253.249 319.768 323.996 345.444 414.449 # service foreman-proxy restart 139.394 148.857 166.8 219.706 185.686 248.607 Additionally, the more and more content-views that are published/republished/promoted there appears to be no way to reduce the timing to its original state. Restarting foreman-proxy helps somewhat however it doesn't restore it all the way down to the original.
Working with Alex, we were able to determine that pulp is not responsible for the increasing times. All pulp tasks seem to be executing within consistent time frames.
Although I have no concrete numbers, I see the same thing happening.
Created redmine issue http://projects.theforeman.org/issues/9647 from this bug
Opened a PR that hopefully addresses this issue. I was not able to replicate exactly, but previously for every publish or promote we were fetching all puppet classes from all environments from the puppet master (we were only importing from the one environment we cared about). The PR changes this to only fetch for the current relevant environment. In my testing with ~12 puppet modules published in ~5 content views, this brought down the total publish time from 30 seconds down to 10 seconds. I will leave it up to QA to validate that the timing does not increase quite as badly (since i was not able to reproduce exactly).
Moving to POST since upstream bug http://projects.theforeman.org/issues/9647 has been closed ------------- Justin Sherrill Applied in changeset commit:katello|a610d6c93a74e1e8c2ca2fdf930955ab99c07e99.
VERIFIED : # rpm -qa | grep foreman rubygem-hammer_cli_foreman_tasks-0.0.3.3-1.el6_6sat.noarch foreman-proxy-1.7.2.3-1.el6_6sat.noarch foreman-ovirt-1.7.2.10-1.el6_6sat.noarch ruby193-rubygem-foreman-tasks-0.6.12.1-1.el6_6sat.noarch rubygem-hammer_cli_foreman_bootdisk-0.1.2.5-1.el6_6sat.noarch ruby193-rubygem-foreman_abrt-0.0.5-2.el6_6sat.noarch foreman-compute-1.7.2.10-1.el6_6sat.noarch foreman-libvirt-1.7.2.10-1.el6_6sat.noarch ruby193-rubygem-foreman_discovery-2.0.0.6-1.el6_6sat.noarch rubygem-hammer_cli_foreman-0.1.4.6-1.el6_6sat.noarch foreman-selinux-1.7.2.8-1.el6_6sat.noarch foreman-gce-1.7.2.10-1.el6_6sat.noarch ruby193-rubygem-foreman_docker-1.2.0.3-1.el6_6sat.noarch ruby193-rubygem-foreman-redhat_access-0.0.9-1.el6_6sat.noarch qe-sat6-rhel66.usersys.redhat.com-foreman-client-1.0-1.noarch qe-sat6-rhel66.usersys.redhat.com-foreman-proxy-client-1.0-1.noarch foreman-debug-1.7.2.10-1.el6_6sat.noarch foreman-vmware-1.7.2.10-1.el6_6sat.noarch ruby193-rubygem-foreman_bootdisk-4.0.2.8-1.el6_6sat.noarch qe-sat6-rhel66.usersys.redhat.com-foreman-proxy-1.0-2.noarch foreman-1.7.2.10-1.el6_6sat.noarch ruby193-rubygem-foreman_hooks-0.3.7-2.el6_6sat.noarch rubygem-hammer_cli_foreman_discovery-0.0.1.2-1.el6_6sat.noarch ruby193-rubygem-foreman_gutterball-0.0.1.9-1.el6_6sat.noarch foreman-postgresql-1.7.2.10-1.el6_6sat.noarch # time hammer content-view publish --name con_view --organization 'Default Organization' /usr/lib/ruby/gems/1.8/gems/hammer_cli-0.1.4/lib/hammer_cli/./apipie/../abstract.rb:68: warning: already initialized constant DEFAULT_LABEL_INDENT [Foreman] Username: admin [Foreman] Password for admin: [......................................................................] [100%] real 0m13.538s user 0m1.174s sys 0m0.407s # time hammer content-view publish --name con_view1 --organization 'Default Organization' /usr/lib/ruby/gems/1.8/gems/hammer_cli-0.1.4/lib/hammer_cli/./apipie/../abstract.rb:68: warning: already initialized constant DEFAULT_LABEL_INDENT [Foreman] Username: admin [Foreman] Password for admin: [......................................................................] [100%] real 0m12.241s user 0m1.211s sys 0m0.425s
*** Bug 1195979 has been marked as a duplicate of this bug. ***
This bug is slated to be released with Satellite 6.1.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2015:1592