Red Hat Bugzilla – Bug 1217789
PurgeEmptyErrata and PurgeEmptyPackageGroups can timeout on large repos
Last modified: 2017-02-23 15:06:12 EST
a user has reported seeing timeouts on the mentioned tasks as part of a content view publish <pre> RestClient::RequestTimeout Request Timeout --- - /opt/rh/ruby193/root/usr/share/gems/gems/rbovirt-0.0.29/lib/restclient_ext/request.rb:56:in `rescue in transmit' - /opt/rh/ruby193/root/usr/share/gems/gems/rbovirt-0.0.29/lib/restclient_ext/request.rb:11:in `transmit' - /opt/rh/ruby193/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/request.rb:64:in `execute' - /opt/rh/ruby193/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/request.rb:33:in `execute' - /opt/rh/ruby193/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/resource.rb:67:in `post' - /opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.1.0/lib/runcible/base.rb:96:in `get_response' - /opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.1.0/lib/runcible/base.rb:87:in `call' - /opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.1.0/lib/runcible/resources/repository.rb:161:in `unit_search' - /opt/rh/ruby193/root/usr/share/gems/gems/katello-2.0.0/app/models/katello/glue/pulp/repo.rb:664:in `package_lists_for_publish' - /opt/rh/ruby193/root/usr/share/gems/gems/katello-2.0.0/app/lib/actions/pulp/repository/purge_empty_errata.rb:25:in `invoke_external_task' - /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/action/polling.rb:70:in </pre> We need to optimize Repository#package_lists_for_publish to fetch the packages in a paged manner
Created from redmine issue http://projects.theforeman.org/issues/8563
Hi, please provide verification steps thanks Thanks and Regards, Tazim
There really isn't any reproducer steps, just sync very large repos, such as the RHEL Server x86_64 6Server repo
Oops, I meant: 1) Sync a very large repo 2) add it to a content view 3) Publish the content view
VERIFIED: # rpm -qa | grep foreman foreman-proxy-1.7.2.5-1.el7sat.noarch ibm-hs22-05.rhts.eng.brq.redhat.com-foreman-client-1.0-1.noarch ibm-hs22-05.rhts.eng.brq.redhat.com-foreman-proxy-1.0-2.noarch foreman-gce-1.7.2.30-1.el7sat.noarch rubygem-hammer_cli_foreman_discovery-0.0.1.10-1.el7sat.noarch foreman-debug-1.7.2.30-1.el7sat.noarch foreman-postgresql-1.7.2.30-1.el7sat.noarch ruby193-rubygem-foreman_discovery-2.0.0.17-1.el7sat.noarch ruby193-rubygem-foreman_bootdisk-4.0.2.13-1.el7sat.noarch foreman-1.7.2.30-1.el7sat.noarch ruby193-rubygem-foreman_docker-1.2.0.18-1.el7sat.noarch ruby193-rubygem-foreman-redhat_access-0.2.0-8.el7sat.noarch rubygem-hammer_cli_foreman_tasks-0.0.3.4-1.el7sat.noarch rubygem-hammer_cli_foreman_docker-0.0.3.9-1.el7sat.noarch ibm-hs22-05.rhts.eng.brq.redhat.com-foreman-proxy-client-1.0-1.noarch foreman-compute-1.7.2.30-1.el7sat.noarch foreman-vmware-1.7.2.30-1.el7sat.noarch ruby193-rubygem-foreman_hooks-0.3.7-2.el7sat.noarch rubygem-hammer_cli_foreman_bootdisk-0.1.2.7-1.el7sat.noarch foreman-libvirt-1.7.2.30-1.el7sat.noarch ruby193-rubygem-foreman_gutterball-0.0.1.9-1.el7sat.noarch ruby193-rubygem-foreman-tasks-0.6.12.8-1.el7sat.noarch puppet-foreman_scap_client-0.3.3-9.el7sat.noarch foreman-selinux-1.7.2.13-1.el7sat.noarch foreman-ovirt-1.7.2.30-1.el7sat.noarch rubygem-hammer_cli_foreman-0.1.4.14-1.el7sat.noarch steps: 1) Sync a very large repo (ex RHEL Server x86_64 6Server repo) 2) add it to a content view 3) Publish the content view
This bug was fixed in Satellite 6.1.1 which was delivered on 12 August, 2015.
Note that this was seen again after this change and a small issue was discovered that could affect performance. Reopening.
An upstream user saw this again and tested the patch with great success.
VERIFIED: # rpm -qa | grep foreman ruby193-rubygem-foreman_hooks-0.3.7-2.el7sat.noarch rubygem-hammer_cli_foreman-0.1.4.14-1.el7sat.noarch ruby193-rubygem-foreman_bootdisk-4.0.2.13-1.el7sat.noarch foreman-vmware-1.7.2.40-1.el7sat.noarch rubygem-hammer_cli_foreman_bootdisk-0.1.2.7-1.el7sat.noarch foreman-debug-1.7.2.40-1.el7sat.noarch foreman-proxy-1.7.2.6-1.el7sat.noarch foreman-1.7.2.40-1.el7sat.noarch foreman-gce-1.7.2.40-1.el7sat.noarch ruby193-rubygem-foreman-redhat_access-0.2.3-1.el7sat.noarch rubygem-hammer_cli_foreman_discovery-0.0.1.10-1.el7sat.noarch hp-xw6600-02.rhts.eng.bos.redhat.com-foreman-client-1.0-1.noarch hp-xw6600-02.rhts.eng.bos.redhat.com-foreman-proxy-1.0-2.noarch foreman-compute-1.7.2.40-1.el7sat.noarch ruby193-rubygem-foreman_docker-1.2.0.22-1.el7sat.noarch ruby193-rubygem-foreman-tasks-0.6.15.7-1.el7sat.noarch rubygem-hammer_cli_foreman_tasks-0.0.3.5-1.el7sat.noarch foreman-libvirt-1.7.2.40-1.el7sat.noarch foreman-postgresql-1.7.2.40-1.el7sat.noarch hp-xw6600-02.rhts.eng.bos.redhat.com-foreman-proxy-client-1.0-1.noarch foreman-ovirt-1.7.2.40-1.el7sat.noarch foreman-selinux-1.7.2.13-1.el7sat.noarch ruby193-rubygem-foreman_discovery-2.0.0.19-1.el7sat.noarch rubygem-hammer_cli_foreman_docker-0.0.3.9-1.el7sat.noarch ruby193-rubygem-foreman_gutterball-0.0.1.9-1.el7sat.noarch steps: 1) Sync a very large repo (ex Red Hat Enterprise Linux 6 Server RPMs x86_64 6.7) 2) add it to a content view 3) Publish the content view
Created attachment 1080218 [details] content_view_large_repo
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2015:1911