Bug 1217789 - PurgeEmptyErrata and PurgeEmptyPackageGroups can timeout on large repos
Summary: PurgeEmptyErrata and PurgeEmptyPackageGroups can timeout on large repos
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: Content Management
Version: 6.0.4
Hardware: Unspecified
OS: Unspecified
unspecified
medium vote
Target Milestone: Unspecified
Assignee: Justin Sherrill
QA Contact: Tazim Kolhar
URL: http://projects.theforeman.org/issues...
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-05-01 16:49 UTC by Mike McCune
Modified: 2019-08-15 04:32 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-10-15 18:20:03 UTC
Target Upstream Version:


Attachments (Terms of Use)
content_view_large_repo (89.85 KB, image/png)
2015-10-06 12:15 UTC, Tazim Kolhar
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2015:1911 0 normal SHIPPED_LIVE Satellite 6.1.3 bug fix update 2015-10-15 22:19:19 UTC

Description Mike McCune 2015-05-01 16:49:39 UTC
a user has reported seeing timeouts on the mentioned tasks as part of a content view publish


<pre>
RestClient::RequestTimeout
 
Request Timeout
 
---
- /opt/rh/ruby193/root/usr/share/gems/gems/rbovirt-0.0.29/lib/restclient_ext/request.rb:56:in
  `rescue in transmit'
- /opt/rh/ruby193/root/usr/share/gems/gems/rbovirt-0.0.29/lib/restclient_ext/request.rb:11:in
  `transmit'
- /opt/rh/ruby193/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/request.rb:64:in
  `execute'
- /opt/rh/ruby193/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/request.rb:33:in
  `execute'
- /opt/rh/ruby193/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/resource.rb:67:in
  `post'
- /opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.1.0/lib/runcible/base.rb:96:in
  `get_response'
- /opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.1.0/lib/runcible/base.rb:87:in
  `call'
- /opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.1.0/lib/runcible/resources/repository.rb:161:in
  `unit_search'
- /opt/rh/ruby193/root/usr/share/gems/gems/katello-2.0.0/app/models/katello/glue/pulp/repo.rb:664:in
  `package_lists_for_publish'
- /opt/rh/ruby193/root/usr/share/gems/gems/katello-2.0.0/app/lib/actions/pulp/repository/purge_empty_errata.rb:25:in
  `invoke_external_task'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/action/polling.rb:70:in
</pre>

We need to optimize Repository#package_lists_for_publish  to fetch the packages in a paged manner

Comment 1 Mike McCune 2015-05-01 16:49:40 UTC
Created from redmine issue http://projects.theforeman.org/issues/8563

Comment 8 Tazim Kolhar 2015-07-15 11:33:57 UTC
Hi,

   please provide verification steps
   thanks

Thanks and Regards,
Tazim

Comment 9 Justin Sherrill 2015-07-15 12:55:30 UTC
There really isn't any reproducer steps, just sync very large repos, such as the RHEL Server x86_64 6Server repo

Comment 10 Justin Sherrill 2015-07-15 12:55:58 UTC
Oops, I meant:

1) Sync a very large repo
2) add it to a content view
3) Publish the content view

Comment 11 Tazim Kolhar 2015-07-16 07:30:13 UTC
VERIFIED:
# rpm -qa | grep foreman
foreman-proxy-1.7.2.5-1.el7sat.noarch
ibm-hs22-05.rhts.eng.brq.redhat.com-foreman-client-1.0-1.noarch
ibm-hs22-05.rhts.eng.brq.redhat.com-foreman-proxy-1.0-2.noarch
foreman-gce-1.7.2.30-1.el7sat.noarch
rubygem-hammer_cli_foreman_discovery-0.0.1.10-1.el7sat.noarch
foreman-debug-1.7.2.30-1.el7sat.noarch
foreman-postgresql-1.7.2.30-1.el7sat.noarch
ruby193-rubygem-foreman_discovery-2.0.0.17-1.el7sat.noarch
ruby193-rubygem-foreman_bootdisk-4.0.2.13-1.el7sat.noarch
foreman-1.7.2.30-1.el7sat.noarch
ruby193-rubygem-foreman_docker-1.2.0.18-1.el7sat.noarch
ruby193-rubygem-foreman-redhat_access-0.2.0-8.el7sat.noarch
rubygem-hammer_cli_foreman_tasks-0.0.3.4-1.el7sat.noarch
rubygem-hammer_cli_foreman_docker-0.0.3.9-1.el7sat.noarch
ibm-hs22-05.rhts.eng.brq.redhat.com-foreman-proxy-client-1.0-1.noarch
foreman-compute-1.7.2.30-1.el7sat.noarch
foreman-vmware-1.7.2.30-1.el7sat.noarch
ruby193-rubygem-foreman_hooks-0.3.7-2.el7sat.noarch
rubygem-hammer_cli_foreman_bootdisk-0.1.2.7-1.el7sat.noarch
foreman-libvirt-1.7.2.30-1.el7sat.noarch
ruby193-rubygem-foreman_gutterball-0.0.1.9-1.el7sat.noarch
ruby193-rubygem-foreman-tasks-0.6.12.8-1.el7sat.noarch
puppet-foreman_scap_client-0.3.3-9.el7sat.noarch
foreman-selinux-1.7.2.13-1.el7sat.noarch
foreman-ovirt-1.7.2.30-1.el7sat.noarch
rubygem-hammer_cli_foreman-0.1.4.14-1.el7sat.noarch


steps:

1) Sync a very large repo (ex RHEL Server x86_64 6Server repo)
2) add it to a content view
3) Publish the content view

Comment 12 Bryan Kearney 2015-08-12 16:05:32 UTC
This bug was fixed in Satellite 6.1.1 which was delivered on 12 August, 2015.

Comment 13 Justin Sherrill 2015-09-14 12:57:42 UTC
Note that this was seen again after this change and a small issue was discovered that could affect performance.  Reopening.

Comment 14 Justin Sherrill 2015-09-14 12:59:06 UTC
An upstream user saw this again and tested the patch with great success.

Comment 17 Tazim Kolhar 2015-10-06 12:13:25 UTC
VERIFIED:
# rpm -qa | grep foreman
ruby193-rubygem-foreman_hooks-0.3.7-2.el7sat.noarch
rubygem-hammer_cli_foreman-0.1.4.14-1.el7sat.noarch
ruby193-rubygem-foreman_bootdisk-4.0.2.13-1.el7sat.noarch
foreman-vmware-1.7.2.40-1.el7sat.noarch
rubygem-hammer_cli_foreman_bootdisk-0.1.2.7-1.el7sat.noarch
foreman-debug-1.7.2.40-1.el7sat.noarch
foreman-proxy-1.7.2.6-1.el7sat.noarch
foreman-1.7.2.40-1.el7sat.noarch
foreman-gce-1.7.2.40-1.el7sat.noarch
ruby193-rubygem-foreman-redhat_access-0.2.3-1.el7sat.noarch
rubygem-hammer_cli_foreman_discovery-0.0.1.10-1.el7sat.noarch
hp-xw6600-02.rhts.eng.bos.redhat.com-foreman-client-1.0-1.noarch
hp-xw6600-02.rhts.eng.bos.redhat.com-foreman-proxy-1.0-2.noarch
foreman-compute-1.7.2.40-1.el7sat.noarch
ruby193-rubygem-foreman_docker-1.2.0.22-1.el7sat.noarch
ruby193-rubygem-foreman-tasks-0.6.15.7-1.el7sat.noarch
rubygem-hammer_cli_foreman_tasks-0.0.3.5-1.el7sat.noarch
foreman-libvirt-1.7.2.40-1.el7sat.noarch
foreman-postgresql-1.7.2.40-1.el7sat.noarch
hp-xw6600-02.rhts.eng.bos.redhat.com-foreman-proxy-client-1.0-1.noarch
foreman-ovirt-1.7.2.40-1.el7sat.noarch
foreman-selinux-1.7.2.13-1.el7sat.noarch
ruby193-rubygem-foreman_discovery-2.0.0.19-1.el7sat.noarch
rubygem-hammer_cli_foreman_docker-0.0.3.9-1.el7sat.noarch
ruby193-rubygem-foreman_gutterball-0.0.1.9-1.el7sat.noarch

steps:
1) Sync a very large repo (ex Red Hat Enterprise Linux 6 Server RPMs x86_64 6.7)
2) add it to a content view
3) Publish the content view

Comment 18 Tazim Kolhar 2015-10-06 12:15:37 UTC
Created attachment 1080218 [details]
content_view_large_repo

Comment 20 errata-xmlrpc 2015-10-15 18:20:03 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2015:1911


Note You need to log in before you can comment on or make changes to this bug.