Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1217789

Summary: PurgeEmptyErrata and PurgeEmptyPackageGroups can timeout on large repos
Product: Red Hat Satellite Reporter: Mike McCune <mmccune>
Component: Content ManagementAssignee: Justin Sherrill <jsherril>
Status: CLOSED ERRATA QA Contact: Tazim Kolhar <tkolhar>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 6.0.4CC: bbuckingham, chpeters, cwelton, fdacunha, jsherril, tkolhar
Target Milestone: UnspecifiedKeywords: Reopened, Triaged
Target Release: Unused   
Hardware: Unspecified   
OS: Unspecified   
URL: http://projects.theforeman.org/issues/11774
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-10-15 18:20:03 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
content_view_large_repo none

Description Mike McCune 2015-05-01 16:49:39 UTC
a user has reported seeing timeouts on the mentioned tasks as part of a content view publish


<pre>
RestClient::RequestTimeout
 
Request Timeout
 
---
- /opt/rh/ruby193/root/usr/share/gems/gems/rbovirt-0.0.29/lib/restclient_ext/request.rb:56:in
  `rescue in transmit'
- /opt/rh/ruby193/root/usr/share/gems/gems/rbovirt-0.0.29/lib/restclient_ext/request.rb:11:in
  `transmit'
- /opt/rh/ruby193/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/request.rb:64:in
  `execute'
- /opt/rh/ruby193/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/request.rb:33:in
  `execute'
- /opt/rh/ruby193/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/resource.rb:67:in
  `post'
- /opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.1.0/lib/runcible/base.rb:96:in
  `get_response'
- /opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.1.0/lib/runcible/base.rb:87:in
  `call'
- /opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.1.0/lib/runcible/resources/repository.rb:161:in
  `unit_search'
- /opt/rh/ruby193/root/usr/share/gems/gems/katello-2.0.0/app/models/katello/glue/pulp/repo.rb:664:in
  `package_lists_for_publish'
- /opt/rh/ruby193/root/usr/share/gems/gems/katello-2.0.0/app/lib/actions/pulp/repository/purge_empty_errata.rb:25:in
  `invoke_external_task'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/action/polling.rb:70:in
</pre>

We need to optimize Repository#package_lists_for_publish  to fetch the packages in a paged manner

Comment 1 Mike McCune 2015-05-01 16:49:40 UTC
Created from redmine issue http://projects.theforeman.org/issues/8563

Comment 8 Tazim Kolhar 2015-07-15 11:33:57 UTC
Hi,

   please provide verification steps
   thanks

Thanks and Regards,
Tazim

Comment 9 Justin Sherrill 2015-07-15 12:55:30 UTC
There really isn't any reproducer steps, just sync very large repos, such as the RHEL Server x86_64 6Server repo

Comment 10 Justin Sherrill 2015-07-15 12:55:58 UTC
Oops, I meant:

1) Sync a very large repo
2) add it to a content view
3) Publish the content view

Comment 11 Tazim Kolhar 2015-07-16 07:30:13 UTC
VERIFIED:
# rpm -qa | grep foreman
foreman-proxy-1.7.2.5-1.el7sat.noarch
ibm-hs22-05.rhts.eng.brq.redhat.com-foreman-client-1.0-1.noarch
ibm-hs22-05.rhts.eng.brq.redhat.com-foreman-proxy-1.0-2.noarch
foreman-gce-1.7.2.30-1.el7sat.noarch
rubygem-hammer_cli_foreman_discovery-0.0.1.10-1.el7sat.noarch
foreman-debug-1.7.2.30-1.el7sat.noarch
foreman-postgresql-1.7.2.30-1.el7sat.noarch
ruby193-rubygem-foreman_discovery-2.0.0.17-1.el7sat.noarch
ruby193-rubygem-foreman_bootdisk-4.0.2.13-1.el7sat.noarch
foreman-1.7.2.30-1.el7sat.noarch
ruby193-rubygem-foreman_docker-1.2.0.18-1.el7sat.noarch
ruby193-rubygem-foreman-redhat_access-0.2.0-8.el7sat.noarch
rubygem-hammer_cli_foreman_tasks-0.0.3.4-1.el7sat.noarch
rubygem-hammer_cli_foreman_docker-0.0.3.9-1.el7sat.noarch
ibm-hs22-05.rhts.eng.brq.redhat.com-foreman-proxy-client-1.0-1.noarch
foreman-compute-1.7.2.30-1.el7sat.noarch
foreman-vmware-1.7.2.30-1.el7sat.noarch
ruby193-rubygem-foreman_hooks-0.3.7-2.el7sat.noarch
rubygem-hammer_cli_foreman_bootdisk-0.1.2.7-1.el7sat.noarch
foreman-libvirt-1.7.2.30-1.el7sat.noarch
ruby193-rubygem-foreman_gutterball-0.0.1.9-1.el7sat.noarch
ruby193-rubygem-foreman-tasks-0.6.12.8-1.el7sat.noarch
puppet-foreman_scap_client-0.3.3-9.el7sat.noarch
foreman-selinux-1.7.2.13-1.el7sat.noarch
foreman-ovirt-1.7.2.30-1.el7sat.noarch
rubygem-hammer_cli_foreman-0.1.4.14-1.el7sat.noarch


steps:

1) Sync a very large repo (ex RHEL Server x86_64 6Server repo)
2) add it to a content view
3) Publish the content view

Comment 12 Bryan Kearney 2015-08-12 16:05:32 UTC
This bug was fixed in Satellite 6.1.1 which was delivered on 12 August, 2015.

Comment 13 Justin Sherrill 2015-09-14 12:57:42 UTC
Note that this was seen again after this change and a small issue was discovered that could affect performance.  Reopening.

Comment 14 Justin Sherrill 2015-09-14 12:59:06 UTC
An upstream user saw this again and tested the patch with great success.

Comment 17 Tazim Kolhar 2015-10-06 12:13:25 UTC
VERIFIED:
# rpm -qa | grep foreman
ruby193-rubygem-foreman_hooks-0.3.7-2.el7sat.noarch
rubygem-hammer_cli_foreman-0.1.4.14-1.el7sat.noarch
ruby193-rubygem-foreman_bootdisk-4.0.2.13-1.el7sat.noarch
foreman-vmware-1.7.2.40-1.el7sat.noarch
rubygem-hammer_cli_foreman_bootdisk-0.1.2.7-1.el7sat.noarch
foreman-debug-1.7.2.40-1.el7sat.noarch
foreman-proxy-1.7.2.6-1.el7sat.noarch
foreman-1.7.2.40-1.el7sat.noarch
foreman-gce-1.7.2.40-1.el7sat.noarch
ruby193-rubygem-foreman-redhat_access-0.2.3-1.el7sat.noarch
rubygem-hammer_cli_foreman_discovery-0.0.1.10-1.el7sat.noarch
hp-xw6600-02.rhts.eng.bos.redhat.com-foreman-client-1.0-1.noarch
hp-xw6600-02.rhts.eng.bos.redhat.com-foreman-proxy-1.0-2.noarch
foreman-compute-1.7.2.40-1.el7sat.noarch
ruby193-rubygem-foreman_docker-1.2.0.22-1.el7sat.noarch
ruby193-rubygem-foreman-tasks-0.6.15.7-1.el7sat.noarch
rubygem-hammer_cli_foreman_tasks-0.0.3.5-1.el7sat.noarch
foreman-libvirt-1.7.2.40-1.el7sat.noarch
foreman-postgresql-1.7.2.40-1.el7sat.noarch
hp-xw6600-02.rhts.eng.bos.redhat.com-foreman-proxy-client-1.0-1.noarch
foreman-ovirt-1.7.2.40-1.el7sat.noarch
foreman-selinux-1.7.2.13-1.el7sat.noarch
ruby193-rubygem-foreman_discovery-2.0.0.19-1.el7sat.noarch
rubygem-hammer_cli_foreman_docker-0.0.3.9-1.el7sat.noarch
ruby193-rubygem-foreman_gutterball-0.0.1.9-1.el7sat.noarch

steps:
1) Sync a very large repo (ex Red Hat Enterprise Linux 6 Server RPMs x86_64 6.7)
2) add it to a content view
3) Publish the content view

Comment 18 Tazim Kolhar 2015-10-06 12:15:37 UTC
Created attachment 1080218 [details]
content_view_large_repo

Comment 20 errata-xmlrpc 2015-10-15 18:20:03 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2015:1911