Note: This bug is displayed in read-only format because
the product is no longer active in Red Hat Bugzilla.
Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Description of problem:
On a scaled environment with thousands of pools, katello:import_subscriptions (called e.g. during an upgrade) does call
Katello::Pool.import_all
that traverses over each and every pool and imports hosts and namely Activation Keys from candlepin. That means, a call like:
/candlepin/owners/RedHat/activation_keys/?include=id&include=pools.pool.id
is repeatedly requested for each and every pool.
If such a request takes longer, overall upgrade time is redundantly increased.
I propose a simple patch to query that once and pass the keys to the proper routine.
Version-Release number of selected component (if applicable):
Sat 6.4.2
How reproducible:
100%
Steps to Reproduce:
1. Have a system with some AKs and more pools in manifest - the more the better.
2. in one terminal:
tail -f /var/log/candlepin/candlepin.log | grep "/activation_keys/?include=id&include=pools.pool.id"
3. in 2nd terminal:
foreman-rake katello:import_subscriptions
4. count number of lines printed in the 1st terminal
Actual results:
4. shows as many logs / queries to candlepin as the number of pools is (plus one, I think).
Expected results:
4. to show just 1 or 2 such queries
Additional info:
Patch:
--- /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.7.0.46/app/models/katello/glue/candlepin/candlepin_object.rb.orig 2019-02-27 15:43:28.390609472 +0100
+++ /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.7.0.46/app/models/katello/glue/candlepin/candlepin_object.rb.new 2019-02-27 15:44:07.067642277 +0100
@@ -37,12 +37,13 @@ module Katello
organizations.each do |org|
candlepin_ids = import_candlepin_ids(org)
+ keys = Resources::Candlepin::ActivationKey.get(nil, "?include=id&include=pools.pool.id", org.label)
objects = self.in_organization(org)
objects.each do |item|
if candlepin_ids.include?(item.cp_id)
item.import_data
- item.import_managed_associations if import_managed_associations && item.respond_to?(:import_managed_associations)
+ item.import_managed_associations(keys) if import_managed_associations && item.respond_to?(:import_managed_associations)
else
item.destroy
end
--- /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.7.0.46/app/models/katello/glue/candlepin/pool.rb.orig 2019-02-27 15:44:38.758669157 +0100
+++ /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.7.0.46/app/models/katello/glue/candlepin/pool.rb.new 2019-02-27 15:50:58.411991079 +0100
@@ -181,17 +181,16 @@ module Katello
Katello::SubscriptionFacetPool.where(:pool_id => self.id, :subscription_facet_id => entries_to_remove).delete_all
end
- def import_managed_associations
+ def import_managed_associations(keys)
import_hosts
- create_activation_key_associations
+ create_activation_key_associations(keys)
end
def hosts
::Host.where(:id => self.subscription_facets.pluck(:host_id))
end
- def create_activation_key_associations
- keys = Resources::Candlepin::ActivationKey.get(nil, "?include=id&include=pools.pool.id", organization.label)
+ def create_activation_key_associations(keys)
activation_key_ids = keys.collect do |key|
key['id'] if key['pools'].present? && key['pools'].any? { |pool| pool['pool'].try(:[], 'id') == cp_id }
end
Re number of such queries after the patch: there should be as many as the number of organizations is (plus one), not "just" 1-2. But still significantly less than # of pools.
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.
For information on the advisory, and where to find the updated
files, follow the link below.
If the solution does not work for you, open a new bug report.
https://access.redhat.com/errata/RHSA-2019:1222
Description of problem: On a scaled environment with thousands of pools, katello:import_subscriptions (called e.g. during an upgrade) does call Katello::Pool.import_all that traverses over each and every pool and imports hosts and namely Activation Keys from candlepin. That means, a call like: /candlepin/owners/RedHat/activation_keys/?include=id&include=pools.pool.id is repeatedly requested for each and every pool. If such a request takes longer, overall upgrade time is redundantly increased. I propose a simple patch to query that once and pass the keys to the proper routine. Version-Release number of selected component (if applicable): Sat 6.4.2 How reproducible: 100% Steps to Reproduce: 1. Have a system with some AKs and more pools in manifest - the more the better. 2. in one terminal: tail -f /var/log/candlepin/candlepin.log | grep "/activation_keys/?include=id&include=pools.pool.id" 3. in 2nd terminal: foreman-rake katello:import_subscriptions 4. count number of lines printed in the 1st terminal Actual results: 4. shows as many logs / queries to candlepin as the number of pools is (plus one, I think). Expected results: 4. to show just 1 or 2 such queries Additional info: Patch: --- /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.7.0.46/app/models/katello/glue/candlepin/candlepin_object.rb.orig 2019-02-27 15:43:28.390609472 +0100 +++ /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.7.0.46/app/models/katello/glue/candlepin/candlepin_object.rb.new 2019-02-27 15:44:07.067642277 +0100 @@ -37,12 +37,13 @@ module Katello organizations.each do |org| candlepin_ids = import_candlepin_ids(org) + keys = Resources::Candlepin::ActivationKey.get(nil, "?include=id&include=pools.pool.id", org.label) objects = self.in_organization(org) objects.each do |item| if candlepin_ids.include?(item.cp_id) item.import_data - item.import_managed_associations if import_managed_associations && item.respond_to?(:import_managed_associations) + item.import_managed_associations(keys) if import_managed_associations && item.respond_to?(:import_managed_associations) else item.destroy end --- /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.7.0.46/app/models/katello/glue/candlepin/pool.rb.orig 2019-02-27 15:44:38.758669157 +0100 +++ /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.7.0.46/app/models/katello/glue/candlepin/pool.rb.new 2019-02-27 15:50:58.411991079 +0100 @@ -181,17 +181,16 @@ module Katello Katello::SubscriptionFacetPool.where(:pool_id => self.id, :subscription_facet_id => entries_to_remove).delete_all end - def import_managed_associations + def import_managed_associations(keys) import_hosts - create_activation_key_associations + create_activation_key_associations(keys) end def hosts ::Host.where(:id => self.subscription_facets.pluck(:host_id)) end - def create_activation_key_associations - keys = Resources::Candlepin::ActivationKey.get(nil, "?include=id&include=pools.pool.id", organization.label) + def create_activation_key_associations(keys) activation_key_ids = keys.collect do |key| key['id'] if key['pools'].present? && key['pools'].any? { |pool| pool['pool'].try(:[], 'id') == cp_id } end