Bug 1445623 - Avoid N+1 query in foreman hosts index by declaring proper scope
Summary: Avoid N+1 query in foreman hosts index by declaring proper scope
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: Remote Execution
Version: 6.2.0
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: Unspecified
Assignee: satellite6-bugs
QA Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-04-26 07:16 UTC by Shimon Shtein
Modified: 2022-03-13 14:16 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-11-01 18:57:28 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Foreman Issue Tracker 18842 0 Normal New Avoid N+1 query in foreman hosts index by declaring proper scope 2019-12-18 10:12:58 UTC

Description Shimon Shtein 2017-04-26 07:16:02 UTC
The query following query gets executed N time during host index API call:

<pre>
SELECT "foreman_tasks_tasks".* FROM "foreman_tasks_tasks" INNER JOIN "foreman_tasks_locks" ON "foreman_tasks_locks"."task_id" = "foreman_tasks_tasks"."id" WHERE "foreman_tasks_tasks"."type" IN ('ForemanTasks::Task::DynflowTask') AND "foreman_tasks_tasks"."label" = 'Actions::RemoteExecution::RunHostJob' AND "foreman_tasks_locks"."resource_id" = 51101 AND "foreman_tasks_locks"."resource_type" = 'Host::Managed'
</pre>

From the following callstack:
<pre>
/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/connection_adapters/postgresql_adapter.rb:822:in `block in exec_no_cache'
/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/connection_adapters/abstract_adapter.rb:373:in `block in log'
/opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/connection_adapters/abstract_adapter.rb:367:in `log'
/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/connection_adapters/postgresql_adapter.rb:822:in `exec_no_cache'
/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/connection_adapters/postgresql/database_statements.rb:137:in `exec_query'
/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/connection_adapters/postgresql_adapter.rb:954:in `select'
/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/connection_adapters/abstract/database_statements.rb:24:in `select_all'
/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/connection_adapters/abstract/query_cache.rb:68:in `block in select_all'
/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/connection_adapters/abstract/query_cache.rb:83:in `cache_sql'
/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/connection_adapters/abstract/query_cache.rb:68:in `select_all'
/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/querying.rb:39:in `find_by_sql'
/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/relation.rb:604:in `exec_queries'
/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/relation.rb:486:in `load'
/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/relation.rb:231:in `to_a'
/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/relation.rb:587:in `blank?'
/opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/core_ext/object/blank.rb:24:in `present?'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman_remote_execution-0.3.0.13/app/models/host_status/execution_status.rb:14:in `relevant?'
/usr/share/foreman/app/models/host_status/global.rb:13:in `block in build'
/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/associations/collection_association.rb:71:in `block in select'
/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/associations/collection_association.rb:71:in `select'
/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/associations/collection_association.rb:71:in `each'
/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/associations/collection_association.rb:71:in `select'
/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/associations/collection_proxy.rb:110:in `select'
/usr/share/foreman/app/models/host_status/global.rb:10:in `build'
/usr/share/foreman/app/models/host/managed.rb:883:in `global_status_label'
/usr/share/foreman/app/views/api/v2/hosts/main.json.rabl:6:in `eval_source'
/opt/theforeman/tfm/root/usr/share/gems/gems/rabl-0.11.6/lib/rabl/engine.rb:417:in `instance_eval'
/opt/theforeman/tfm/root/usr/share/gems/gems/rabl-0.11.6/lib/rabl/engine.rb:417:in `eval_source'
/opt/theforeman/tfm/root/usr/share/gems/gems/rabl-0.11.6/lib/rabl/engine.rb:35:in `apply'
/opt/theforeman/tfm/root/usr/share/gems/gems/rabl-0.11.6/lib/rabl/helpers.rb:136:in `object_to_engine'
/opt/theforeman/tfm/root/usr/share/gems/gems/rabl-0.11.6/lib/rabl/partials.rb:15:in `partial_as_engine'
/opt/theforeman/tfm/root/usr/share/gems/gems/rabl-0.11.6/lib/rabl/builder.rb:196:in `extends'
/opt/theforeman/tfm/root/usr/share/gems/gems/rabl-0.11.6/lib/rabl/builder.rb:114:in `block in compile_settings'
/opt/theforeman/tfm/root/usr/share/gems/gems/rabl-0.11.6/lib/rabl/builder.rb:113:in `each'
/opt/theforeman/tfm/root/usr/share/gems/gems/rabl-0.11.6/lib/rabl/builder.rb:113:in `compile_settings'
/opt/theforeman/tfm/root/usr/share/gems/gems/rabl-0.11.6/lib/rabl/builder.rb:33:in `engines'
/opt/theforeman/tfm/root/usr/share/gems/gems/rabl-0.11.6/lib/rabl/multi_builder.rb:45:in `block in map_engines_to_builders'
/opt/theforeman/tfm/root/usr/share/gems/gems/rabl-0.11.6/lib/rabl/multi_builder.rb:44:in `each'
/opt/theforeman/tfm/root/usr/share/gems/gems/rabl-0.11.6/lib/rabl/multi_builder.rb:44:in `map_engines_to_builders'
/opt/theforeman/tfm/root/usr/share/gems/gems/rabl-0.11.6/lib/rabl/multi_builder.rb:22:in `to_a'
/opt/theforeman/tfm/root/usr/share/gems/gems/rabl-0.11.6/lib/rabl/engine.rb:88:in `to_hash'
/opt/theforeman/tfm/root/usr/share/gems/gems/rabl-0.11.6/lib/rabl/engine.rb:101:in `to_dumpable'
/opt/theforeman/tfm/root/usr/share/gems/gems/rabl-0.11.6/lib/rabl/engine.rb:110:in `to_json'
/opt/theforeman/tfm/root/usr/share/gems/gems/rabl-0.11.6/lib/rabl/engine.rb:48:in `block in render'
/opt/theforeman/tfm/root/usr/share/gems/gems/rabl-0.11.6/lib/rabl/engine.rb:382:in `cache_results'
/opt/theforeman/tfm/root/usr/share/gems/gems/rabl-0.11.6/lib/rabl/engine.rb:47:in `render'
/usr/share/foreman/app/views/api/v2/hosts/index.json.rabl:3:in `_54bcd28489908e6a80cc10544a40c553'
/opt/rh/rh-ror41/root/usr/share/gems/gems/actionview-4.1.5/lib/action_view/template.rb:145:in `block in render'
/opt/rh/rh-ror41/root/usr/share/gems/gems/activesupport-4.1.5/lib/active_support/notifications.rb:161:in `instrument'
/opt/rh/rh-ror41/root/usr/share/gems/gems/actionview-4.1.5/lib/action_view/template.rb:339:in `instrument'
/opt/rh/rh-ror41/root/usr/share/gems/gems/actionview-4.1.5/lib/action_view/template.rb:143:in `render'
/opt/theforeman/tfm/root/usr/share/gems/gems/deface-1.0.2/lib/deface/action_view_extensions.rb:41:in `render'
/opt/rh/rh-ror41/root/usr/share/gems/gems/actionview-4.1.5/lib/action_view/renderer/template_renderer.rb:55:in `block (2 levels) in render_template'
</pre>

Comment 1 Shimon Shtein 2017-04-26 07:16:10 UTC
Created from redmine issue http://projects.theforeman.org/issues/18842

Comment 3 Shimon Shtein 2017-04-26 07:37:06 UTC
There is discussion in the upstream about fixing this bug in a more generalized way, it's tracked under http://projects.theforeman.org/issues/18975.

Comment 4 Chris Duryee 2017-06-03 16:53:01 UTC
adding needinfo on myself to get before/after timings when pulling down a large host list with and without http://projects.theforeman.org/issues/18845.

Comment 6 Chris Duryee 2017-10-16 19:21:47 UTC
I tested this with pulling down 100 hosts, the activerecord portion of the request went from 3.5 sec to 1sec.

Comment 8 Bryan Kearney 2018-11-01 14:44:48 UTC
The Satellite Team is attempting to provide an accurate backlog of bugzilla requests which we feel will be resolved in the next few releases. We do not believe this bugzilla will meet that criteria, and have plans to close it out in 1 month. This is  not a reflection on the validity of the request, but a reflection of the many priorities for the product. If you have any concerns about this, feel free to contact Rich Jerrido or Bryan Kearney or your account team. If we do not hear from you, we will close this bug out. Thank you.

Comment 10 Bryan Kearney 2018-11-01 18:57:28 UTC
This was released in katello3.4.1

Comment 13 Bryan Kearney 2018-11-30 14:50:44 UTC
Thank you for your interest in Satellite 6. We have evaluated this request, and while we recognize that it is a valid request, we do not expect this to be implemented in the product in the foreseeable future. This is due to other priorities for the product, and not a reflection on the request itself. We are therefore closing this out as WONTFIX. If you have any concerns about this, please do not reopen. Instead, feel free to contact Rich Jerrido or Bryan Kearney. Thank you.


Note You need to log in before you can comment on or make changes to this bug.