Bug 1793875

Summary:	Job Invocations page is extremely slow for non-admin users
Product:	Red Hat Satellite	Reporter:	Ranjan Kumar <rankumar>
Component:	Remote Execution	Assignee:	Leos Stejskal <lstejska>
Status:	CLOSED ERRATA	QA Contact:	Peter Ondrejka <pondrejk>
Severity:	medium	Docs Contact:
Priority:	unspecified
Version:	6.6.0	CC:	ahumbe, aruzicka, dhjoshi, inecas, lstejska, mkalyat, pdwyer
Target Milestone:	6.9.0	Keywords:	Performance, PrioBumpGSS
Target Release:	Unused
Hardware:	All
OS:	All
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:
Clones:	1943262 1943264 (view as bug list)		Environment:
Last Closed:	2021-04-21 13:12:25 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Ranjan Kumar 2020-01-22 07:27:06 UTC

Description of problem:  Job invocations on satellite as non-admin is extremely slow approx ~8 min and also takes excessive RAM.

Affected Url Endpoint: /job_invocations/[id]

Foreman Bug: https://projects.theforeman.org/issues/28723

Version-Release number of selected component (if applicable): satellite-6.6.1

Not Reproducible on local setup

Steps to reproduce: Create a user, assign viewer role and visit https://satellite.example.com/job_invocations/<id>

Observation:

1. As per log _host_actions_td.html.erb partial take a long time to load

2. After  comment of  <%= render 'host_actions_td', options %> in _host_actions_td page loads normal

3. Hammer works normally for job invocation 

Here are production logs

Production logs:  JobInvocationsController#show takes approx 8 min to complete

2020-01-21T08:41:39 [I|app|d489d82d] Processing by JobInvocationsController#show as HTML
2020-01-21T08:49:54 [I|app|d489d82d] Completed 200 OK in 494394ms (Views: 491473.6ms | ActiveRecord: 2848.3ms)
2020-01-21T08:49:54 [I|app|e71a6bdd] Processing by JobInvocationsController#show as HTML
2020-01-21T08:59:01 [I|app|e71a6bdd] Completed 200 OK in 547684ms (Views: 544908.5ms | ActiveRecord: 2747.8ms)

JobInvocationsController#show  logs 

2020-01-21T08:49:54 [I|app|e71a6bdd] Started GET "/job_invocations/138" for 10.104.12.65 at 2020-01-21 08:49:54 +0100
2020-01-21T08:49:54 [I|app|e71a6bdd] Processing by JobInvocationsController#show as HTML
2020-01-21T08:49:54 [I|app|e71a6bdd]   Parameters: {"id"=>"138"}
2020-01-21T08:49:54 [I|app|e71a6bdd] Current user set to test-viewer (regular)
2020-01-21T08:49:54 [I|app|e71a6bdd]   Rendering /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_remote_execution-1.8.0.4/app/views/job_invocations/show.html.erb within layouts/application
2020-01-21T08:49:54 [I|app|e71a6bdd]   Rendered /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_remote_execution-1.8.0.4/app/views/job_invocations/_card_results.html.erb (0.4ms)
2020-01-21T08:49:54 [I|app|e71a6bdd]   Rendered /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_remote_execution-1.8.0.4/app/views/job_invocations/_card_schedule.html.erb (0.8ms)
2020-01-21T08:49:54 [I|app|e71a6bdd]   Rendered /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_remote_execution-1.8.0.4/app/views/job_invocations/_card_target_hosts.html.erb (1.6ms)
2020-01-21T08:57:13 [I|app|e71a6bdd]   Rendered /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_remote_execution-1.8.0.4/app/views/job_invocations/_host_status_td.html.erb (0.3ms)
2020-01-21T08:57:40 [I|app|e71a6bdd]   Rendered /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_remote_execution-1.8.0.4/app/views/job_invocations/_host_actions_td.html.erb (26836.3ms)
2020-01-21T08:57:40 [I|app|e71a6bdd]   Rendered /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_remote_execution-1.8.0.4/app/views/job_invocations/_host_name_td.html.erb (6.2ms)
<Record truncated ....>
2020-01-21T08:58:34 [I|app|e71a6bdd]   Rendered /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_remote_execution-1.8.0.4/app/views/job_invocations/_host_actions_td.html.erb (26712.1ms)
2020-01-21T08:58:34 [I|app|e71a6bdd]   Rendered /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_remote_execution-1.8.0.4/app/views/job_invocations/_host_name_td.html.erb (0.6ms)
2020-01-21T08:58:34 [I|app|e71a6bdd]   Rendered /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_remote_execution-1.8.0.4/app/views/job_invocations/_host_status_td.html.erb (0.3ms)
2020-01-21T08:59:01 [I|app|e71a6bdd]   Rendered /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_remote_execution-1.8.0.4/app/views/job_invocations/_host_actions_td.html.erb (26910.0ms)
2020-01-21T08:59:01 [I|app|e71a6bdd]   Rendered common/_pagination.html.erb (5.4ms)
2020-01-21T08:59:01 [I|app|e71a6bdd]   Rendered /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_remote_execution-1.8.0.4/app/views/job_invocations/_tab_hosts.html.erb (547109.3ms)
2020-01-21T08:59:01 [I|app|e71a6bdd]   Rendered /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_remote_execution-1.8.0.4/app/views/job_invocations/_tab_overview.html.erb (547140.9ms)
2020-01-21T08:59:01 [I|app|e71a6bdd]   Rendered /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_remote_execution-1.8.0.4/app/views/job_invocations/_user_input.html.erb (6.6ms)
2020-01-21T08:59:01 [I|app|e71a6bdd]   Rendered /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_remote_execution-1.8.0.4/app/views/job_invocations/_user_input.html.erb (8.2ms)
<Record Truncate ...>
2020-01-21T08:59:01 [I|app|e71a6bdd]   Rendered /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_remote_execution-1.8.0.4/app/views/job_invocations/_user_input.html.erb (15.1ms)
2020-01-21T08:59:01 [I|app|e71a6bdd]   Rendered /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_remote_execution-1.8.0.4/app/views/job_invocations/_tab_preview_templates.html.erb (447.2ms)
2020-01-21T08:59:01 [I|app|e71a6bdd]   Rendered /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_remote_execution-1.8.0.4/app/views/job_invocations/show.html.erb within layouts/application (547612.1ms)
2020-01-21T08:59:01 [I|app|e71a6bdd]   Rendered layouts/_application_content.html.erb (0.7ms)
2020-01-21T08:59:01 [I|app|e71a6bdd]   Rendering layouts/base.html.erb
2020-01-21T08:59:01 [I|app|e71a6bdd]   Rendered /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_theme_satellite-4.0.1.7/app/views/foreman_theme_satellite/_theme_client_side_branding.js.erb (0.5ms)
2020-01-21T08:59:01 [I|app|e71a6bdd]   Rendered layouts/base.html.erb (33.6ms)
2020-01-21T08:59:01 [I|app|e71a6bdd] Completed 200 OK in 547684ms (Views: 544908.5ms | ActiveRecord: 2747.8ms)

Comment 4 Bryan Kearney 2020-06-17 16:02:58 UTC

Moving this bug to POST for triage into Satellite since the upstream issue https://projects.theforeman.org/issues/28723 has been resolved.

Comment 5 Brad Buckingham 2020-11-13 19:57:42 UTC

Fix is in Satellite 6.9 SNAP 1 with tfm-rubygem-foreman_remote_execution-4.1.0-1.el7sat.noarch

Comment 6 Peter Ondrejka 2020-11-20 13:45:31 UTC

Verified in Satellite 6.9 snap 2, did not experience a significant slowdown in the aforementioned scenario.

Comment 9 errata-xmlrpc 2021-04-21 13:12:25 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: Satellite 6.9 Release), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:1313