Bug 1691416 - Delays when many clients upload tracer data simultaneously
Summary: Delays when many clients upload tracer data simultaneously
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: katello-tracer
Version: 6.3.5
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: 6.8.0
Assignee: Jonathon Turel
QA Contact: Imaan
URL:
Whiteboard:
Depends On: 1869812
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-03-21 14:54 UTC by Brian Smith
Modified: 2023-12-15 16:24 UTC (History)
4 users (show)

Fixed In Version: tfm-rubygem-katello-3.16.0-0
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-10-27 12:58:15 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Foreman Issue Tracker 29397 0 Normal Closed Delays when many clients upload tracer data simultaneously 2021-02-05 14:42:15 UTC
Red Hat Issue Tracker SAT-20029 0 None None None 2023-09-13 11:24:10 UTC
Red Hat Knowledge Base (Solution) 7033201 0 None None None 2023-09-13 11:23:40 UTC
Red Hat Product Errata RHSA-2020:4366 0 None None None 2020-10-27 12:58:34 UTC

Description Brian Smith 2019-03-21 14:54:31 UTC
Description of problem:
When customer patches around 60 servers simultaneously from Satellite, many of them get to the "Uploading Tracer Profile" step around the same time, and some clients have delays of over 15 minutes.  It is also possible to reproduce this by running a remote execution "katello-tracer-upload" command on multiple hosts.  


Version-Release number of selected component (if applicable):  Satellite 6.3.5 


How reproducible:  Consistently


Steps to Reproduce:
1.  Have around 160 Satellite clients with katello-host-tools-tracer installed.
2.  Create a Remote Execution job to run on the 160 clients and run the command "time katello-tracer-upload", with a 15 minute time out.


Actual results:  In customers environment, 3 of the servers timed out after 15 minutes, and 2 servers ran for over 5 minutes.  


Expected results:  Servers not taking over 15 minutes to run katello-host-tools-tracer


Additional info:
Customer ran the same Remote execution job, but set the concurrency level to 1, so that it only ran on one server at a time, and did not have any failures or delays.   

See also:  https://community.theforeman.org/t/katello-tracer-troubleshoot/8676

Comment 3 Jonathon Turel 2020-03-24 01:17:34 UTC
Created redmine issue https://projects.theforeman.org/issues/29397 from this bug

Comment 4 Bryan Kearney 2020-03-24 02:01:41 UTC
Upstream bug assigned to jturel

Comment 5 Bryan Kearney 2020-03-24 02:01:43 UTC
Upstream bug assigned to jturel

Comment 6 Bryan Kearney 2020-03-26 16:02:19 UTC
Moving this bug to POST for triage into Satellite 6 since the upstream issue https://projects.theforeman.org/issues/29397 has been resolved.

Comment 14 errata-xmlrpc 2020-10-27 12:58:15 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: Satellite 6.8 release), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:4366


Note You need to log in before you can comment on or make changes to this bug.