Bug 1575099

Summary:	Bulk host actions are slow at scale
Product:	Red Hat Satellite	Reporter:	Dylan Gross <dgross>
Component:	Tasks Plugin	Assignee:	satellite6-bugs <satellite6-bugs>
Status:	CLOSED WONTFIX	QA Contact:	Peter Ondrejka <pondrejk>
Severity:	medium	Docs Contact:
Priority:	high
Version:	6.5.0	CC:	ahumbe, aruzicka, bkearney, dgross, dsynk, inecas, ktordeur, mmccune, sokeeffe, tasander, wclark
Target Milestone:	Unspecified	Keywords:	PrioBumpField, PrioBumpGSS, PrioBumpPM, Triaged
Target Release:	Unused
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2021-03-18 16:37:52 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Dylan Gross 2018-05-04 18:44:05 UTC

Description of problem:

  The bulk action to "Manage Repository Sets" is slow at scale

Version-Release number of selected component (if applicable):

  Red Hat Satellite 6.3.1

How reproducible:


Steps to Reproduce:
1.   Select a large number of Content Hosts (WebUI -> "Content" -> "Content Hosts"

2.  "Select Action" -> "Manage Repository Sets"

3.  Select a repo and either "Override to Enabled" or "Override to Disabled"


Actual results:

   Task took days to complete

Expected results:

   Hoping for a much quicker turnaround to set the override of a repo for the whole environment.

Additional info:

Comment 10 Bryan Kearney 2019-11-04 14:34:06 UTC

The Satellite Team is attempting to provide an accurate backlog of bugzilla requests which we feel will be resolved in the next few releases. We do not believe this bugzilla will meet that criteria, and have plans to close it out in 1 month. This is not a reflection on the validity of the request, but a reflection of the many priorities for the product. If you have any concerns about this, feel free to contact Red Hat Technical Support or your account team. If we do not hear from you, we will close this bug out. Thank you.

Comment 11 Mike McCune 2019-11-04 15:04:19 UTC

Going to reword this bug to include all host bulk actions which are generally slow and fail to perform at scale. Will be attaching additional cases to this bug.

Comment 12 Mike McCune 2019-11-04 15:05:00 UTC

Bulk host delete and Bulk host subscription assignment also see similar issues at scale where they take hours to perform action against sub-1000 hosts.

Comment 14 Mike McCune 2021-03-11 18:51:15 UTC

Upon review of our valid but aging backlog the Satellite Team has concluded that this Bugzilla does not meet the criteria for a resolution in the near term, and are planning to close in one month's time. If you have any concerns about this, please contact your Red Hat Account team.  Thank you.

Comment 15 Mike McCune 2021-03-18 16:37:52 UTC

There have been various performance and scale related fixes in Foreman Tasks subsystem's ability to handle bulk operations.

With Satellite 6.8 we are able to process bulk host operations on low-end hardware (32G RAM, 8 core, magnetic disks) a rate of roughly 180 host actions per minute.

This would allow processing of 50,000 host actions in roughly ~4-5 hours which is an improvement from prior performance of 24 hours or more.

With this, we are going to close this out as working within our expectations in the current release. If you are still experiencing extreme times in completing bulk operations against hosts in Satellite, feel free to re-open.

Comment 16 Mike McCune 2021-04-13 17:40:02 UTC

Thank you for your interest in Satellite 6. We have evaluated this request, and while we recognize that it is a valid request, we do not expect this to be implemented in the product in the foreseeable future. This is due to other priorities for the product, and not a reflection on the request itself. We are therefore closing this out as WONTFIX. If you have any concerns about this, please do not reopen. Instead, feel free to contact your Red Hat Account Team. Thank you.