Bug 1810588

Summary:	Load and cache cluster status proactively to prevent putting extra load on a cluster
Product:	Red Hat Enterprise Linux 8	Reporter:	Tomas Jelinek <tojeline>
Component:	pcs	Assignee:	Tomas Jelinek <tojeline>
Status:	NEW ---	QA Contact:	cluster-qe <cluster-qe>
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	8.2	CC:	cluster-maint, idevat, mlisik, mpospisi, nwahl, omular, sbradley, tojeline
Target Milestone:	rc	Keywords:	Triaged
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:		Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Tomas Jelinek 2020-03-05 14:42:50 UTC

Description of problem:
JS part of the pcsd web UI refreshes cluster status every 30ish seconds. Upon receiving the refresh request, pcsd backend gathers the status by running several cluster tools. If several users are viewing the same cluster or have the web UI opened in multiple tabs, each of them is asking for the cluster status individually and all such requests are handled independently. This can cause a significant load in the cluster even if load related to a single status refresh is reasonable.

The idea is:
* whenever a user starts viewing a cluster in web UI, start a loop for loading its status in the backend
* the loop automatically loads cluster status every 30ish seconds (this may be configurable) and caches it
* when web UI asks for cluster status, it gets response immediately from the cache
* if the loop detects nobody asked for cluster status for a while, it stops to prevent unnecessary load
* we may want to have a way to force loading the status / override the cache

In a similar way, we need to deal with the clusters overview view.

Version-Release number of selected component (if applicable):
pcs-0.10.4-5.el8

How reproducible:
always, easily

Steps to Reproduce:
1. display a cluster in web ui
2. run top on cluster machines, observe refreshing the status periodically causes load
3. display the same cluster in more browsers or tabs
4. run top on cluster machines, observe refreshing the status periodically causes bigger load

Actual results:
each web UI instance loads the cluster status individually causing extra load

Expected results:
the cluster status is refreshed once in while no matter how many clients are connected keeping the load at reasonable levels