DashboardDataServlet maintains a DB query cache of the data it servers to the frontend UI plugin. In a sufficiently large environment, it is possible that the auto-eviction time will be shorter then the run time of the query. Currently the query cache is updated at a fixed rate, regardless of how long the query takes to run. Also, if the cache is empty at the time of a data request, the servlet will start a new query to return non-cached results. If the dashboard DWH queries take longer then 5 minutes to run, there could be problems. Every 5 minutes the background updater will start a new query, even though the previous hasn't finished yet. Between the 5 minute mark and the time when the initial query completes, a user may request the data and cause a 3rd query to start. Now 3 queries will be running. The user who caused the 3rd query to be spawned will have to wait for that query to complete before getting their data. The 1st query will be pushed into the cache, but only for 5 minutes. The 2nd query will complete and it will be pushed into the cache for 5 minutes. All the while another query is kicked off every 5 minutes by the background cache update. The easiest way to fix this problem is 2 fold. First, change the background cache update to wait a certain amount of time AFTER the task is completed instead of starting a task at a regular interval. Second, change the cache itself so it doesn't auto-evict the data. There will always be data in the cache and the background updater will never have multiple update queries running at the same time.
If the query take more than 5 minutes, we have a very very serious bug! Nevertheless, I agree it should fire the next event 5 minutes after the last cache was refreshed, not every 5 minutes.
[QE: not really anything to test here. It's a code change / refactor. As long as dashboard loads and successfully does its usual partial refresh after 5 minutes, it's good.]
Based on Greg's comment, adding CodeChange keyword. Dashboard sanity passed, no regressions found, moving to verified: rhevm-4.1.2.1-0.1.el7.noarch ovirt-engine-dashboard-1.1.2-1.el7ev.noarch