Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1438962

Summary: DB query cache auto-eviction could be longer then the query run time in a large environment
Product: [oVirt] ovirt-engine Reporter: Scott Dickerson <sdickers>
Component: Frontend.WebAdminAssignee: Scott Dickerson <sdickers>
Status: CLOSED CURRENTRELEASE QA Contact: Pavel Novotny <pnovotny>
Severity: medium Docs Contact:
Priority: medium    
Version: 4.1.0CC: bugs, gshereme, lveyde, oourfali
Target Milestone: ovirt-4.1.2Keywords: CodeChange
Target Release: 4.1.2Flags: rule-engine: ovirt-4.1?
pnovotny: planning_ack?
sdickers: devel_ack+
pnovotny: testing_ack+
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-05-23 08:21:34 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: UX RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Scott Dickerson 2017-04-04 21:52:35 UTC 
DashboardDataServlet maintains a DB query cache of the data it servers to the frontend UI plugin.  In a sufficiently large environment, it is possible that the auto-eviction time will be shorter then the run time of the query.

Currently the query cache is updated at a fixed rate, regardless of how long the query takes to run.  Also, if the cache is empty at the time of a data request, the servlet will start a new query to return non-cached results.

If the dashboard DWH queries take longer then 5 minutes to run, there could be problems.  Every 5 minutes the background updater will start a new query, even though the previous hasn't finished yet.  Between the 5 minute mark and the time when the initial query completes, a user may request the data and cause a 3rd query to start.  Now 3 queries will be running.  

The user who caused the 3rd query to be spawned will have to wait for that query to complete before getting their data.  The 1st query will be pushed into the cache, but only for 5 minutes.  The 2nd query will complete and it will be pushed into the cache for 5 minutes.  All the while another query is kicked off every 5 minutes by the background cache update.

The easiest way to fix this problem is 2 fold.  First, change the background cache update to wait a certain amount of time AFTER the task is completed instead of starting a task at a regular interval.  Second, change the cache itself so it doesn't auto-evict the data.  There will always be data in the cache and the background updater will never have multiple update queries running at the same time.
Comment 1 Yaniv Kaul 2017-04-05 06:43:27 UTC 
If the query take more than 5 minutes, we have a very very serious bug!
Nevertheless, I agree it should fire the next event 5 minutes after the last cache was refreshed, not every 5 minutes.
Comment 2 Greg Sheremeta 2017-05-05 15:58:05 UTC 
[QE: not really anything to test here. It's a code change / refactor. As long as dashboard loads and successfully does its usual partial refresh after 5 minutes, it's good.]
Comment 3 Pavel Novotny 2017-05-10 14:59:25 UTC 
Based on Greg's comment, adding CodeChange keyword.

Dashboard sanity passed, no regressions found, moving to verified:
  rhevm-4.1.2.1-0.1.el7.noarch
  ovirt-engine-dashboard-1.1.2-1.el7ev.noarch