Red Hat Bugzilla – Bug 1398546
[z-stream clone - 4.0.6] RHV-M Web UI performance degrades over time
Last modified: 2017-03-03 10:12:53 EST
Description of problem: The web admin portal is slow in IE 11. The same client behaves much better with chrome browser on the same client system. Although the chrome browser behaves better, the performance degradates over the time. Version-Release number of selected component (if applicable): rhevm-3.6.6.2-0.1.el6 How reproducible: 100% with IE11 after a while with other browsers Steps to Reproduce: 1. Start IE11 and connect to the webAmin 2. Work with that for a while Actual results: The memory grows rapidly 1GRAM in 10 min. The average CPU load is ~60% and 99% in peaks. even though the browser is not used. This is most visible in the Virtual Machine tab. Expected results: The usage is fluent. No delay between tab switching. (not counting the data loading) This comment was originaly posted by rhodain@redhat.com
*** Bug 1348150 has been marked as a duplicate of this bug. *** This comment was originaly posted by gshereme@redhat.com
I have good news on our progress. After applying several patches to fix memory leaks (and one more not-yet-merged patch), I can say we are now very stable on 4.1 master. I've done several performance tests to verify. The most important one was an endurance test I did that shows webadmin still very stable after 6.25 hours of heavy use under webdriver. See attachment '6 hour webdriver results'. This shows that we're still leaking some memory over time (graph on right), and while we have more work to do on that, the relatively flat response time line (graph on left) means the app stayed performant over the 6 hour test. [Ignore the spike in the left graph. That was a one-time webdriver hiccup.] This comment was originaly posted by gshereme@redhat.com
Created attachment 1220554 [details] 6 hour webdriver results This comment was originaly posted by gshereme@redhat.com
(In reply to Greg Sheremeta from comment #62) > I have good news on our progress. After applying several patches to fix > memory leaks (and one more not-yet-merged patch), I can say we are now very > stable on 4.1 master. I've done several performance tests to verify. The > most important one was an endurance test I did that shows webadmin still > very stable after 6.25 hours of heavy use under webdriver. This is great news. All memory leak fixes should land in 4.0.6 eventually. Once that happens, we can decide how to proceed regarding 3.6.z. > > See attachment '6 hour webdriver results'. > > This shows that we're still leaking some memory over time (graph on right), > and while we have more work to do on that, the relatively flat response time > line (graph on left) means the app stayed performant over the 6 hour test. > > [Ignore the spike in the left graph. That was a one-time webdriver hiccup.] Given Greg's performance analysis results, I'd conclude this BZ once the "not-yet-merged patch" lands in 4.0.6. Any further improvements should be done as part of tracker bug 1378935. This comment was originaly posted by vszocs@redhat.com
Attaching another test result that shows the dramatic improvements made by Vojtech's awesome tooltips fix. See 'tooltip leak fix results.png' The flat lines are with tooltips fix onboard. The mountainous slope is without it. The degradation was severe, as you can see. That patch alone shaved 40 minutes off of an hour run, so we are now 66% faster. :) This comment was originaly posted by gshereme@redhat.com
Created attachment 1220952 [details] tooltip leak fix results This comment was originaly posted by gshereme@redhat.com
*** Bug 1395911 has been marked as a duplicate of this bug. *** This comment was originaly posted by oourfali@redhat.com
Interim results for ovirt-engine-webadmin-portal-4.0.6.3-0.1.el7ev.noarch: engine evironment: 1 host, 1 nfs storage, 41 VMs (blank, w/o OS), 4 of them running Client machine (VM): Windows 8.1 Ent., 8 GB RAM, dual-core (virtualized from baremetal host has Intel Xeon E5-2420 v2 @ 2.20GHz); IE 11 The manual scenario was to go through all main tabs and in each select the first row to load the sub-tab panel. Then I just opened and closed following dialogs: New VM dialog + New VM Disk, Make Template and New VM Pool dialog. Then I left the browser opened on the VMs tab and monitored the IE RAM consumption. Take #1: time elapsed: IE RAM usage -------------------------- 00m: 190 MB (right after login to webadmin) 10m: 610 MB 45m: 1 653 MB (the VM list gets blank & UI gets completely frozen, page must be reloaded to fix it) See the screenshot atached for details. Take #2: 00m: 187 MB (right after login) 10m: 623 MB 45m: 1 667 MB (blank VM list, fromzen UI, reload needed) So basically after 45 minutes IE gets fried even with a small RHV environment. However, I need to compare these numbers with 4.0.5 to see if it differs (basically if 4.0.6 numbers are an improvement or not).
Created attachment 1232294 [details] IE11 webadmin: VMs list gets blank with high RAM usage
Verified in rhevm-4.0.6.3-0.1.el7ev.noarch ovirt-engine-webadmin-portal-4.0.6.3-0.1.el7ev.noarch Browser: Firefox 45.6 ESR @ RHEL 6.8 I ran a test which (in a loop) went through the main tabs and then opened & closed the New VM dialog. I used 100 passes and for each measured the RAM usage via the browser built-in tool at the about:memory page. Results: 4.0.5: RAM usage started around 100 MB and continuously grew up to ~850 MB. 4.0.6: Started at ~140 MB and stayed around this value the whole time. (chart attached) Timewise, 100 passes took: 4.0.5: 3h55m 4.0.6: 2h6m The improvements in RAM usage and in UI performance are obvious. Please see the RAM usage chart attached.
Created attachment 1234007 [details] RAM usage chart: 4.0.5 vs. 4.0.6
Thanks Pavel for the verification! > Browser: Firefox 45.6 ESR @ RHEL 6.8 > 4.0.6: Started at ~140 MB and stayed around this value the whole time. (chart attached) > Test took 2h6m [100 passes] This proves that memory usage of (mem-leak-patched) WebAdmin in Firefox is generally consistent. > Windows 8.1 Ent., IE 11 > After 45m: 1 653 MB (the VM list gets blank & UI gets completely frozen, page must be reloaded to fix it) This proves that IE 11, given practically the same application [*], still suffers from the inability to deal with complex JavaScript code. [*] WebAdmin GWT permutation for IE 11 vs. Firefox diff == bridging gaps (APIs, standard compliance) between browsers only; actual application logic is the same Any customer using IE 11 should be aware of the above.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHBA-2017-0043.html