Created attachment 942807 [details]
Webadmin UI tests comparison of 3.4 and 3.5 version
Description of problem:
Webadmin GWT UI in RHEVM 3.5 is significantly slower than in RHEVM 3.4.
Comparison of automated UI tests shows, that the same test scenarios executed in RHEVM 3.5 take about 20-25% more time than in 3.4.
Version-Release number of selected component (if applicable):
Client machine & browser:
Steps to Reproduce:
1. Two different test suites were executed on RHEVM 3.4 and RHEVM 3.5. See attached file webadmin-tests-3.4_3.5-comparison.txt for details of each test suite.
The same actions in RHEVM 3.5 UI took about 20-25% more time than in 3.4 (rhevm-3.4.2-1.1.el6ev):
- test suite #1: 40 cases, clean browser is started for each test case
3.5: 50m23s -> slower by 25%
- test suite #2: 14 test cases, all cases run in one browser instance, in the same Webadmin session
3.5: 09m04s -> slower by 23%
Taking into account new features included in 3.5 UI, probably 5-10% slowdown would be still acceptable/understandable, but not 20% or more.
@Alexander/Pavel - please work together in order to try and figure out the problem.
from what I know, the Login process is much quicker, since we are not loading all of the GUI parts (i.e. main tabs) in advance, but doing more of a "lazy" loading.
this means that whenever we are going into a "new" main tab, there are a couple of seconds in which the main tab is loading.
so I understand that the overall performance within the Selenium tests has regressed, however the question is if the user experience is overall better / different but not worse. if so - I recommend to close the BZ.
So comparing the tests between 3.4 and 3.5 we notice the following things:
1. The initial login/setup in 3.5 is significantly faster than in 3.4. This is expected due to the lazy loading of the main/sub tab presenters.
2. Certain operations in 3.5 are significantly slower in 3.5 than in 3.4, most notably the popup windows take a lot longer in the tests. In the 3.4 tests opening up the window takes a 100-200ms. In 3.5 this takes 10+ seconds. This should not be affected by the lazy loading of the other presenters, so definitely something else going on.
3. There is some slow down in the loading of the actual main tabs, but this is expected, although the slow down is very significant compared to what we would expect.
In short, something changed, but at this point I haven't put my finger on what yet. I will work with Pavel a little more to see if I can determine what.
I am investigating this issue as we speak. At this point in time we know the following:
1. For 3.5 in the tests some time-outs were increased which would account for some of the increase in time. Some of the test check for non existence of elements, and the time-outs need to expire to pass the test.
2. I misinterpreted some of the logs that let me to believe that the pop-ups were slow. After some more digging with Pavel, it seems the increase in time is partially 1 and partially an increase in time on XPATH lookups in the browser, which don't interact with the UI. It basically just looks through the DOM tree to find a particular element that matches the expression. If in 3.5 the tree increased a lot, that would explain some of the extra time as well.
So in short I do not think this is a regression, but there are some loose ends we are still investigating before I want to say that.
Verified in rhevm-3.5.0-0.19.beta.el6ev.noarch (vt9).
So the problem here was in general UI plugin initialization. Alexander's patch seems to solve it because our UI tests are now noticeably faster.
Tests results summary:
Test suite | avg. before | avg. after | speedup [%]
bookmarks | 7:23 | 3:13 | 56
data centers | 3:53 | 2:20 | 40
clusters | 3:30 | 1:46 | 50
sanity1 | 11:34 | 9:46 | 16
sanity2 | 28:45 | 16:28 | 43
I didn't compare test results between 3.4 and 3.5 directly because our 3.5 test suites differ from 3.4 and it would require a lot of manual data normalization and computing.
But - the bug description says that slow down between 3.4 and 3.5 was 20-25%.
The above results give us average UI response speedup by 44%.
So we can say that 3.5 UI should be now ~20% faster than in 3.4.