Bug 1064907

Summary: Listing templates takes noticeable amount of time, while listing many more VMs is prompt
Product: Red Hat Enterprise Virtualization Manager Reporter: Tomas Dosek <tdosek>
Component: ovirt-engineAssignee: Omer Frenkel <ofrenkel>
Status: CLOSED ERRATA QA Contact: Ilanit Stein <istein>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 3.3.0CC: acathrow, derez, ecohen, iheim, lpeer, mavital, michal.skrivanek, mkalinin, ofrenkel, sherold, s.kieske, tjelinek, tpoitras, yeylon
Target Milestone: ---Keywords: Reopened, UserExperience, ZStream
Target Release: 3.4.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: virt
Fixed In Version: is35 Doc Type: Bug Fix
Doc Text:
Previously, listing templates took up to 30 seconds. Now, the issues has been corrected and listing templates takes no more than a few seconds.
 Story Points: ---
Clone Of:
: 1070265 (view as bug list) Environment:
Last Closed: 2014-06-09 15:04:14 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1024889, 1070265, 1078909, 1142926    

Description Tomas Dosek 2014-02-13 14:21:21 UTC 
Description of problem:
Customer is reproducing an issue where listing 160 Templates takes around 30 seconds while listing 1000 VMs is very prompt (1 sec).

Version-Release number of selected component (if applicable):
3.3.0 - GA (is32.2)

How reproducible:
100 %

Steps to Reproduce:
1. Have 1000 VMs owner by various users
2. Have 160 templates owned by various users
3. Try switching between the VMs and Template tabs

Actual results:
For 160 Templates listing takes around 30
while listing 1000 VMs is very prompt (1 sec)

Expected results:
Template listing should not be any slower than listing VMs

Additional info:
We tried to perform the same with API and the result was pretty much the same.
We tried to analyze the DB query used for particular listings of VMs and templates. Both returned result within 1.5 ms time frame so the database is not a setback here.

No error logs, nothing helpful was found.
Comment 2 Tomas Dosek 2014-02-14 07:56:58 UTC 
Getdisks by VM id:

-[ RECORD 1 ]-------+---------------------------------------------------------------------------------------------------------------------------------------
Schema              | public
Name                | getdisksvmguid
Result data type    | SETOF all_disks_including_snapshots
Argument data types | v_vm_guid uuid, v_only_plugged boolean, v_user_id uuid, v_is_filtered boolean
Type                | normal
Volatility          | stable
Owner               | engine
Language            | plpgsql
Source code         | 
                    : BEGIN
                    :       RETURN QUERY SELECT all_disks_including_snapshots.*
                    :       FROM all_disks_including_snapshots
                    :       LEFT JOIN vm_device ON vm_device.device_id = all_disks_including_snapshots.image_group_id AND (NOT v_only_plugged OR is_plugged)
                    :       WHERE vm_device.vm_id = v_vm_guid
                    :       AND ((vm_device.snapshot_id IS NULL AND all_disks_including_snapshots.active IS NOT FALSE)
                    :           OR vm_device.snapshot_id = all_disks_including_snapshots.vm_snapshot_id)
                    :       AND (NOT v_is_filtered OR EXISTS (SELECT 1
                    :                                         FROM   user_disk_permissions_view
                    :                                         WHERE  user_id = v_user_id AND entity_id = all_disks_including_snapshots.disk_id));
                    : 
                    : END; 
Description         | 

This doesn't really seem suspicious on the first sight, however we have to bare in mind customer runs this in environment with more than 1k records in the vm_static table and the all_disks_including_snapshots listing might take really noticeable amount of time.

I'll let the customer do the explain analyze on top of their database to see how fast we can get the result.
Comment 3 Tomas Dosek 2014-02-19 09:08:08 UTC 
Created attachment 865004 [details]
Jstack dumps
Comment 4 Omer Frenkel 2014-02-20 13:26:12 UTC 
Tomas, any reason for UI to expect templates contains disks when returned from search query?
this is weird and i think we should remove it.
looking at rest usage also doesnt seem to expect disks returned with the template
Comment 5 Tomas Jelinek 2014-02-20 14:33:14 UTC 
Looking at the code it seems that the result of search query is not expected to contain also the disks.

@Daniel, you know the disks related code better, do you agree?
Comment 6 Daniel Erez 2014-02-20 21:44:01 UTC 
(In reply to Tomas Jelinek from comment #5)
> Looking at the code it seems that the result of search query is not expected
> to contain also the disks.
> 
> @Daniel, you know the disks related code better, do you agree?

It seems indeed safe to remove the disks update from search query.
I don't recall any usage of disks in Templates main tab context.
We probably can just amend SearchQuery -> searchVMTemplates method (should be verified though...).
Comment 8 Michal Skrivanek 2014-02-27 15:42:18 UTC 
identified issues with editing templates, reopening
Comment 9 Ilanit Stein 2014-03-19 12:51:56 UTC 
Verified on rhevm 3.4 av3.1:

Check rhevm 3.3, move from VMs tab (30 VMs), that contain 30 VMs to Templates tab (160 Templates) - It takes several seconds till templates list is displayed.

Check on rhevm 3.4, av3.1, move from VM tab to Templates tab (160 templates) - take around 1 sec.
Comment 10 errata-xmlrpc 2014-06-09 15:04:14 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHSA-2014-0506.html