Bugzilla will be upgraded to version 5.0. The upgrade date is tentatively scheduled for 2 December 2018, pending final testing and feedback.
Bug 971237 - RHEVM slow due to stored procedure getdisksvmguid() consuming most CPU
RHEVM slow due to stored procedure getdisksvmguid() consuming most CPU
Status: CLOSED ERRATA
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-engine (Show other bugs)
3.1.3
Unspecified Unspecified
urgent Severity high
: ---
: 3.3.0
Assigned To: Eli Mesika
Barak Dagan
infra
: ZStream
Depends On:
Blocks: 998523
  Show dependency treegraph
 
Reported: 2013-06-06 01:52 EDT by Mark Huth
Modified: 2016-02-10 14:06 EST (History)
21 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Previously, the getdisksvmguid() procedure would consume a large quantity of available CPU resources due to the inclusion of a large volume of irrelevant data in database queries, causing those queries to be slow. With this update, queries now only retrieve relevant data, increasing the speed of queries.
Story Points: ---
Clone Of:
: 986700 998523 (view as bug list)
Environment:
Last Closed: 2014-01-21 12:26:09 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: Infra
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
oVirt gerrit 16657 None None None Never
oVirt gerrit 18268 None None None Never
Red Hat Product Errata RHSA-2014:0038 normal SHIPPED_LIVE Important: Red Hat Enterprise Virtualization Manager 3.3.0 update 2014-01-21 17:03:06 EST

  None (edit)
Comment 2 Kevein Liu 2013-06-12 22:01:07 EDT
Hi team,

Any update here? 
Please contact me if you need any assistance.
Thank you,

Kevein
Comment 10 Liran Zelkha 2013-06-25 06:16:52 EDT
Any chance you can supply a stack trace of the engine server when this happens?
Comment 11 Kevein Liu 2013-06-25 07:30:41 EDT
(In reply to Liran Zelkha from comment #10)
> Any chance you can supply a stack trace of the engine server when this
> happens?

Sure, I can do it. But could you please tell me how to do it?
Can I just use this command "echo t > /proc/sysrq-trigger" ?

Regards,
Kevein
Comment 12 Liran Zelkha 2013-06-25 08:52:42 EDT
Hi Kevein,

You should use the jstack utility. It should be under JDK_HOME/bin/jstack. 
jstack -l -F engine_pid >jstack.txt

It's best if you can do it 2-3 times, so I can compare the results and see if we have some stuck threads.
Comment 14 Kevein Liu 2013-06-25 21:12:31 EDT
(In reply to Liran Zelkha from comment #10)
> Any chance you can supply a stack trace of the engine server when this
> happens?

Hi,
Could you please check comment #13? The high CPU usage was not caused by java process, but `postmaster`. And we can see the main tasks are "getdisksvmguid()".

Cheers,
Kevein
Comment 17 Barak 2013-07-11 06:21:12 EDT
The problem occurs on UP basic view,
each 5 sec (when in focus) the browser retrieves all the information about the VM in focus.
I don't see any reason to refresh the information displayed on the right side of the screen(specifically the disks, but it can be applies also on ("Operating system, defined memory and number of cores). as it rarely changes.

I would say that this information should be retrieved only when changing focus of VM (and of course at login time ... which that has a default focused VM), and all the rest of the refreshes should bring the minimal information required to update the VM status.
Comment 18 Barak 2013-07-11 06:23:10 EDT
(In reply to Barak from comment #17)
> The problem occurs on UP basic view,
> each 5 sec (when in focus) the browser retrieves all the information about
> the VM in focus.
> I don't see any reason to refresh the information displayed on the right
> side of the screen(specifically the disks, but it can be applies also on
> ("Operating system, defined memory and number of cores). as it rarely
> changes.
> 
> I would say that this information should be retrieved only when changing
> focus of VM (and of course at login time ... which that has a default
> focused VM), and all the rest of the refreshes should bring the minimal
> information required to update the VM status.

Continuing that thought,
If someone edits the VM and changes this information (rarely changed), than it looks reasonable to simply change focus or even re-login.
Comment 19 Andrew Cathrow 2013-07-11 07:22:11 EDT
> 
> Continuing that thought,
> If someone edits the VM and changes this information (rarely changed), than
> it looks reasonable to simply change focus or even re-login.

or hit refresh.

ACK agree.
Comment 20 Itamar Heim 2013-07-11 07:43:29 EDT
(In reply to Andrew Cathrow from comment #19)
> > 
> > Continuing that thought,
> > If someone edits the VM and changes this information (rarely changed), than
> > it looks reasonable to simply change focus or even re-login.
> 
> or hit refresh.

that's a distinction between manual and auto refresh (which i'm fine with, just not implied by barak's comment).

barak - everything else on the right would be refreshed by refreshing the vm status probably anyway - it all comes from the same table. i would just avoid asking to refresh the disks on auto-refresh.

question: what happens if a new VM is added to the user? since it is via auto-refresh which isn't populating the disks, it will be without the disks until a manual refresh will be hit or focus change (i'm fine with that - just need to remember while implementing)
Comment 21 Barak 2013-07-11 12:56:03 EDT
(In reply to Itamar Heim from comment #20)
> (In reply to Andrew Cathrow from comment #19)
> > > 
> > > Continuing that thought,
> > > If someone edits the VM and changes this information (rarely changed), than
> > > it looks reasonable to simply change focus or even re-login.
> > 
> > or hit refresh.
> 
> that's a distinction between manual and auto refresh (which i'm fine with,
> just not implied by barak's comment).
> 
> barak - everything else on the right would be refreshed by refreshing the vm
> status probably anyway - it all comes from the same table. i would just
> avoid asking to refresh the disks on auto-refresh.
> 
> question: what happens if a new VM is added to the user? since it is via
> auto-refresh which isn't populating the disks, it will be without the disks
> until a manual refresh will be hit or focus change (i'm fine with that -
> just need to remember while implementing)

Adding a new VM will continue behave the same as today (when a VM is added while user logged in to UP) meaning when this VM will get focus it will retrieve info with Disks all the rest will be without the disk info
Comment 22 Itamar Heim 2013-07-11 13:20:50 EDT
just make sure getting the focus isn't happening for the VM after every refresh, only on the first time it appears (if focus happened to be set to it automatically)
Comment 26 Eli Mesika 2013-08-19 06:49:07 EDT
fixed in commit: f00e810
Comment 28 Barak Dagan 2013-09-22 10:59:20 EDT
following https://bugzilla.redhat.com/show_bug.cgi?id=986700#c3, verified using is15

1. create 10 VMs with 3 disks each.
2. add different users with "view VM" permissions to the VMs
3. opened 7 UserPortals browsers with different users
4. check that all disks appear for all vms
5. from webadmin, remove 2 out of 3 disks (for 3 VMs, one after another)
6, return to the UserPortal and check nothing updated.
7, switch between VM's tabs or wait for 10 seconds - check that disks list is updated.
Comment 29 Charlie 2013-11-27 19:14:49 EST
This bug is currently attached to errata RHEA-2013:15231. If this change is not to be documented in the text for this errata please either remove it from the errata, set the requires_doc_text flag to minus (-), or leave a "Doc Text" value of "--no tech note required" if you do not have permission to alter the flag.

Otherwise to aid in the development of relevant and accurate release documentation, please fill out the "Doc Text" field above with these four (4) pieces of information:

* Cause: What actions or circumstances cause this bug to present.
* Consequence: What happens when the bug presents.
* Fix: What was done to fix the bug.
* Result: What now happens when the actions or circumstances above occur. (NB: this is not the same as 'the bug doesn't present anymore')

Once filled out, please set the "Doc Type" field to the appropriate value for the type of change made and submit your edits to the bug.

For further details on the Cause, Consequence, Fix, Result format please refer to:

https://bugzilla.redhat.com/page.cgi?id=fields.html#cf_release_notes 

Thanks in advance.
Comment 30 errata-xmlrpc 2014-01-21 12:26:09 EST
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHSA-2014-0038.html

Note You need to log in before you can comment on or make changes to this bug.