Bug 1095907 - VDSM is consuming a lot of cpu time even with no active VMs
Summary: VDSM is consuming a lot of cpu time even with no active VMs
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: vdsm
Version: 3.3.0
Hardware: x86_64
OS: Linux
high
high
Target Milestone: ---
: 3.3.3
Assignee: Nir Soffer
QA Contact: Aharon Canan
URL:
Whiteboard: storage
Depends On: 1090664
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-05-08 19:31 UTC by rhev-integ
Modified: 2019-04-28 10:52 UTC (History)
16 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
* Red Hat Enterprise Virtualization Hypervisor 6.5 added support for memory-based file systems (for the purposes of testing), which included code with O(N^2) complexity. This introduced performance issues, such as high cpu consumption by vdsm, in certain environments that utilized numerous storage domains. Now, the additional support for memory-based file systems has been disabled and does not contribute to performance problems.
Clone Of: 1090664
Environment:
Last Closed: 2014-05-27 08:57:39 UTC
oVirt Team: Storage
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Knowledge Base (Solution) 752423 0 None None None Never
Red Hat Product Errata RHBA-2014:0548 0 normal SHIPPED_LIVE vdsm 3.3.3 bug fix update 2014-05-27 12:56:53 UTC

Comment 1 Nir Soffer 2014-05-09 18:59:02 UTC 
Remove upstream patches.
Comment 2 Nir Soffer 2014-05-11 22:24:07 UTC 
Marina, until 3.3.4 is released, the attached patch should be used on customers machines with this problem.
Comment 3 Marina Kalinin 2014-05-12 15:04:36 UTC 
Nir,
Thank you!
Comment 6 Nir Soffer 2014-05-15 11:44:32 UTC 
How to verify this:

Reproducing with old version:
- You probbably cannot reproduce this - only the customer could
  reproduce on his RHEV-H systems.
- Like new version, but cpu usage should be very high
 (we have seen 240% cpu on 4 core machine)

Testing new version:
1. set up a RHEV-H host system with 30 ISCIS storage domains
2. Activate the host and wait until it it become the spm
3. Check cpu usage, should be 5-15% on 8 core system
4. Standard sanity tests as needed (e.g create vm, run it, delete it)
Comment 7 Nir Soffer 2014-05-15 12:00:20 UTC 
Hint for trying to reproduce:
Try to add lot of (100?) mounts to the system. The expensive code in vdsm is looking up the mountpoint for a file and getting its filesystem type.
Comment 9 Aharon Canan 2014-05-21 15:22:20 UTC 
on 4 core machine, using is36.4 - verified


1. create ISCSI DC, 2 hosts
2. create 30 ISCSI storage domains

cpu - 12.8% idle
Comment 11 errata-xmlrpc 2014-05-27 08:57:39 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2014-0548.html
Note You need to log in before you can comment on or make changes to this bug.