Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1095907

Summary: VDSM is consuming a lot of cpu time even with no active VMs
Product: Red Hat Enterprise Virtualization Manager Reporter: rhev-integ
Component: vdsmAssignee: Nir Soffer <nsoffer>
Status: CLOSED ERRATA QA Contact: Aharon Canan <acanan>
Severity: high Docs Contact:
Priority: high    
Version: 3.3.0CC: aberezin, aburden, amureini, bazulay, eedri, gwatson, iheim, loberman, lpeer, mkalinin, ngupta, nsoffer, obasan, scohen, smizrahi, yeylon
Target Milestone: ---Keywords: ZStream
Target Release: 3.3.3   
Hardware: x86_64   
OS: Linux   
Whiteboard: storage
Fixed In Version: Doc Type: Bug Fix
Doc Text:
* Red Hat Enterprise Virtualization Hypervisor 6.5 added support for memory-based file systems (for the purposes of testing), which included code with O(N^2) complexity. This introduced performance issues, such as high cpu consumption by vdsm, in certain environments that utilized numerous storage domains. Now, the additional support for memory-based file systems has been disabled and does not contribute to performance problems.
Story Points: ---
Clone Of: 1090664 Environment:
Last Closed: 2014-05-27 08:57:39 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Storage RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1090664    
Bug Blocks:    

Comment 1 Nir Soffer 2014-05-09 18:59:02 UTC
Remove upstream patches.

Comment 2 Nir Soffer 2014-05-11 22:24:07 UTC
Marina, until 3.3.4 is released, the attached patch should be used on customers machines with this problem.

Comment 3 Marina Kalinin 2014-05-12 15:04:36 UTC
Nir,
Thank you!

Comment 6 Nir Soffer 2014-05-15 11:44:32 UTC
How to verify this:

Reproducing with old version:
- You probbably cannot reproduce this - only the customer could
  reproduce on his RHEV-H systems.
- Like new version, but cpu usage should be very high
 (we have seen 240% cpu on 4 core machine)

Testing new version:
1. set up a RHEV-H host system with 30 ISCIS storage domains
2. Activate the host and wait until it it become the spm
3. Check cpu usage, should be 5-15% on 8 core system
4. Standard sanity tests as needed (e.g create vm, run it, delete it)

Comment 7 Nir Soffer 2014-05-15 12:00:20 UTC
Hint for trying to reproduce:
Try to add lot of (100?) mounts to the system. The expensive code in vdsm is looking up the mountpoint for a file and getting its filesystem type.

Comment 9 Aharon Canan 2014-05-21 15:22:20 UTC
on 4 core machine, using is36.4 - verified


1. create ISCSI DC, 2 hosts
2. create 30 ISCSI storage domains

cpu - 12.8% idle

Comment 11 errata-xmlrpc 2014-05-27 08:57:39 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2014-0548.html