Bug 680849 - The CPU occupancy rate is around 100% on rhev-h 5.6-9
Summary: The CPU occupancy rate is around 100% on rhev-h 5.6-9
Keywords:
Status: CLOSED NEXTRELEASE
Alias: None
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: vdsm22
Version: 5.6
Hardware: Unspecified
OS: Unspecified
urgent
urgent
Target Milestone: rc
: ---
Assignee: Dan Kenigsberg
QA Contact: yeylon@redhat.com
URL:
Whiteboard:
Depends On:
Blocks: 681148
TreeView+ depends on / blocked
 
Reported: 2011-02-28 07:10 UTC by Guohua Ouyang
Modified: 2016-04-18 06:38 UTC (History)
15 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2011-03-02 15:51:30 UTC
Target Upstream Version:


Attachments (Terms of Use)
ps.log (133.51 KB, text/plain)
2011-02-28 07:13 UTC, Guohua Ouyang
no flags Details
top.output (85.80 KB, image/png)
2011-02-28 07:17 UTC, Guohua Ouyang
no flags Details

Description Guohua Ouyang 2011-02-28 07:10:42 UTC
Description of problem:
The CPU occupancy rate can reach 100% on a clean new installed build of rhev-h 5.6-9.

#top
Tasks: 410 total,  83 running, 309 sleeping,   0 stopped,  18 zombie
Cpu(s): 97.0%us,  3.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   3848572k total,  1549424k used,  2299148k free,    56696k buffers
Swap:     8184k total,        0k used,     8184k free,   462944k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND           
 6813 vdsm      16  -5 85808 6172 2800 R 31.6  0.2   0:06.15 python             
 6926 vdsm      16  -5 85808 6180 2800 R 31.6  0.2   0:07.15 python             
 8312 vdsm      17  -5 85808 6172 2800 R 31.6  0.2   0:01.04 python             
 7036 vdsm      20  -5 85812 6180 2800 R 16.1  0.2   0:07.79 python             
 7984 vdsm      20  -5 85812 6176 2800 R 16.1  0.2   0:07.15 python             
 8397 vdsm      15  -5 85812 6180 2800 R 16.1  0.2   0:00.55 python             
 6575 vdsm      20  -5 85812 6184 2800 R 15.8  0.2   0:09.54 python             
 6600 vdsm      20  -5 85812 6180 2800 R 15.8  0.2   0:04.65 python             
 6645 vdsm      20  -5 85816 6184 2800 R 15.8  0.2   0:10.07 python             
 6672 vdsm      20  -5 85816 6176 2800 R 15.8  0.2   0:04.58 python             
 6724 vdsm      20  -5 85816 6176 2800 R 15.8  0.2   0:06.15 python             
 6770 vdsm      20  -5 85816 6188 2800 R 15.8  0.2   0:05.58 python             
 6822 vdsm      18  -5 85808 6176 2800 R 15.8  0.2   0:01.54 python             
 6833 vdsm      18  -5 85780 6172 2800 R 15.8  0.2   0:06.65 python             
 6944 vdsm      20  -5 85808 6180 2800 R 15.8  0.2   0:05.04 python             
 7009 vdsm      20  -5 85812 6180 2800 R 15.8  0.2   0:03.66 python             
 8388 vdsm      20  -5 85812 6176 2800 R 15.8  0.2   0:02.04 python 

#free
total      used       free     shared    buffers     cached       
Mem:       3848572    1705556    2143016          0      56748     463080       
-/+ buffers/cache:    1185728    2662844                                        
Swap:         8184          0       8184


Version-Release number of selected component (if applicable):
rhev-hypervisor-5.6-9

vdsm version:
vdsm22-4.5-63.20.el5_6

How reproducible:
Always.

Steps to Reproduce:
1. Install RHEVH via PXE/USB/CDROM.
2. Login by root, check CPU status by command "top".

Actual results:
The cpu occupancy rate is about 100%.

Expected results:
The cpu occupancy rate should lower than 5%.

Additional info: 
Attaching a log from 'ps'. Seems a plenty of 'storage/remoteFileHandler.pyc' with strange paths were running at the same time.

Comment 1 Guohua Ouyang 2011-02-28 07:13:03 UTC
Created attachment 481316 [details]
ps.log

Comment 2 Guohua Ouyang 2011-02-28 07:17:27 UTC
Created attachment 481321 [details]
top.output

Comment 3 Dan Kenigsberg 2011-02-28 20:35:04 UTC
The problem here is that when vdsm imports storage.misc it checks if it is running on rhev-h by firing the new out-of-process mechanism, which in its turn imports misc again. It's all turtles from their to the abyss.

There was no need for such a convoluted check in the first place. And there was certainly no need to do it out-of-process.

The following patch takes ovirtness check from RHEL6's vdsm and avoids the curse of endless recursion:

http://post-office.corp.redhat.com/archives/rhev-patches/2011-February/msg01944.html

Comment 5 Dan Kenigsberg 2011-03-02 15:51:30 UTC
grrr. closing this annoying clone.


Note You need to log in before you can comment on or make changes to this bug.