Bug 719516

Summary: [vdsm][vdsClient] exeption when executing list table just after or while stopping vms.
Product: Red Hat Enterprise Linux 6 Reporter: David Naori <dnaori>
Component: vdsmAssignee: David Naori <dnaori>
Status: CLOSED ERRATA QA Contact: David Naori <dnaori>
Severity: low Docs Contact:
Priority: unspecified    
Version: 6.1CC: abaron, bazulay, danken, dnaori, iheim, ilvovsky, mgoldboi, ykaul
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Linux   
Whiteboard:
Fixed In Version: vdsm-4.9-81.el6 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2011-12-06 07:30:44 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description David Naori 2011-07-07 06:10:43 UTC
Description of problem:

exeption in vdsClient when shutting-down\stoping\killing vms and using vdsClient 'list table' meanwhile or just afterwards.  

the cause is a race condition between list and getAllVmStats.

 if allStats[id].get('monitorResponse') == '-1':

the id took from list is no longer exist by the time getAllVmStats is called. 


[root@camel-vdsb ~]# vdsClient -s 0 list table
a116dfec-23fd-4676-b9a6-480ce1cdf765   4483  TOEXPORT-01          Up                                       
9302f367-a08a-44e5-a128-e9944745c92c   3400  TOEXPORT-03          Up                                       
cb206c72-7d64-473d-b784-25e63b7fd055   3805  TOEXPORT-06          Up                                       
44fdcd33-db49-4889-a8c9-63c81aa86f97   5251  TOEXPORT-09          Up                                       
802f8f95-3a57-4eaf-bad2-85357614644d   3589  TOEXPORT-05          Up                                       
fbf21a41-2e54-4632-bbc9-34f97ade7fc0   4258  TOEXPORT-07          Up                                       
13e2dc53-69d9-4873-8e39-4a2f55a4ec2b   3239  TOEXPORT-10          Up                                       
c29070a2-22d8-41a0-a273-c0b17856c7c3   4021  TOEXPORT-11          Up                                       
ae71d9e9-2896-4289-ac13-50a62b49139d   4739  TOEXPORT-04          Up   

                                    
[root@camel-vdsb ~]# kill -9 4483 && vdsClient -s 0 list table
^[[ATraceback (most recent call last):
  File "/usr/share/vdsm/vdsClient.py", line 2002, in <module>
    code, message = commands[command][0](commandArgs)
  File "/usr/share/vdsm/vdsClient.py", line 186, in do_list
    for s in self.s.getAllVmStats()['statsList']:
KeyError: 'statsList'


Version-Release number of selected component (if applicable):
vdsm-cli-4.9-80.el6.x86_64

How reproducible:
100%

Steps to Reproduce:
1.
2.
3.
  
Actual results:


Expected results:


Additional info:

Comment 2 David Naori 2011-07-07 07:01:05 UTC
http://gerrit.usersys.redhat.com/#change,671

Comment 5 Dan Kenigsberg 2011-07-17 11:48:01 UTC
Bug report changed to ON_QA status by me since Errata System refused to do so.
A QE request has been submitted for advisory RHEA-2011:11186-01
http://errata.devel.redhat.com/errata/show/11186

Comment 6 David Naori 2011-07-18 07:35:29 UTC
[root@camel-vdsc ~]# vdsClient -s 0 list table
3aedf776-8e7e-422a-a3c9-365d90f32299  29387  EXPORTING-03         Up                                       
044177e2-c475-4853-a2f8-a66602c71a18  29266  EXPORTING-04         Up                                       
49b6559f-086b-4091-bc11-1a607928945c  29531  EXPORTING-07         Up                                       
3c18c708-9712-40a8-9c58-89c4565290fe  29942  EXPORTING-10         Up                                       
5f9f39d4-6be6-418e-b66b-6871bdcb5989  30223  EXPORTING-08         Up                                       
f45b37e3-d43e-441b-848b-896aa247b068  29675  EXPORTING-06         Up                                       
f9e93411-b4ad-48ae-ae30-051e48edf348  29795  EXPORTING-09         Up   
                                    
[root@camel-vdsc ~]# kill -9 29387 && vdsClient -s 0 list table 
3aedf776-8e7e-422a-a3c9-365d90f32299  29387  EXPORTING-03         Up                                       
044177e2-c475-4853-a2f8-a66602c71a18  29266  EXPORTING-04         Up                                       
49b6559f-086b-4091-bc11-1a607928945c  29531  EXPORTING-07         Up                                       
3c18c708-9712-40a8-9c58-89c4565290fe  29942  EXPORTING-10         Up                                       
5f9f39d4-6be6-418e-b66b-6871bdcb5989  30223  EXPORTING-08         Up                                       
f45b37e3-d43e-441b-848b-896aa247b068  29675  EXPORTING-06         Up                                       
f9e93411-b4ad-48ae-ae30-051e48edf348  29795  EXPORTING-09         Up   

Verified vdsm-4.9-82.el6

Comment 7 errata-xmlrpc 2011-12-06 07:30:44 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHEA-2011-1782.html