Bug 708773 - VDSM: if umount fails during create domain the storage is created but the user will get "can't find master domain" error
Summary: VDSM: if umount fails during create domain the storage is created but the use...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: vdsm
Version: 6.2
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: rc
: ---
Assignee: Dan Kenigsberg
QA Contact: Dafna Ron
URL:
Whiteboard: storage
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2011-05-29 13:43 UTC by Dafna Ron
Modified: 2011-12-06 07:19 UTC (History)
7 users (show)

Fixed In Version: vdsm-4.9-72.el6
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2011-12-06 07:19:36 UTC
Target Upstream Version:


Attachments (Terms of Use)
logs (369.21 KB, application/x-gzip)
2011-05-29 13:43 UTC, Dafna Ron
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHEA-2011:1782 0 normal SHIPPED_LIVE new packages: vdsm 2011-12-06 11:55:51 UTC

Description Dafna Ron 2011-05-29 13:43:33 UTC
Created attachment 501603 [details]
logs

Description of problem:

if umount fails during domain create the storage will be created but user will not be able to use it. 
the domain will be created and a "can't find master storage domain" error will also be sent. 



Version-Release number of selected component (if applicable):

vdsm-4.9-70.el6.x86_64
libvirt-0.8.7-18.el6.x86_64

How reproducible:


Steps to Reproduce:
1. fail umount during SD create
2.
3.
  
Actual results:

the storage domain will remain mounted and allthough SD will be successfully created the user will get "can't find master domain" error  and storage will not be usable. 


Expected results:

we should be able to handle cases where umount fails during creating of SD

Additional info: full logs attached

Thread-542::ERROR::2011-05-29 14:52:54,852::sp::344::Storage.StoragePool::(create) Create domain canceled due to an unexpected error
Traceback (most recent call last):
  File "/usr/share/vdsm/storage/sp.py", line 327, in create
    self.createMaster(poolName, msd, masterVersion, safeLease)
  File "/usr/share/vdsm/storage/sp.py", line 500, in createMaster
    domain.unmountMaster()
  File "/usr/share/vdsm/storage/blockSD.py", line 893, in unmountMaster
    self.doUnmountMaster(masterdir)
  File "/usr/share/vdsm/storage/blockSD.py", line 886, in doUnmountMaster
    raise se.StorageDomainMasterUnmountError(masterdir, rc)
StorageDomainMasterUnmountError: Error unmounting master storage domain: 'masterdir=/rhev/data-center/mnt/blockSD/e6cae5cc-65fa-411c-a7f7-27f600e13da0/master, rc=1'
Thread-542::INFO::2011-05-29 14:52:54,868::sp::743::Storage.StoragePool::(detachSD) sdUUID=<storage.blockSD.BlockStorageDomain instance at 0x7ff23c238fc8> spUUID=bb0c60f3-0df2-4305-a81e-e0df186bfab7 msdUUID=e6cae5cc-65fa-411c-a7f7-27f600e13da0



Thread-542::ERROR::2011-05-29 14:52:57,374::sp::1169::Storage.StoragePool::(getMasterDomain) Couldn't find master domain for pool bb0c60f3-0df2-4305-a81e-e0df186bfab7
Traceback (most recent call last):
  File "/usr/share/vdsm/storage/sp.py", line 190, in __cleanupDomains
    self.detachSD(d, msdUUID, masterVersion)
  File "/usr/share/vdsm/storage/sp.py", line 745, in detachSD
    dom = SDF.produce(sdUUID)
  File "/usr/share/vdsm/storage/sdf.py", line 32, in produce
    raise se.StorageDomainDoesNotExist(sdUUID)
StorageDomainDoesNotExist: Storage domain does not exist: (<storage.blockSD.BlockStorageDomain instance at 0x7ff23c238fc8>,)
Thread-542::ERROR::2011-05-29 14:52:57,374::sp::350::Storage.StoragePool::(create) Cleanup failed due to an unexpected error
Traceback (most recent call last):
  File "/usr/share/vdsm/storage/sp.py", line 348, in create
    self.__cleanupDomains(domList, msdUUID, masterVersion)
  File "/usr/share/vdsm/storage/sp.py", line 193, in __cleanupDomains
    self.refresh()
  File "/usr/share/vdsm/storage/sp.py", line 960, in refresh
    self.__rebuild(msdUUID=msdUUID, masterVersion=masterVersion)
  File "/usr/share/vdsm/storage/sp.py", line 906, in __rebuild
    msdUUID = self.getMasterDomain(msdUUID=msdUUID, masterVersion=masterVersion).sdUUID
  File "/usr/share/vdsm/storage/sp.py", line 1170, in getMasterDomain
    raise se.StoragePoolMasterNotFound(self.spUUID, str(msdUUID))
StoragePoolMasterNotFound: Cannot find master domain: 'spUUID=bb0c60f3-0df2-4305-a81e-e0df186bfab7, msdUUID=None'
Thread-542::INFO::2011-05-29 14:52:57,375::safelease::83::ClusterLock::(release) Releasing cluster lock for domain e6cae5cc-65fa-411c-a7f7-27f600e13da0
Thread-542::DEBUG::2011-05-29 14:52:57,375::safelease::84::Storage.Misc.excCmd::(release) '/usr/libexec/vdsm/spmstop.sh e6cae5cc-65fa-411c-a7f7-27f600e13da0' (cwd /usr/libexec/vdsm/)

Comment 2 Eduardo Warszawski 2011-05-30 08:33:09 UTC
http://gerrit.usersys.redhat.com/#change,491

Comment 5 Jakub Libosvar 2011-06-07 09:40:28 UTC
Verified in vdsm-4.9-73.el6.x86_64

Comment 6 errata-xmlrpc 2011-12-06 07:19:36 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHEA-2011-1782.html


Note You need to log in before you can comment on or make changes to this bug.