Bug 770287

Summary: [ovirt] [vdsm] getSpmStatus returns with null when master domain is down
Product: [Retired] oVirt Reporter: Haim <hateya>
Component: vdsmAssignee: Dan Kenigsberg <danken>
Status: CLOSED WONTFIX QA Contact:
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: unspecifiedCC: abaron, acathrow, amureini, bazulay, iheim, mgoldboi, yeylon, ykaul
Target Milestone: ---   
Target Release: 3.3.4   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: storage
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-03-12 09:36:54 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Storage RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Haim 2011-12-25 16:51:08 UTC
Description of problem:

This bug was opened once, got fixed, but somehow regression was introduced. 

case: 

when master domain is down, getSpmStatus is sent on the backend side, vdsm tries to read metadata, failed, returns the following exceptions: 

AttributeError: Failed reload: 56974101-96dc-4eff-b579-49822a51bc96

and: 

MetaDataParamError: Meta Data parameter invalid: ('Version or spm id invalid',)

backend parse the second exception, and interpretors it as null, which creates a sever delay in their process of reconstructing master domain.
-----------------------------------------------------------------------------

the original problem was fixed by adding new exception: se.LogicalVolumeRefreshError

but, from some reason, other exception is now caught:

raise se.MetaDataParamError("Version or spm id invalid")

 def public_getSpmStatus(self, spUUID, options = None):
        pool = self.getPool(spUUID)
        try:
            status = {'spmStatus':pool.getSpmRole(), 'spmLver': pool.getSpmLver(), 'spmId':pool.getSpmId()}
        except se.LogicalVolumeRefreshError:
            # This happens when we cannot read the MD LV
            raise se.CannotRetrieveSpmStatus()
        except se.StorageException:
            self.log.error("Unexpected error", exc_info=True)
            raise
        except Exception:
            self.log.error("Unexpected error", exc_info=True)
            raise se.MetaDataParamError("Version or spm id invalid")

        return dict(spm_st=status)


Traceback: 

Thread-501::ERROR::2011-12-25 20:02:29,108::hsm::432::Storage.HSM::(public_getSpmStatus) Unexpected error
Traceback (most recent call last):
  File "/usr/share/vdsm/storage/hsm.py", line 424, in public_getSpmStatus
    status = {'spmStatus':pool.getSpmRole(), 'spmLver': pool.getSpmLver(), 'spmId':pool.getSpmId()}
  File "/usr/share/vdsm/storage/sp.py", line 210, in getSpmLver
    return self.getMetaParam(PMDK_LVER)
  File "/usr/share/vdsm/storage/sp.py", line 1475, in getMetaParam
    return self._metadata[key]
  File "/usr/share/vdsm/storage/persistentDict.py", line 75, in __getitem__
    return dec(self._dict[key])
  File "/usr/share/vdsm/storage/persistentDict.py", line 183, in __getitem__
    with self._accessWrapper():
  File "/usr/lib64/python2.6/contextlib.py", line 16, in __enter__
    return self.gen.next()
  File "/usr/share/vdsm/storage/persistentDict.py", line 137, in _accessWrapper
    self.refresh()
  File "/usr/share/vdsm/storage/persistentDict.py", line 214, in refresh
    lines = self._metaRW.readlines()
  File "/usr/share/vdsm/storage/blockSD.py", line 135, in readlines
    for tag in vg.tags:
  File "/usr/share/vdsm/storage/lvm.py", line 165, in __getattr__
    raise AttributeError("Failed reload: %s" % self.name)
AttributeError: Failed reload: 56974101-96dc-4eff-b579-49822a51bc96
Thread-501::ERROR::2011-12-25 20:02:29,109::task::855::TaskManager.Task::(_setError) Task=`a0d2f255-a190-484c-83a7-0408a7ea2253`::Unexpected error
Traceback (most recent call last):
  File "/usr/share/vdsm/storage/task.py", line 863, in _run
    return fn(*args, **kargs)
  File "/usr/share/vdsm/logUtils.py", line 38, in wrapper
    res = f(*args, **kwargs)
  File "/usr/share/vdsm/storage/hsm.py", line 433, in public_getSpmStatus
    raise se.MetaDataParamError("Version or spm id invalid")
MetaDataParamError: Meta Data parameter invalid: ('Version or spm id invalid',)

git commit: d458e25fddbe920085f39d1d7eeffa9ac2eba341

Comment 1 Itamar Heim 2013-03-12 09:36:54 UTC
Closing old bugs. If this issue is still relevant/important in current version, please re-open the bug.