Bug 790014

Summary: [ovirt] [vdsm] stop spm fails when trying to deactivate last domain in pool which is master
Product: [Retired] oVirt Reporter: Haim <hateya>
Component: vdsmAssignee: Dan Kenigsberg <danken>
Status: CLOSED CURRENTRELEASE QA Contact:
Severity: urgent Docs Contact:
Priority: unspecified    
Version: unspecifiedCC: abaron, acathrow, alitke, bazulay, ewarszaw, fsimonce, hateya, iheim, mgoldboi, oramraz, yeylon, ykaul
Target Milestone: ---Keywords: Regression, TestBlocker
Target Release: 3.1   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: vdsm-4.9.4-3 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-08-09 08:05:17 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Haim 2012-02-13 13:50:46 UTC
Description of problem:

- pool with 1 iscsi storage domain which is master 
- put last domain in maintenance
- operation (deactivate domain, spm stop)fails on secure error 

The problem:

in deactivateSD flow, we check if domain to be deactivated is master, if its master, we call the migrateMaster flow, in migrareMaster flow, we check if domain == blankUUID, if it does, then we assume its the last domain in pool, and eventually call the spmStop flow, in the flow, we intentionally call the setUnsafe() function, later on in this function, we try to set some params in metadata, but we're not allowed, since we are no longer SPM (which is not so true, we just in the process of releasing SPM).

Thread-2663::ERROR::2012-02-13 12:21:33,201::task::855::TaskManager.Task::(_setError) Task=`c2dd038d-1805-467f-87ca-5fb12b23480a`::Unexpected error
Traceback (most recent call last):
  File "/usr/share/vdsm/storage/task.py", line 863, in _run
    return fn(*args, **kargs)
  File "/usr/share/vdsm/logUtils.py", line 38, in wrapper
    res = f(*args, **kwargs)
  File "/usr/share/vdsm/storage/hsm.py", line 897, in deactivateStorageDomain
    pool.deactivateSD(sdUUID, msdUUID, masterVersion)
  File "/usr/share/vdsm/storage/securable.py", line 80, in wrapper
    return f(*args, **kwargs)
  File "/usr/share/vdsm/storage/sp.py", line 1080, in deactivateSD
    self.setMetaParam(PMDK_DOMAINS, domList)
  File "/usr/share/vdsm/storage/securable.py", line 78, in wrapper
    raise SecureError()
SecureError


Thread-77::INFO::2012-02-13 12:46:30,327::logUtils::37::dispatcher::(wrapper) Run and protect: spmStop(spUUID='9bc9f96c-b065-40db-a28b-6efe4dd836c4', options=None)
Thread-77::DEBUG::2012-02-13 12:46:30,328::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.9bc9f96c-b065-40db-a28b-6efe4dd836c4`ReqID=`63690d1d-ffef-4681-b873-b1ff40ad085e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '433' at 'spmStop'
Thread-77::DEBUG::2012-02-13 12:46:30,328::resourceManager::486::ResourceManager::(registerResource) Trying to register resource 'Storage.9bc9f96c-b065-40db-a28b-6efe4dd836c4' for lock type 'exclusive'
Thread-77::DEBUG::2012-02-13 12:46:30,328::resourceManager::528::ResourceManager::(registerResource) Resource 'Storage.9bc9f96c-b065-40db-a28b-6efe4dd836c4' is free. Now locking as 'exclusive' (1 active user)
Thread-77::DEBUG::2012-02-13 12:46:30,329::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.9bc9f96c-b065-40db-a28b-6efe4dd836c4`ReqID=`63690d1d-ffef-4681-b873-b1ff40ad085e`::Granted request
Thread-77::DEBUG::2012-02-13 12:46:30,329::task::819::TaskManager.Task::(resourceAcquired) Task=`a5486112-b93c-40ad-a96d-1286b736daf6`::_resourcesAcquired: Storage.9bc9f96c-b065-40db-a28b-6efe4dd836c4 (exclusive)
Thread-77::DEBUG::2012-02-13 12:46:30,330::task::980::TaskManager.Task::(_decref) Task=`a5486112-b93c-40ad-a96d-1286b736daf6`::ref 1 aborting False
Thread-77::ERROR::2012-02-13 12:46:30,330::task::855::TaskManager.Task::(_setError) Task=`a5486112-b93c-40ad-a96d-1286b736daf6`::Unexpected error
Traceback (most recent call last):
  File "/usr/share/vdsm/storage/task.py", line 863, in _run
    return fn(*args, **kargs)
  File "/usr/share/vdsm/logUtils.py", line 38, in wrapper
    res = f(*args, **kwargs)
  File "/usr/share/vdsm/storage/hsm.py", line 436, in spmStop
    pool.stopSpm()
  File "/usr/share/vdsm/storage/securable.py", line 78, in wrapper
    raise SecureError()
SecureError

Comment 1 Federico Simoncelli 2012-02-13 16:56:52 UTC
*** Bug 789381 has been marked as a duplicate of this bug. ***

Comment 2 Dan Kenigsberg 2012-02-17 09:42:52 UTC
For the record: this problem has been exposed after adding support of sanlock-based locking for SPM. This made it apparent that Vdsm has attempted keeping the SPM role even after deactivating the last domain of a pool.

http://gerrit.ovirt.org/#change,1876

silently ignores such a request.

Comment 3 Itamar Heim 2012-08-09 08:05:17 UTC
closing ON_QA bugs as oVirt 3.1 was released:
http://www.ovirt.org/get-ovirt/