Bug 783930

Created attachment 556884 [details]
engine + vdsm logs

Description of problem:

Unable to move SPM to an Active host.

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1. Working setup with 2 hosts. host A is Active and SPM. host B is in maintenence .
2.Disconnect host A from network .
3.Activate host B. Verify it is active.
4.On host A - Confirm host has been rebooted 

See attached vdsm and engine logs. 


Actual results:
SPM is locked on a "dead" host A.
Error messages

Expected results:
Host B is SPM

Additional info:
Host A dosn't have PM configuration in engine.

SecureError
Thread-1959::DEBUG::2012-01-23 11:02:08,850::task::874::TaskManager.Task::(_run) Task=`9a8feb84-fd3f-45b7-b656-2c2ecd12c994`::Task._run: 9a8feb84-fd3f-45b7-b656-2c2ecd12c994 ('eedadac2-0559-4037-98
c5-669b059f7ab2', 5, '14') {} failed - stopping task
Thread-1959::DEBUG::2012-01-23 11:02:08,850::task::1201::TaskManager.Task::(stop) Task=`9a8feb84-fd3f-45b7-b656-2c2ecd12c994`::stopping in state preparing (force False)
Thread-1959::DEBUG::2012-01-23 11:02:08,851::task::980::TaskManager.Task::(_decref) Task=`9a8feb84-fd3f-45b7-b656-2c2ecd12c994`::ref 1 aborting True
Thread-1959::INFO::2012-01-23 11:02:08,851::task::1159::TaskManager.Task::(prepare) Task=`9a8feb84-fd3f-45b7-b656-2c2ecd12c994`::aborting: Task is aborted: '' - code 100
Thread-1959::DEBUG::2012-01-23 11:02:08,851::task::1164::TaskManager.Task::(prepare) Task=`9a8feb84-fd3f-45b7-b656-2c2ecd12c994`::Prepare: aborted: 
Thread-1959::DEBUG::2012-01-23 11:02:08,852::task::980::TaskManager.Task::(_decref) Task=`9a8feb84-fd3f-45b7-b656-2c2ecd12c994`::ref 0 aborting True
Thread-1959::DEBUG::2012-01-23 11:02:08,852::task::915::TaskManager.Task::(_doAbort) Task=`9a8feb84-fd3f-45b7-b656-2c2ecd12c994`::Task._doAbort: force False
Thread-1959::DEBUG::2012-01-23 11:02:08,853::resourceManager::841::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
Thread-1959::DEBUG::2012-01-23 11:02:08,853::task::588::TaskManager.Task::(_updateState) Task=`9a8feb84-fd3f-45b7-b656-2c2ecd12c994`::moving from state preparing -> state aborting
Thread-1959::DEBUG::2012-01-23 11:02:08,853::task::537::TaskManager.Task::(__state_aborting) Task=`9a8feb84-fd3f-45b7-b656-2c2ecd12c994`::_aborting: recover policy none
Thread-1959::DEBUG::2012-01-23 11:02:08,854::task::588::TaskManager.Task::(_updateState) Task=`9a8feb84-fd3f-45b7-b656-2c2ecd12c994`::moving from state aborting -> state failed
Thread-1959::DEBUG::2012-01-23 11:02:08,854::resourceManager::806::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.eedadac2-0559-4037-98c5-669b059f7ab2': < Reso
urceRef 'Storage.eedadac2-0559-4037-98c5-669b059f7ab2', isValid: 'True' obj: 'None'>}
Thread-1959::DEBUG::2012-01-23 11:02:08,854::resourceManager::841::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
Thread-1959::DEBUG::2012-01-23 11:02:08,855::resourceManager::535::ResourceManager::(releaseResource) Trying to release resource 'Storage.eedadac2-0559-4037-98c5-669b059f7ab2'
Thread-1959::DEBUG::2012-01-23 11:02:08,855::resourceManager::550::ResourceManager::(releaseResource) Released resource 'Storage.eedadac2-0559-4037-98c5-669b059f7ab2' (0 active users)
Thread-1959::DEBUG::2012-01-23 11:02:08,856::resourceManager::555::ResourceManager::(releaseResource) Resource 'Storage.eedadac2-0559-4037-98c5-669b059f7ab2' is free, finding out if anyone is waiti
ng for it.
Thread-1959::DEBUG::2012-01-23 11:02:08,856::resourceManager::562::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.eedadac2-0559-4037-98c5-669b059f7ab2', Clearing records
.
Thread-1959::ERROR::2012-01-23 11:02:08,857::dispatcher::93::Storage.Dispatcher.Protect::(run) 
Traceback (most recent call last):
  File "/usr/share/vdsm/storage/dispatcher.py", line 85, in run
    result = ctask.prepare(self.func, *args, **kwargs)
  File "/usr/share/vdsm/storage/task.py", line 1166, in prepare
    raise self.error
SecureError
Thread-1963::DEBUG::2012-01-23 11:02:15,578::task::588::TaskManager.Task::(_updateState) Task=`77f1960c-90d9-42b5-8575-dd51fc228527`::moving from state init -> state preparing
Thread-1963::INFO::2012-01-23 11:02:15,578::logUtils::37::dispatcher::(wrapper) Run and protect: repoStats(options=None)
Thread-1963::INFO::2012-01-23 11:02:15,579::logUtils::39::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'924de01f-187f-411b-8615-36ba2a6a5e0e': {'delay': '0.000598192214966', 
'lastCheck': 1327309325.865981, 'code': 0, 'valid': True}, '183d2408-1f0c-4ea4-ae8e-00dd3eca15f8': {'delay': '0.00192904472351', 'lastCheck': 1327309334.789749, 'code': 0, 'valid': True}}
Thread-1963::DEBUG::2012-01-23 11:02:15,579::task::1174::TaskManager.Task::(prepare) Task=`77f1960c-90d9-42b5-8575-dd51fc228527`::finished: {'924de01f-187f-411b-8615-36ba2a6a5e0e': {'delay': '0.000
598192214966', 'lastCheck': 1327309325.865981, 'code': 0, 'valid': True}, '183d2408-1f0c-4ea4-ae8e-00dd3eca15f8': {'delay': '0.00192904472351', 'lastCheck': 1327309334.789749, 'code': 0, 'valid': T
rue}}
Thread-1963::DEBUG::2012-01-23 11:02:15,580::task::588::TaskManager.Task::(_updateState) Task=`77f1960c-90d9-42b5-8575-dd51fc228527`::moving from state preparing -> state finished