Bug 853704

Summary: ovirt-engine-backend [TEXT]: same action in progress error when putting spm and master domain in maintenance at the same time
Product: Red Hat Enterprise Virtualization Manager Reporter: Dafna Ron <dron>
Component: ovirt-engineAssignee: Oved Ourfali <oourfali>
Status: CLOSED CURRENTRELEASE QA Contact: Leonid Natapov <lnatapov>
Severity: low Docs Contact:
Priority: medium    
Version: 3.1.0CC: acathrow, bazulay, dyasny, hateya, iheim, lpeer, pstehlik, Rhev-m-bugs, sgrinber, yeylon, ykaul, yzaslavs
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard: infra
Fixed In Version: si19 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-12-04 20:03:51 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Infra RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
log none

Description Dafna Ron 2012-09-02 09:26:36 UTC
Created attachment 609038 [details]
log

Description of problem:

when we try to put SPM host and master SD in maintenance at the same time we are getting CanDoAction: "Same Action in Progress" from locking mechanism. 

2012-09-02 12:15:43,521 WARN  [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (ajp-/127.0.0.1:8009-4) CanDoAction of action DeactivateStorageDomain failed. R
easons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ACTION_TYPE_FAILED_OBJECT_LOCKED


Version-Release number of selected component (if applicable):

si16

How reproducible:

100%

Steps to Reproduce:
1. in two hosts cluster with multiple domains -> put spm + master domain in maintenance at the same time. 
2.
3.
  
Actual results:

we are failing on lock with CanDoAction: 
2012-09-02 12:15:43,521 WARN  [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (ajp-/127.0.0.1:8009-4) CanDoAction of action DeactivateStorageDomain failed. R
easons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ACTION_TYPE_FAILED_OBJECT_LOCKED


Expected results:

since spm recontending and master domain reconstruct are not the same actions we should have a different canDoAction. 

Additional info:backend log

Comment 1 Ayal Baron 2012-09-02 11:16:29 UTC
(In reply to comment #0)
> Created attachment 609038 [details]
> log
> 
> Description of problem:
> 
> when we try to put SPM host and master SD in maintenance at the same time we
> are getting CanDoAction: "Same Action in Progress" from locking mechanism. 
> 
> 2012-09-02 12:15:43,521 WARN 
> [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand]
> (ajp-/127.0.0.1:8009-4) CanDoAction of action DeactivateStorageDomain
> failed. R
> easons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,
> ACTION_TYPE_FAILED_OBJECT_LOCKED
> 
> 
> Version-Release number of selected component (if applicable):
> 
> si16
> 
> How reproducible:
> 
> 100%
> 
> Steps to Reproduce:
> 1. in two hosts cluster with multiple domains -> put spm + master domain in
> maintenance at the same time. 
> 2.
> 3.
>   
> Actual results:
> 
> we are failing on lock with CanDoAction: 
> 2012-09-02 12:15:43,521 WARN 
> [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand]
> (ajp-/127.0.0.1:8009-4) CanDoAction of action DeactivateStorageDomain
> failed. R
> easons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,
> ACTION_TYPE_FAILED_OBJECT_LOCKED
> 
> 
> Expected results:
> 
> since spm recontending and master domain reconstruct are not the same
> actions we should have a different canDoAction. 
> 
> Additional info:backend log

This is not reconstruct, but in any event the error is wrong in general.

This is in CommandBase.java:

log.infoFormat("Failed to Acquire Lock to object {0}", lock);                    addCanDoActionMessage(VdcBllMessages.ACTION_TYPE_FAILED_OBJECT_LOCKED);

Comment 2 Oved Ourfali 2012-09-04 05:59:28 UTC
(In reply to comment #1)
> (In reply to comment #0)
> > Created attachment 609038 [details]
> > log
> > 
> > Description of problem:
> > 
> > when we try to put SPM host and master SD in maintenance at the same time we
> > are getting CanDoAction: "Same Action in Progress" from locking mechanism. 
> > 
> > 2012-09-02 12:15:43,521 WARN 
> > [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand]
> > (ajp-/127.0.0.1:8009-4) CanDoAction of action DeactivateStorageDomain
> > failed. R
> > easons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,
> > ACTION_TYPE_FAILED_OBJECT_LOCKED
> > 
> > 
> > Version-Release number of selected component (if applicable):
> > 
> > si16
> > 
> > How reproducible:
> > 
> > 100%
> > 
> > Steps to Reproduce:
> > 1. in two hosts cluster with multiple domains -> put spm + master domain in
> > maintenance at the same time. 
> > 2.
> > 3.
> >   
> > Actual results:
> > 
> > we are failing on lock with CanDoAction: 
> > 2012-09-02 12:15:43,521 WARN 
> > [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand]
> > (ajp-/127.0.0.1:8009-4) CanDoAction of action DeactivateStorageDomain
> > failed. R
> > easons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,
> > ACTION_TYPE_FAILED_OBJECT_LOCKED
> > 
> > 
> > Expected results:
> > 
> > since spm recontending and master domain reconstruct are not the same
> > actions we should have a different canDoAction. 
> > 
> > Additional info:backend log
> 
> This is not reconstruct, but in any event the error is wrong in general.
> 
> This is in CommandBase.java:
> 
> log.infoFormat("Failed to Acquire Lock to object {0}", lock);               
> addCanDoActionMessage(VdcBllMessages.ACTION_TYPE_FAILED_OBJECT_LOCKED);

ACTION_TYPE_FAILED_OBJECT_LOCKED is mapped to the following message:

Cannot ${action} ${type}. Same action in progress.

Maybe we should change it to:

Cannot ${action} ${type}. Related operation is currently on progress. Please try again later.
Other suggestions are welcome.

Ayal - any thoghts about that?

Comment 5 Oved Ourfali 2012-09-24 14:49:26 UTC
Posted patch:
http://gerrit.ovirt.org/8166

Comment 6 Oved Ourfali 2012-09-24 14:55:58 UTC
Commit: be6f6fcc8a5df1d3c6ab1f07b91dfef6bc448bcc

http://gerrit.ovirt.org/gitweb?p=ovirt-engine.git;a=commit;h=be6f6fcc8a5df1d3c6ab1f07b91dfef6bc448bcc

Comment 8 Leonid Natapov 2012-10-02 13:05:36 UTC
si19. fixed. getting the following message:

"Error while executing action: Cannot maintenance Host. Related operation is currently in progress. Please try again later."