Bug 1404601 - [engine-backend] Storage server connection removal fails after a storage domain creation failure
Summary: [engine-backend] Storage server connection removal fails after a storage doma...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: ovirt-engine
Classification: oVirt
Component: Backend.Core
Version: 4.0.6.3
Hardware: x86_64
OS: Unspecified
unspecified
high
Target Milestone: ovirt-4.1.0-beta
: 4.1.0.3
Assignee: Tal Nisan
QA Contact: Elad
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-12-14 08:41 UTC by Elad
Modified: 2017-02-15 14:51 UTC (History)
3 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2017-02-15 14:51:59 UTC
oVirt Team: Storage
Embargoed:
rule-engine: ovirt-4.1+
ylavi: blocker-


Attachments (Terms of Use)
engine.log and vdsm.log (1.53 MB, application/x-gzip)
2016-12-14 08:41 UTC, Elad
no flags Details


Links
System ID Private Priority Status Summary Last Updated
oVirt gerrit 71119 0 master MERGED core: Fix storage domain creation rollback on failure 2017-01-25 14:00:00 UTC
oVirt gerrit 71165 0 ovirt-engine-4.1.0.z MERGED core: Fix storage domain creation rollback on failure 2017-01-25 14:29:35 UTC
oVirt gerrit 71170 0 ovirt-engine-4.1.0.z MERGED core: Fix storage domain creation rollback on failure 2017-01-25 14:30:15 UTC

Description Elad 2016-12-14 08:41:25 UTC
Created attachment 1231502 [details]
engine.log and vdsm.log

Description of problem:
When storage domain creation fails, engine performs roll-back to the operation and fails to remove the storage connection. It seems that no storage domain removal is being done as part of the failed storage domain creation roll-back.


Version-Release number of selected component (if applicable):
ovirt-engine-4.1.0-0.2.master.20161210231201.git26a385e.el7.centos.noarch
vdsm-4.18.999-1147.git951e122.el7.centos.x86_64


How reproducible:
Always

Steps to Reproduce:
1. Remove or destroy an NFS storage domain. Do not format the domain, leave the domain data in the storage
2. Try to add a new storage domain with the same connection details


Actual results:
Storage domain creation fails because it's not empty as should be:

2016-12-14 10:23:02,920+02 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateStorageDomainVDSCommand] (default task-7) [3515edb5] START, CreateStorageDomainVDSCommand(HostName = host1, 
CreateStorageDomainVDSCommandParameters:{runAsync='true', hostId='ea312772-52bd-47af-bd04-63097511fbda', storageDomain='StorageDomainStatic:{name='nfs1', id='9c8c04b5-326f-411d-acfd-7966a006
8e74'}', args='10.35.110.11:/Storage_NFS/elad/1'}), log id: 6fb8ed36
2016-12-14 10:23:05,082+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-7) [3515edb5] Correlation ID: null, Call Stack: null, Custom Event ID: -
1, Message: VDSM host1 command failed: Storage domain is not empty - requires cleaning: ('10.35.110.11:/Storage_NFS/elad/1',)



But then, storage connection removal fails:

2016-12-14 10:23:05,367+02 WARN  [org.ovirt.engine.core.bll.storage.connection.RemoveStorageServerConnectionCommand] (default task-14) [17d96f84-257c-4960-8f4c-f4cd283d0f68] Validation of action 'RemoveStorageServerConnection' failed for user admin@internal-authz. Reasons: VAR__ACTION__REMOVE,VAR__TYPE__STORAGE__CONNECTION,$domainNames nfs1,ACTION_TYPE_FAILED_STORAGE_CONNECTION_BELONGS_TO_SEVERAL_STORAGE_DOMAINS


The domain remains as 'unattached' in the system.
It seems that the storage connection removal attempt is being done without storage domain removal first.

Expected results:
After a failure in storage domain creation, the connection and the domain should be removed successfully

Additional info:
engine.log and vdsm.log

Comment 3 Red Hat Bugzilla Rules Engine 2016-12-27 16:39:47 UTC
This bug report has Keywords: Regression or TestBlocker.
Since no regressions or test blockers are allowed between releases, it is also being identified as a blocker for this release. Please resolve ASAP.

Comment 4 Elad 2017-02-07 14:31:25 UTC
Steps I did:

1. Destroyed an NFS storage domain
2. From the storage, cleaned the path content 
3. Tried to add a new storage domain with the same connection details


Storage domain creation succeeded.
Used:
rhevm-4.1.0.4-0.1.el7.noarch
vdsm-4.19.4-1.el7ev.x86_64


Note You need to log in before you can comment on or make changes to this bug.