Created attachment 983961[details]
logs
Description of problem:
When an operation of importing ISO domain fails due to, "failure to mount target", oVirt engine should clear it's 'storage_server_connections" table entry, but it doesn't:
engine=# SELECT id,connection, storage_type from storage_server_connections ;
id | connection | storage_type
--------------------------------------+----------------------------------------+--------------
c76a4683-a857-4173-9abb-da6cbe064cf2 | 10.35.160.108:/RHEV/ogofen/1 | 1
336dd53a-ce21-4866-8f5c-cdbeaa1efe17 | 10.35.160.105 | 3
ec00b904-d599-4d6b-a05a-fac647c3a151 | 10.35.160.104 | 3
8840a42b-f152-4a01-9818-f7c1f2db4746 | 10.35.160.106 | 3
b7a4aabb-351d-4856-9194-48f2dc402ccc | 10.35.160.107 | 3
2f4e8340-bbd2-42db-86ad-65b62c673653 | 10.35.160.108:/RHEV/ogofen/2 | 1
3a48031d-18ad-48d4-a213-940aa8b01dd8 | 10.35.160.108:/RHEV/ogofen/rhevm-3-iso | 1
685a747d-9f79-43c4-9af1-a6c34a9473ef | 10.35.160.108:/RHEV/ogofen/3 | 1
Every attempt to add the same ISO path fails.
2015-01-25 15:11:59,091 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [12786e59] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: The error message for connection 10.35.160.108:/RHEV/ogofen/iso-rhevm-3 returned by VDSM was: Problem while trying to mount target
2015-01-25 15:11:59,091 ERROR [org.ovirt.engine.core.bll.storage.NFSStorageHelper] (ajp-/127.0.0.1:8702-4) [12786e59] The connection with details 10.35.160.108:/RHEV/ogofen/iso-rhevm-3 failed because of error code 477 and error message is: problem while trying to mount target
2015-01-25 15:11:59,091 ERROR [org.ovirt.engine.core.bll.storage.AddStorageServerConnectionCommand] (ajp-/127.0.0.1:8702-4) [12786e59] Command org.ovirt.engine.core.bll.storage.AddStorageServerConnectionCommand throw Vdc Bll exception. With error message VdcBLLException: ProblemWhileTryingToMountTarget (Failed with error ProblemWhileTryingToMountTarget and code 477)
2015-01-25 15:11:59,094 ERROR [org.ovirt.engine.core.bll.storage.AddStorageServerConnectionCommand] (ajp-/127.0.0.1:8702-4) [12786e59] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.AddStorageServerConnectionCommand.
2015-01-25 15:11:59,100 INFO [org.ovirt.engine.core.bll.storage.AddStorageServerConnectionCommand] (ajp-/127.0.0.1:8702-4) [12786e59] Lock freed to object EngineLock [exclusiveLocks= key: 10.35.160.108:/RHEV/ogofen/iso-rhevm-3 value: STORAGE_CONNECTION
, sharedLocks= ]
2015-01-25 15:11:59,247 INFO [org.ovirt.engine.core.bll.storage.DisconnectStorageServerConnectionCommand] (ajp-/127.0.0.1:8702-9) [4f15bc84] Running command: DisconnectStorageServerConnectionCommand internal: false. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with role type ADMIN
2015-01-25 15:11:59,252 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (ajp-/127.0.0.1:8702-9) [4f15bc84] START, DisconnectStorageServerVDSCommand(HostName = purple-vds1.qa.lab.tlv.redhat.com, HostId = 704b1dd5-6e29-4595-9fcd-e622ac5b781e, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: null, connection: 10.35.160.108:/RHEV/ogofen/iso-rhevm-3, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 1fa2b19c
2015-01-25 15:11:59,544 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (ajp-/127.0.0.1:8702-9) [4f15bc84] FINISH, DisconnectStorageServerVDSCommand, return: {00000000-0000-0000-0000-000000000000=477}, log id: 1fa2b19c
2015-01-25 15:13:18,134 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-10) Running command: LoginUserCommand internal: false.
2015-01-25 15:13:18,138 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in.
2015-01-25 15:13:20,294 INFO [org.ovirt.engine.core.bll.storage.AddStorageServerConnectionCommand] (ajp-/127.0.0.1:8702-5) [6cde5c84] Lock Acquired to object EngineLock [exclusiveLocks= key: 10.35.160.108:/RHEV/ogofen/rhevm-3-iso value: STORAGE_CONNECTION
, sharedLocks= ]
2015-01-25 15:13:20,300 WARN [org.ovirt.engine.core.bll.storage.AddStorageServerConnectionCommand] (ajp-/127.0.0.1:8702-5) [6cde5c84] CanDoAction of action AddStorageServerConnection failed. Reasons:VAR__ACTION__ADD,VAR__TYPE__STORAGE__CONNECTION,ACTION_TYPE_FAILED_STORAGE_CONNECTION_ALREADY_EXISTS
2015-01-25 15:13:20,300 INFO [org.ovirt.engine.core.bll.storage.AddStorageServerConnectionCommand] (ajp-/127.0.0.1:8702-5) [6cde5c84] Lock freed to object EngineLock [exclusiveLocks= key: 10.35.160.108:/RHEV/ogofen/rhevm-3-iso value: STORAGE_CONNECTION
, sharedLocks= ]
2015-01-25 15:13:20,311 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (ajp-/127.0.0.1:8702-5) Operation Failed: [Cannot add Storage Connection. Storage connection already exists.]
Version-Release number of selected component (if applicable):
vt13.8
How reproducible:
100%
Steps to Reproduce:
1.add an iso domain without having the proper permissions (operation will fail)
2.fix the permissions and try to add the domain
Actual results:
oVirt doesn't clear iso server connection in case of, "failure to mount target"
Expected results:
The 'storage_server_connections" db table should be cleared after a failure to create iso domain, the operation of adding an iso domain should be successful if all the terms to create one are met
Additional info:
Created attachment 983961 [details] logs Description of problem: When an operation of importing ISO domain fails due to, "failure to mount target", oVirt engine should clear it's 'storage_server_connections" table entry, but it doesn't: engine=# SELECT id,connection, storage_type from storage_server_connections ; id | connection | storage_type --------------------------------------+----------------------------------------+-------------- c76a4683-a857-4173-9abb-da6cbe064cf2 | 10.35.160.108:/RHEV/ogofen/1 | 1 336dd53a-ce21-4866-8f5c-cdbeaa1efe17 | 10.35.160.105 | 3 ec00b904-d599-4d6b-a05a-fac647c3a151 | 10.35.160.104 | 3 8840a42b-f152-4a01-9818-f7c1f2db4746 | 10.35.160.106 | 3 b7a4aabb-351d-4856-9194-48f2dc402ccc | 10.35.160.107 | 3 2f4e8340-bbd2-42db-86ad-65b62c673653 | 10.35.160.108:/RHEV/ogofen/2 | 1 3a48031d-18ad-48d4-a213-940aa8b01dd8 | 10.35.160.108:/RHEV/ogofen/rhevm-3-iso | 1 685a747d-9f79-43c4-9af1-a6c34a9473ef | 10.35.160.108:/RHEV/ogofen/3 | 1 Every attempt to add the same ISO path fails. 2015-01-25 15:11:59,091 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [12786e59] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: The error message for connection 10.35.160.108:/RHEV/ogofen/iso-rhevm-3 returned by VDSM was: Problem while trying to mount target 2015-01-25 15:11:59,091 ERROR [org.ovirt.engine.core.bll.storage.NFSStorageHelper] (ajp-/127.0.0.1:8702-4) [12786e59] The connection with details 10.35.160.108:/RHEV/ogofen/iso-rhevm-3 failed because of error code 477 and error message is: problem while trying to mount target 2015-01-25 15:11:59,091 ERROR [org.ovirt.engine.core.bll.storage.AddStorageServerConnectionCommand] (ajp-/127.0.0.1:8702-4) [12786e59] Command org.ovirt.engine.core.bll.storage.AddStorageServerConnectionCommand throw Vdc Bll exception. With error message VdcBLLException: ProblemWhileTryingToMountTarget (Failed with error ProblemWhileTryingToMountTarget and code 477) 2015-01-25 15:11:59,094 ERROR [org.ovirt.engine.core.bll.storage.AddStorageServerConnectionCommand] (ajp-/127.0.0.1:8702-4) [12786e59] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.AddStorageServerConnectionCommand. 2015-01-25 15:11:59,100 INFO [org.ovirt.engine.core.bll.storage.AddStorageServerConnectionCommand] (ajp-/127.0.0.1:8702-4) [12786e59] Lock freed to object EngineLock [exclusiveLocks= key: 10.35.160.108:/RHEV/ogofen/iso-rhevm-3 value: STORAGE_CONNECTION , sharedLocks= ] 2015-01-25 15:11:59,247 INFO [org.ovirt.engine.core.bll.storage.DisconnectStorageServerConnectionCommand] (ajp-/127.0.0.1:8702-9) [4f15bc84] Running command: DisconnectStorageServerConnectionCommand internal: false. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with role type ADMIN 2015-01-25 15:11:59,252 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (ajp-/127.0.0.1:8702-9) [4f15bc84] START, DisconnectStorageServerVDSCommand(HostName = purple-vds1.qa.lab.tlv.redhat.com, HostId = 704b1dd5-6e29-4595-9fcd-e622ac5b781e, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: null, connection: 10.35.160.108:/RHEV/ogofen/iso-rhevm-3, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 1fa2b19c 2015-01-25 15:11:59,544 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (ajp-/127.0.0.1:8702-9) [4f15bc84] FINISH, DisconnectStorageServerVDSCommand, return: {00000000-0000-0000-0000-000000000000=477}, log id: 1fa2b19c 2015-01-25 15:13:18,134 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-10) Running command: LoginUserCommand internal: false. 2015-01-25 15:13:18,138 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. 2015-01-25 15:13:20,294 INFO [org.ovirt.engine.core.bll.storage.AddStorageServerConnectionCommand] (ajp-/127.0.0.1:8702-5) [6cde5c84] Lock Acquired to object EngineLock [exclusiveLocks= key: 10.35.160.108:/RHEV/ogofen/rhevm-3-iso value: STORAGE_CONNECTION , sharedLocks= ] 2015-01-25 15:13:20,300 WARN [org.ovirt.engine.core.bll.storage.AddStorageServerConnectionCommand] (ajp-/127.0.0.1:8702-5) [6cde5c84] CanDoAction of action AddStorageServerConnection failed. Reasons:VAR__ACTION__ADD,VAR__TYPE__STORAGE__CONNECTION,ACTION_TYPE_FAILED_STORAGE_CONNECTION_ALREADY_EXISTS 2015-01-25 15:13:20,300 INFO [org.ovirt.engine.core.bll.storage.AddStorageServerConnectionCommand] (ajp-/127.0.0.1:8702-5) [6cde5c84] Lock freed to object EngineLock [exclusiveLocks= key: 10.35.160.108:/RHEV/ogofen/rhevm-3-iso value: STORAGE_CONNECTION , sharedLocks= ] 2015-01-25 15:13:20,311 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (ajp-/127.0.0.1:8702-5) Operation Failed: [Cannot add Storage Connection. Storage connection already exists.] Version-Release number of selected component (if applicable): vt13.8 How reproducible: 100% Steps to Reproduce: 1.add an iso domain without having the proper permissions (operation will fail) 2.fix the permissions and try to add the domain Actual results: oVirt doesn't clear iso server connection in case of, "failure to mount target" Expected results: The 'storage_server_connections" db table should be cleared after a failure to create iso domain, the operation of adding an iso domain should be successful if all the terms to create one are met Additional info: