Bug 2022660 - Removing a iSCSI storage connection removes all newly added connections
Summary: Removing a iSCSI storage connection removes all newly added connections
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-engine
Version: 4.4.8
Hardware: All
OS: Linux
high
high
Target Milestone: ovirt-4.4.10
: 4.4.10
Assignee: shani
QA Contact: sshmulev
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-11-12 10:21 UTC by nijin ashok
Modified: 2022-12-08 09:08 UTC (History)
7 users (show)

Fixed In Version: ovirt-engine-4.4.10.1
Doc Type: Bug Fix
Doc Text:
Previously, when unassigning a storage connection from a storage domain for a LUN associated with another storage server connection, all storage connections for that LUN were removed and the LUN was also removed. In this release, only the specified storage connection is removed. The LUN is removed only if it has no storage connections.
Clone Of:
Environment:
Last Closed: 2022-02-08 10:04:44 UTC
oVirt Team: Storage
Target Upstream Version:
Embargoed:
sshmulev: testing_plan_complete+


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker RHV-43992 0 None None None 2021-11-12 10:24:56 UTC
Red Hat Product Errata RHBA-2022:0461 0 None None None 2022-02-08 10:05:02 UTC
oVirt gerrit 117912 0 master MERGED core: Remove a specific storage connection instead of a LUN 2021-12-08 13:36:08 UTC
oVirt gerrit 118006 0 ovirt-engine-4.4 MERGED core: Remove a specific storage connection instead of a LUN 2021-12-13 16:03:17 UTC

Description nijin ashok 2021-11-12 10:21:34 UTC
Description of problem:

Two new storage connection was added and attached to an existing storage domain using RHV API. This created a new LUN entry in the lun_storage_server_connection_map as DUMMY_LUN_SDUUID:

~~~
engine=# select * from lun_storage_server_connection_map where lun_id = 'DUMMY_LUN_b30addc2-3067-42a2-98e6-765e4893d866';
                     lun_id                     |      storage_server_connection       
------------------------------------------------+--------------------------------------
 DUMMY_LUN_b30addc2-3067-42a2-98e6-765e4893d866 | f4c1e377-98cd-4508-b359-96a43940d772
 DUMMY_LUN_b30addc2-3067-42a2-98e6-765e4893d866 | 4aa0e44a-de51-4d11-ac8a-7ddba7f681b0
 DUMMY_LUN_b30addc2-3067-42a2-98e6-765e4893d866 | b78ce692-bf85-481f-b7e0-5b168b713d40
(3 rows)
~~~

Out of this, detached one of the newly added connections from the storage domain.

~~~
SD_UUID="b30addc2-3067-42a2-98e6-765e4893d866"
SD_CONNECTION_UUID="4aa0e44a-de51-4d11-ac8a-7ddba7f681b0"
system_service = connection.system_service()
sd_service = system_service.storage_domains_service().storage_domain_service(id=SD_UUID)
sd_connections_service = sd_service.storage_connections_service()
sd_connection_service = sd_connections_service.connection_service(SD_CONNECTION_UUID)
sd_connection_service.remove()
~~~

However, this removed all the newly added connections mapped to the SD.

~~~
engine=# select * from lun_storage_server_connection_map where lun_id = 'DUMMY_LUN_b30addc2-3067-42a2-98e6-765e4893d866';
 lun_id | storage_server_connection 
--------+---------------------------
(0 rows)
~~~

Version-Release number of selected component (if applicable):

rhvm-4.4.8.6-0.1.el8ev.noarch


How reproducible:

100%

Steps to Reproduce:

1. Add two new iSCSI connections to an existing storage domain.
2. Remove one of the newly added connections.
3. It removed both the connections.

Actual results:

Removing an iSCSI storage connection removes all newly added connections.

Expected results:

It should only remove the connection which it is asked to remove.

Additional info:

Comment 1 shani 2021-11-30 14:47:07 UTC
Hi,
Can you please share the API request for creating the connections for the same LUN?

Comment 2 nijin ashok 2021-12-01 01:50:24 UTC
(In reply to shani from comment #1)
> Hi,
> Can you please share the API request for creating the connections for the
> same LUN?

Please refer https://access.redhat.com/solutions/3401771

Comment 3 shani 2021-12-01 11:38:37 UTC
Thanks for the info.
It seems that Unassign the Storage Connection (DetachStorageConnectionFromStorageDomainCommand) unassigns all storage connections, and the entries from lun_storage_server_connection_map are being removed, as you've mentioned.
But the storage connections are still available on the system (unless you've executed step 2 and removed them).
They're still available in storage_server_connections table.

I'll have a look at the detachStorageConnection command

Comment 7 sshmulev 2021-12-27 06:59:41 UTC
Version:
ovirt-engine-4.4.10.2-0.1.el8ev.noarch
vdsm-4.40.100.2-1.el8ev

steps to reproduce:
1) Create new iscsi connection from using Rest API
https://access.redhat.com/solutions/3401771
2) Delete one of the new added connection -> "6254cd8e-032f-4e8f-a9ec-5d6cac1d35b6"

expected results:
Only this connection should be removed.

Actual results:
Only this connection has been removed, although it removed this connection from two different LUNS and left another two LUNS that already had this connection before.

Before the remove:(4 connections of "6254cd8e-032f-4e8f-a9ec-5d6cac1d35b6")
engine=# Select * from lun_storage_server_connection_map;
                     lun_id                     |      storage_server_connection       
------------------------------------------------+--------------------------------------
 3600a098038304479363f4c4870454f79              | 4af82f45-c453-4207-8c3f-5e479f3ef3e9
 3600a098038304479363f4c4870454f79              | 1e87580a-8c2a-4b49-b064-23db174723ef
 3600a098038304479363f4c4870454f79              | 6254cd8e-032f-4e8f-a9ec-5d6cac1d35b6
 3600a098038304479363f4c4870454f7a              | 1e87580a-8c2a-4b49-b064-23db174723ef
 3600a098038304479363f4c4870454f7a              | 4af82f45-c453-4207-8c3f-5e479f3ef3e9
 3600a098038304479363f4c4870454f7a              | 6254cd8e-032f-4e8f-a9ec-5d6cac1d35b6
 3600a098038304479363f4c4870455030              | 4af82f45-c453-4207-8c3f-5e479f3ef3e9
 3600a098038304479363f4c4870455030              | 6254cd8e-032f-4e8f-a9ec-5d6cac1d35b6
 3600a098038304479363f4c4870455030              | 1e87580a-8c2a-4b49-b064-23db174723ef
 DUMMY_LUN_53531191-49ef-453e-863e-b584d4d60d36 | 4af82f45-c453-4207-8c3f-5e479f3ef3e9
 DUMMY_LUN_53531191-49ef-453e-863e-b584d4d60d36 | 1e87580a-8c2a-4b49-b064-23db174723ef
 DUMMY_LUN_53531191-49ef-453e-863e-b584d4d60d36 | 6254cd8e-032f-4e8f-a9ec-5d6cac1d35b6
 DUMMY_LUN_53531191-49ef-453e-863e-b584d4d60d36 | f78a119d-8fd2-4654-bda3-b3725e02f24a
(13 rows)

After the remove(2 onnections left of"6254cd8e-032f-4e8f-a9ec-5d6cac1d35b6")

engine=# Select * from lun_storage_server_connection_map;
                     lun_id                     |      storage_server_connection       
------------------------------------------------+--------------------------------------
 3600a098038304479363f4c4870454f79              | 4af82f45-c453-4207-8c3f-5e479f3ef3e9
 3600a098038304479363f4c4870454f79              | 1e87580a-8c2a-4b49-b064-23db174723ef
 3600a098038304479363f4c4870454f7a              | 1e87580a-8c2a-4b49-b064-23db174723ef
 3600a098038304479363f4c4870454f7a              | 4af82f45-c453-4207-8c3f-5e479f3ef3e9
 3600a098038304479363f4c4870454f7a              | 6254cd8e-032f-4e8f-a9ec-5d6cac1d35b6
 3600a098038304479363f4c4870455030              | 4af82f45-c453-4207-8c3f-5e479f3ef3e9
 3600a098038304479363f4c4870455030              | 6254cd8e-032f-4e8f-a9ec-5d6cac1d35b6
 3600a098038304479363f4c4870455030              | 1e87580a-8c2a-4b49-b064-23db174723ef
 DUMMY_LUN_53531191-49ef-453e-863e-b584d4d60d36 | 4af82f45-c453-4207-8c3f-5e479f3ef3e9
 DUMMY_LUN_53531191-49ef-453e-863e-b584d4d60d36 | 1e87580a-8c2a-4b49-b064-23db174723ef
 DUMMY_LUN_53531191-49ef-453e-863e-b584d4d60d36 | f78a119d-8fd2-4654-bda3-b3725e02f24a
(11 rows)

@sleviim can you confirm if this is how it's supposed to remove the connection if we already have it in other LUNS?

Comment 8 shani 2021-12-27 15:29:14 UTC
After re-checking the env, and trying to reproduce the issue with another env, here's the reason for the mysterious deletion:
The updated code first finds all LUNs for a specific storage connection, and then finds the LUNs on a specific domain (by using the volume group id).
By sending the API request DELETE /ovirt-engine/api/storagedomains/<iscsi0-id>/storageconnections/6254cd8e-032f-4e8f-a9ec-5d6cac1d35b6,
since both DUMMY_LUN_53531191-49ef-453e-863e-b584d4d60d36 and 3600a098038304479363f4c4870454f79 luns were on that domain using this connection id, they both were deleted.

Comment 10 sshmulev 2021-12-27 15:47:14 UTC
(In reply to shani from comment #8)
> After re-checking the env, and trying to reproduce the issue with another
> env, here's the reason for the mysterious deletion:
> The updated code first finds all LUNs for a specific storage connection, and
> then finds the LUNs on a specific domain (by using the volume group id).
> By sending the API request DELETE
> /ovirt-engine/api/storagedomains/<iscsi0-id>/storageconnections/6254cd8e-
> 032f-4e8f-a9ec-5d6cac1d35b6,
> since both DUMMY_LUN_53531191-49ef-453e-863e-b584d4d60d36 and
> 3600a098038304479363f4c4870454f79 luns were on that domain using this
> connection id, they both were deleted.

Thanks, yes this makes sense now because we first choose the SD id from where we want to remove the connection.
Moving the bug to verified.

Comment 11 Eli Marcus 2022-01-20 16:15:16 UTC
Hi Shani, please review this doc text: 
Previously, when unassigning a storage connection from a storage domain, for an LUN associated with another storage server connection, all storage connections for that LUN were removed (as the LUN was removed).
In this release, only the specified storage connection is removed, and the LUN is removed only if it has no storage connections.

Comment 12 shani 2022-01-20 16:28:13 UTC
LGTM.
(changed 'an LUN' to 'a LUN')

Comment 16 errata-xmlrpc 2022-02-08 10:04:44 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (RHV Manager (ovirt-engine) [ovirt-4.4.10]), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:0461


Note You need to log in before you can comment on or make changes to this bug.