Created attachment 754868 [details] vdsm + engine logs Description of problem: After creating a new DC with 1 host and 1 storage domain, and 1 additional domain which is not attached, all storages were destroyed (directly on the storage server). To clean the RHEVM the host was moved to maintenance, DC was force-removed and additional domain was destroyed. However the domain remained in the storage_domain_static table. Now it is not possible to add another domain to the RHEVM with the same name as that one. Queries to the DB: # psql -d engine -U postgres -c "select * from storage_domain_static;" id | storage | storage_name | storage_domain_type | storage_type | storage_domain_format_type | _create_date | _update_dat --------------------------------------+----------------------------------------+--------------+---------------------+--------------+----------------------------+-------------------------------+-------------------- feb6aa0e-4302-42b9-86a7-84d902fea07e | AxHc8t-CRgE-FKft-qrSl-PG8O-Svs5-9m5dL3 | iscsi_2 | 1 | 3 | 3 | 2013-05-26 14:39:33.223723+03 | 7d3d2d42-270f-45dc-b6e4-20fc1f389e37 | iDGExr-4Kch-TKl0-GzEV-BxgG-bvsP-rfK6Ly | iscsi_0 | 0 | 3 | 3 | 2013-05-30 14:25:00.450573+03 | 2013-05-30 14:25:04 f1103bf0-65cb-4d7a-a7cd-1656ea1252cd | RYPDXv-AfXt-7Dyz-SA4d-nVwI-xFOe-I0FjU6 | iscsi_1 | 1 | 3 | 0 | 2013-05-30 14:26:22.152056+03 | 2013-05-30 14:26:22 # # psql -d engine -U postgres -c "select * from storage_domains;" id | storage | storage_name | storage_pool_id | available_disk_size | used_disk_size | commited_disk_size | status | storage_p --------------------------------------+----------------------------------------+--------------+--------------------------------------+---------------------+----------------+--------------------+--------+---------- 7d3d2d42-270f-45dc-b6e4-20fc1f389e37 | iDGExr-4Kch-TKl0-GzEV-BxgG-bvsP-rfK6Ly | iscsi_0 | 7cc34b9a-ea43-49ba-a73d-f667bb93f664 | | | 0 | 4 | TestDC f1103bf0-65cb-4d7a-a7cd-1656ea1252cd | RYPDXv-AfXt-7Dyz-SA4d-nVwI-xFOe-I0FjU6 | iscsi_1 | | 10 | 4 | 0 | | (2 rows) Version-Release number of selected component (if applicable): rhevm-3.2.0-11.28.el6ev.noarch How reproducible: ? Steps to Reproduce: 1. Create datacenter, cluster and add 1 host 2. Create storage domain and attach it to DC 3. Create additonal storage domain (not attached to DC) 4. Destroy storage (directly on the storage server) 5. When host is in non-operational move it to maintenance and force-remove DC 6. Destroy remaining storage domain Actual results: After repeating this procedure several times I was suddenly unable to re-add a storage domain with the same name as the additional domain. Checking the DB showed that the storage domain entry remained in the storage_domain_static table, but not in the storage_domains table Expected results: destroying a domain should remove it completely from the DB Additional info:
What did you mean by storage_domains table? there is no such table in the db... storage_domains is a view. in which specific table did you see the records remained? Also, you wrote "After repeating this procedure several times" -is this consistent? How many times the procedure needs to be repeated for reproduction?
Some information and the cause for the problem in the bug: relevant storage domain id is feb6aa0e-4302-42b9-86a7-84d902fea07e 1. ReconstructMasterDomainCommand starts, compensates storage_domain_static. 2. ForceRemoveStorageDomainCommand starts. It removes the storage domain from the db. 3. ReconstructMasterDomainCommand fails. It triggers the compensation of the storage domain. Since it was removed from db by step 2 (ForceRemoveStorageDomainCommand), the compensation gets storage domain inserted again to storage_domain_static table.
Liron, was this solved as part of your on reconstruct?
Allon, no. I provided patches to solve the issue, added to the tracker.
All instances of the storage domains were removed from the database after performing the above scenario ONCE. Is this sufficient to move this defect to Verified? Gadi?
(In reply to Kevin Alon Goldblatt from comment #7) > All instances of the storage domains were removed from the database after > performing the above scenario ONCE. Is this sufficient to move this defect > to Verified? Gadi? If it worked, then I guess so
verified on oVirt beta.2
RHEV-M 3.5.0 has been released, closing this bug.