Bug 969026 - [engine] Storage Domain that was destroyed from GUI did not get completely removed from DB
[engine] Storage Domain that was destroyed from GUI did not get completely re...
Status: CLOSED CURRENTRELEASE
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-engine (Show other bugs)
3.2.0
All Linux
unspecified Severity high
: ---
: 3.5.0
Assigned To: Liron Aravot
Ori Gofen
storage
:
Depends On:
Blocks: rhev3.5beta 1156165
  Show dependency treegraph
 
Reported: 2013-05-30 09:58 EDT by Gadi Ickowicz
Modified: 2016-05-25 21:48 EDT (History)
12 users (show)

See Also:
Fixed In Version: ovirt-3.5.0-alpha2
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2015-02-16 14:10:13 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: Storage
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
amureini: Triaged+


Attachments (Terms of Use)
vdsm + engine logs (1.15 MB, application/gzip)
2013-05-30 09:58 EDT, Gadi Ickowicz
no flags Details


External Trackers
Tracker ID Priority Status Summary Last Updated
oVirt gerrit 25231 None None None Never
oVirt gerrit 25232 master MERGED core: proceedStorageDomainTreatmentByDomainType - proper compensation Never

  None (edit)
Description Gadi Ickowicz 2013-05-30 09:58:51 EDT
Created attachment 754868 [details]
vdsm + engine logs

Description of problem:
After creating a new DC with 1 host and 1 storage domain, and 1 additional domain which is not attached, all storages were destroyed (directly on the storage server). To clean the RHEVM the host was moved to maintenance, DC was force-removed and additional domain was destroyed. However the domain remained in the storage_domain_static table. Now it is not possible to add another domain to the RHEVM with the same name as that one.


Queries to the DB:
# psql -d engine -U postgres -c "select * from storage_domain_static;"
                 id                  |                storage                 | storage_name | storage_domain_type | storage_type | storage_domain_format_type |         _create_date          |         _update_dat
--------------------------------------+----------------------------------------+--------------+---------------------+--------------+----------------------------+-------------------------------+--------------------
 feb6aa0e-4302-42b9-86a7-84d902fea07e | AxHc8t-CRgE-FKft-qrSl-PG8O-Svs5-9m5dL3 | iscsi_2      |                   1 |            3 | 3                          | 2013-05-26 14:39:33.223723+03 |                    
 7d3d2d42-270f-45dc-b6e4-20fc1f389e37 | iDGExr-4Kch-TKl0-GzEV-BxgG-bvsP-rfK6Ly | iscsi_0      |                   0 |            3 | 3                          | 2013-05-30 14:25:00.450573+03 | 2013-05-30 14:25:04
 f1103bf0-65cb-4d7a-a7cd-1656ea1252cd | RYPDXv-AfXt-7Dyz-SA4d-nVwI-xFOe-I0FjU6 | iscsi_1      |                   1 |            3 | 0                          | 2013-05-30 14:26:22.152056+03 | 2013-05-30 14:26:22



# # psql -d engine -U postgres -c "select * from storage_domains;"
                  id                  |                storage                 | storage_name |           storage_pool_id            | available_disk_size | used_disk_size | commited_disk_size | status | storage_p
--------------------------------------+----------------------------------------+--------------+--------------------------------------+---------------------+----------------+--------------------+--------+----------
 7d3d2d42-270f-45dc-b6e4-20fc1f389e37 | iDGExr-4Kch-TKl0-GzEV-BxgG-bvsP-rfK6Ly | iscsi_0      | 7cc34b9a-ea43-49ba-a73d-f667bb93f664 |                     |                |                  0 |      4 | TestDC   
 f1103bf0-65cb-4d7a-a7cd-1656ea1252cd | RYPDXv-AfXt-7Dyz-SA4d-nVwI-xFOe-I0FjU6 | iscsi_1      |                                      |                  10 |              4 |                  0 |        |          
(2 rows)



Version-Release number of selected component (if applicable):
rhevm-3.2.0-11.28.el6ev.noarch

How reproducible:
?

Steps to Reproduce:
1. Create datacenter, cluster and add 1 host
2. Create storage domain and attach it to DC
3. Create additonal storage domain (not attached to DC)
4. Destroy storage (directly on the storage server)
5. When host is in non-operational move it to maintenance and force-remove DC
6. Destroy remaining storage domain

Actual results:
After repeating this procedure several times I was suddenly unable to re-add a storage domain with the same name as the additional domain. Checking the DB showed that the storage domain entry remained in the storage_domain_static table, but not in the storage_domains table

Expected results:
destroying a domain should remove it completely from the DB


Additional info:
Comment 1 Alissa 2013-07-09 08:40:03 EDT
What did you mean by storage_domains table? there is no such table in the db... storage_domains is a view. in which specific table did you see the records remained?

Also, you wrote "After repeating this procedure several times" -is this consistent? How many times the procedure needs to be repeated for reproduction?
Comment 3 Alissa 2013-08-19 10:47:22 EDT
Some information and the cause for the problem in the bug:
relevant storage domain id is feb6aa0e-4302-42b9-86a7-84d902fea07e
1. ReconstructMasterDomainCommand starts, compensates storage_domain_static. 
2. ForceRemoveStorageDomainCommand starts.
It removes the storage domain from the db.
3. ReconstructMasterDomainCommand fails.
It triggers the compensation of the storage domain. Since it was removed from db by step 2 (ForceRemoveStorageDomainCommand), the compensation gets storage domain inserted  again to storage_domain_static table.
Comment 5 Allon Mureinik 2014-02-16 04:09:22 EST
Liron, was this solved as part of your on reconstruct?
Comment 6 Liron Aravot 2014-03-02 08:51:29 EST
Allon, no.
I provided patches to solve the issue, added to the tracker.
Comment 7 Kevin Alon Goldblatt 2014-07-07 03:19:48 EDT
All instances of the storage domains were removed from the database after performing the above scenario ONCE. Is this sufficient to move this defect to Verified? Gadi?
Comment 8 Gadi Ickowicz 2014-07-08 04:16:41 EDT
(In reply to Kevin Alon Goldblatt from comment #7)
> All instances of the storage domains were removed from the database after
> performing the above scenario ONCE. Is this sufficient to move this defect
> to Verified? Gadi?

If it worked, then I guess so
Comment 9 Ori Gofen 2014-07-29 03:51:16 EDT
verified on oVirt beta.2
Comment 11 Allon Mureinik 2015-02-16 14:10:13 EST
RHEV-M 3.5.0 has been released, closing this bug.

Note You need to log in before you can comment on or make changes to this bug.