Bug 1543218 - Engine allows importing HE specific volumes and then deleting them.
Summary: Engine allows importing HE specific volumes and then deleting them.
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-engine
Version: 4.1.8
Hardware: x86_64
OS: Linux
unspecified
high
Target Milestone: ovirt-4.2.2
: ---
Assignee: Simone Tiraboschi
QA Contact: Nikolai Sednev
URL:
Whiteboard:
Depends On:
Blocks: 1458709 1520566
TreeView+ depends on / blocked
 
Reported: 2018-02-08 01:28 UTC by Germano Veit Michel
Modified: 2021-06-10 14:33 UTC (History)
9 users (show)

Fixed In Version: ovirt-hosted-engine-setup-2.2.12
Doc Type: Bug Fix
Doc Text:
The descriptions of the hosted engine disks have been improved in the Manager to reduce the risk of their being accidentally deleted.
Clone Of:
Environment:
Last Closed: 2018-05-15 17:48:28 UTC
oVirt Team: Integration
Target Upstream Version:
lsvaty: testing_plan_complete-


Attachments (Terms of Use)
Screenshot - no disk alias for HE disks (58.97 KB, image/png)
2018-02-09 18:27 UTC, Marina Kalinin
no flags Details
Screenshot from 2018-03-21 10-14-34.png (101.04 KB, image/png)
2018-03-21 08:17 UTC, Nikolai Sednev
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Knowledge Base (Solution) 4731761 0 None None None 2020-01-14 01:08:17 UTC
Red Hat Product Errata RHEA-2018:1488 0 None None None 2018-05-15 17:49:44 UTC
oVirt gerrit 88040 0 master MERGED ansible: disk descriptions and HE VM protection 2020-09-24 09:03:23 UTC
oVirt gerrit 88180 0 ovirt-hosted-engine-setup-2.2 MERGED ansible: disk descriptions and HE VM protection 2020-09-24 09:03:23 UTC

Description Germano Veit Michel 2018-02-08 01:28:45 UTC
Description of problem:

User can completely destroy a HostedEngine install while thinking he is just cleaning up unused disks.

In a fresh Hosted-Engine 4.1 install, go to:
Administration Portal -> Storage -> hosted_storage -> Disk Import.

There are 3 disks showing there, all used by the HostedEngine HA mechanism.
- conf_image
- metadata_image
- lockspace_image

There is no description of what they are, all is blank (they should not even be listed there). User thinks it might be some leftover from previous upgrade/deploy and can import and delete them, destroying the environment.

See:
                  id                  |  storage_name  
--------------------------------------+----------------
 1bfd3c51-74f6-45ac-83e7-392c57eb3759 | hosted_storage

engine=# select disk_id,size from unregistered_disks where storage_domain_id = '1bfd3c51-74f6-45ac-83e7-392c57eb3759';
               disk_id                |    size     
--------------------------------------+-------------
 8a822870-9629-4215-9b5d-cbd586a8f9a7 |   134217728   <-- metadata_image
 863ca357-8f5e-4863-ac5a-db8c1cdd8a52 |  1073741824   <-- lockspace_image
 fc2edf2c-4f16-426f-bb00-c9562b258be2 | 62277025792   <-- HE Disk 
 9a5c545d-c0e8-4740-9bb1-22d7c3cb6e27 |   134217728   <-- conf_image

HE Disk is in base_disks/images. The other 3 are not and show up for the user to import them.

Then the engine allows the user to import them and do whatever he wants, including removing.

Version-Release number of selected component (if applicable):
4.1.8.2-0.1.el7

How reproducible:
100%

Steps to Reproduce:
1. Deploy HE 
2. Administration Portal -> Storage -> hosted_storage -> Disk Import
3. Import the Disks
4. delete/change/move them
5. environment is destroyed, need to follow disaster recovery.

Actual results:
Hosted-Engine environment destroyed. The only thin

Expected results:
Do not allow user to shoot himself in the foot.

Additional Information:

Importing and deleting the 3 disks above:

1. Just to identify which is which:
2018-02-08 12:19:51,932+11 WARN  [org.ovirt.engine.core.bll.storage.disk.image.GetUnregisteredDiskQuery] (default task-25) [4286a967-835c-4f81-8b2d-e83e67a6bbf6] Could not parse the description (hosted-engine.metadata) of disk ID '8a822870-9629-4215-9b5d-cbd586a8f9a7'. The description is expected to be in JSON format.

2018-02-08 12:19:55,685+11 WARN  [org.ovirt.engine.core.bll.storage.disk.image.GetUnregisteredDiskQuery] (default task-21) [86a606d6-6f58-4169-b71d-773a943bc047] Could not parse the description (hosted-engine.lockspace) of disk ID '9a5c545d-c0e8-4740-9bb1-22d7c3cb6e27'. The description is expected to be in JSON format.

2018-02-08 12:19:56,948+11 WARN  [org.ovirt.engine.core.bll.storage.disk.image.GetUnregisteredDiskQuery] (default task-15) [07b6c4b4-f677-4c1d-bad2-ea09023cf7f1] Could not parse the description (HostedEngineConfigurationImage) of disk ID '863ca357-8f5e-4863-ac5a-db8c1cdd8a52'. The description is expected to be in JSON format.

2. All imported
2018-02-08 12:19:53,163+11 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-14) [4286a967-835c-4f81-8b2d-e83e67a6bbf6] EVENT_ID: USER_REGISTER_DISK_FINISHED_SUCCESS(378), Correlation ID: 4286a967-835c-4f81-8b2d-e83e67a6bbf6, Job ID: eb00e197-89d0-4f9d-b627-a5a713cf2e91, Call Stack: null, Custom ID: null, Custom Event ID: -1, Message: Disk 'RegisteredDisk_2018-02-08_12-19-52' has been successfully registered as a floating disk.

2018-02-08 12:19:56,145+11 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-7) [86a606d6-6f58-4169-b71d-773a943bc047] EVENT_ID: USER_REGISTER_DISK_FINISHED_SUCCESS(378), Correlation ID: 86a606d6-6f58-4169-b71d-773a943bc047, Job ID: 73540cfd-62c3-4c8e-be41-57efc6c61be9, Call Stack: null, Custom ID: null, Custom Event ID: -1, Message: Disk 'RegisteredDisk_2018-02-08_12-19-55' has been successfully registered as a floating disk.

2018-02-08 12:20:00,972+11 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-12) [07b6c4b4-f677-4c1d-bad2-ea09023cf7f1] EVENT_ID: USER_REGISTER_DISK_FINISHED_SUCCESS(378), Correlation ID: 07b6c4b4-f677-4c1d-bad2-ea09023cf7f1, Job ID: 63973046-15d3-4e80-b596-daffb4c9cebd, Call Stack: null, Custom ID: null, Custom Event ID: -1, Message: Disk 'RegisteredDisk_2018-02-08_12-20-00' has been successfully registered as a floating disk.

3. All deleted.
2018-02-08 12:20:13,137+11 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-48) [131b2e7e-5684-43c6-8780-607b5a8d983d] EVENT_ID: USER_FINISHED_REMOVE_DISK(2,014), Correlation ID: 131b2e7e-5684-43c6-8780-607b5a8d983d, Job ID: 47f32f76-0b58-4cea-acca-f05a074a5072, Call Stack: null, Custom ID: null, Custom Event ID: -1, Message: Disk RegisteredDisk_2018-02-08_12-19-55 was successfully removed from domain hosted_storage (User admin@internal-authz).

2018-02-08 12:20:19,361+11 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-48) [131b2e7e-5684-43c6-8780-607b5a8d983d] EVENT_ID: USER_FINISHED_REMOVE_DISK(2,014), Correlation ID: 131b2e7e-5684-43c6-8780-607b5a8d983d, Job ID: 5ecb85b7-e549-4085-923e-c46ac6a38283, Call Stack: null, Custom ID: null, Custom Event ID: -1, Message: Disk RegisteredDisk_2018-02-08_12-20-00 was successfully removed from domain hosted_storage (User admin@internal-authz).

2018-02-08 12:20:23,387+11 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-48) [131b2e7e-5684-43c6-8780-607b5a8d983d] EVENT_ID: USER_FINISHED_REMOVE_DISK(2,014), Correlation ID: 131b2e7e-5684-43c6-8780-607b5a8d983d, Job ID: f66fdb60-b338-46f0-a6e4-e367dfe3c263, Call Stack: null, Custom ID: null, Custom Event ID: -1, Message: Disk RegisteredDisk_2018-02-08_12-19-52 was successfully removed from domain hosted_storage (User admin@internal-authz).

Comment 1 Yaniv Kaul 2018-02-08 10:14:25 UTC
For start, let's add description to the disks...

Comment 2 Simone Tiraboschi 2018-02-09 16:57:40 UTC
The description is there but it's just a plain string and not a valid json so I think that the engine simply ignores it.

Comment 3 Marina Kalinin 2018-02-09 18:27:00 UTC
Created attachment 1393940 [details]
Screenshot - no disk alias for HE disks

Comment 4 Simone Tiraboschi 2018-02-09 23:25:27 UTC
I think that the description issue comes from here:

 [root@c74he20180108h1 ~]# source /etc/ovirt-hosted-engine/hosted-engine.conf
 [root@c74he20180108h1 ~]# vdsm-client Volume getInfo storagepoolID=$spUUID storagedomainID=$sdUUID imageID=$lockspace_image_UUID volumeID=$lockspace_volume_UUID
 {
     "status": "OK", 
     "lease": {
         "owners": [], 
         "version": null
     }, 
     "domain": "997acd46-4433-4154-b6ea-99ff26da6fe9", 
     "capacity": "134217728", 
     "voltype": "LEAF", 
     "description": "hosted-engine.lockspace", 
     "parent": "00000000-0000-0000-0000-000000000000", 
     "format": "RAW", 
     "generation": 0, 
     "image": "6ed0316a-b777-4679-96a0-835002440109", 
     "uuid": "68b1e121-e47c-41b3-bed9-95fc4440a86b", 
     "disktype": "2", 
     "legality": "LEGAL", 
     "mtime": "0", 
     "apparentsize": "134217728", 
     "truesize": "134217728", 
     "type": "PREALLOCATED", 
     "children": [], 
     "pool": "", 
     "ctime": "1518191441"
 }
 [root@c74he20180108h1 ~]# vdsm-client Volume getInfo storagepoolID=$spUUID storagedomainID=$sdUUID imageID=c78da68d-fc54-48a1-9e99-bcd1f9712b21 volumeID=37e76796-21f3-4531-85a1-97e213fb534f
 {
     "status": "OK", 
     "lease": {
         "owners": [], 
         "version": null
     }, 
     "domain": "997acd46-4433-4154-b6ea-99ff26da6fe9", 
     "capacity": "134217728", 
     "voltype": "LEAF", 
     "description": "{\"Updated\":true,\"Size\":20480,\"Last Updated\":\"Fri Feb 09 19:49:35 CET 2018\",\"Storage Domains\":[{\"uuid\":\"997acd46-4433-4154-b6ea-99ff26da6fe9\"}],\"Disk Description\":\"OVF_STORE\"}", 
     "parent": "00000000-0000-0000-0000-000000000000", 
     "format": "RAW", 
     "generation": 0, 
     "image": "c78da68d-fc54-48a1-9e99-bcd1f9712b21", 
     "uuid": "37e76796-21f3-4531-85a1-97e213fb534f", 
     "disktype": "OVFS", 
     "legality": "LEGAL", 
     "mtime": "0", 
     "apparentsize": "134217728", 
     "truesize": "134217728", 
     "type": "PREALLOCATED", 
     "children": [], 
     "pool": "", 
     "ctime": "1518195252"
 }

On hosted-engine volumes we simply set (from 3.3 I think) the volume description on VDSM side as a plain text label while the volume description on engine generated disks (the OVF_STORE one for example) is a json where the disk description, as shown by the engine, is just "Disk Description" field inside that json. I suppose that the engine simply ignores the plain text volume description since is not a valid json.
On node-zero all the disks are created by the engine and so the description is fine by design.

We need to find a way to fix the disk descriptions on system deployed in the past and a way to prevent the engine from deleting them (a new special disktype?)

Comment 9 Simone Tiraboschi 2018-03-01 16:44:12 UTC
I built ovirt-hosted-engine-setup-2.2.12 half an hour ago

Comment 10 RHV bug bot 2018-03-16 15:01:29 UTC
INFO: Bug status (ON_QA) wasn't changed but the folowing should be fixed:

[No relevant external trackers attached]

For more info please contact: rhv-devops@redhat.com

Comment 11 Nikolai Sednev 2018-03-21 08:16:14 UTC
Works for me on these components on hosts:
ovirt-hosted-engine-ha-2.2.7-1.el7ev.noarch
ovirt-hosted-engine-setup-2.2.13-1.el7ev.noarch
rhvm-appliance-4.2-20180202.0.el7.noarch
Linux 3.10.0-861.el7.x86_64 #1 SMP Wed Mar 14 10:21:01 EDT 2018 x86_64 x86_64 x86_64 GNU/Linux
Red Hat Enterprise Linux Server release 7.5 (Maipo)

I saw proper description for all SHE disks, please see the attachment.
Moving to verified.

Comment 12 Nikolai Sednev 2018-03-21 08:17:06 UTC
Created attachment 1411027 [details]
Screenshot from 2018-03-21 10-14-34.png

Comment 16 errata-xmlrpc 2018-05-15 17:48:28 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2018:1488

Comment 17 Franta Kust 2019-05-16 13:03:10 UTC
BZ<2>Jira Resync

Comment 18 Daniel Gur 2019-08-28 13:11:36 UTC
sync2jira

Comment 19 Daniel Gur 2019-08-28 13:15:48 UTC
sync2jira


Note You need to log in before you can comment on or make changes to this bug.