Bug 1910858 - vm_ovf_generations is not cleared while detaching the storage domain causing VM import with old stale configuration
Summary: vm_ovf_generations is not cleared while detaching the storage domain causing ...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-engine
Version: 4.3.4
Hardware: All
OS: Linux
unspecified
high
Target Milestone: ovirt-4.4.6
: 4.4.6
Assignee: Pavel Bar
QA Contact: Evelina Shames
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-12-25 09:10 UTC by nijin ashok
Modified: 2024-03-25 17:41 UTC (History)
6 users (show)

Fixed In Version: ovirt-engine-4.4.6.4
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-06-01 13:22:11 UTC
oVirt Team: Storage
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2021:2179 0 None None None 2021-06-01 13:22:39 UTC
oVirt gerrit 113940 0 master MERGED core: fix VM and VM template import with old stale configuration 2021-04-14 07:21:27 UTC

Description nijin ashok 2020-12-25 09:10:39 UTC
Description of problem:

While detaching the storage domain, the VM information from vm_ovf_generations is not cleared causing VM import with old stale configuration. 

A storage domain was detached from environment A.
This storage domain was then attached to a new environment B.
Changed the configuration of the VMs from B. 
Detached the storage domain from B.
Attached it back to the old environment A.
Imported the VMs
The VMs are imported without the configuration changes done on the new environment B.

     
Version-Release number of selected component (if applicable):

vdsm-client-4.40.35.1-1.el8ev.noarch
rhvm-4.4.3.12-0.1.el8ev.noarch

How reproducible:

100%

Steps to Reproduce:

1. Create a VM in environment A with 1 GB memory and 1 CPU.

2. Increase Memory to 2 GB and CPU to 2.

3. Put the storage domain in maintenance mode. The vm_ovf_generation is updated with the new configuration.

===
# /usr/share/ovirt-engine/dbscripts/engine-psql.sh -c "select ovf_data from vm_ovf_generations where vm_guid='cbc144ab-50ae-4dc6-bd10-0f729fbc68cd'" | cut -c 2-|grep xml|xmllint --format - |egrep -B1 "Memory Size|Number of virtual CPU"

        <rasd:Caption>2 virtual cpu</rasd:Caption>
        <rasd:Description>Number of virtual CPU</rasd:Description>
--
        <rasd:Caption>2048 MB of memory</rasd:Caption>
        <rasd:Description>Memory Size</rasd:Description>
===


4. Detach the storage domain. vm_ovf_generations still contains the VMs OVF data and was not cleaned. VMs entry was removed from all other tables.

===
engine=# select vm_guid,ovf_generation from vm_ovf_generations ;
               vm_guid                | ovf_generation 
--------------------------------------+----------------
 cbc144ab-50ae-4dc6-bd10-0f729fbc68cd |              5
(1 row)
===

5. Attach the storage domain in environment B and import the VM.

6. The VM was imported with the correct configuration. Increase the memory and CPU to 4 GB and 4 CPU from environment B.

7. Detach the storage domain from environment B.

8. Attach the storage domain back to environment A.

9. Confirm that the OVF_STORE is updated with the new value.  

# dd if=/dev/1ca21662-7c9b-42f6-a001-faf56d3114d7/82b1413b-8068-4a5c-bf58-f01d5037e157 bs=1M |tar -xvf -
info.json
cbc144ab-50ae-4dc6-bd10-0f729fbc68cd.ovf
metadata.json
472aa645-c290-4584-9d4f-7272c2cf0a76.ovf

# cat cbc144ab-50ae-4dc6-bd10-0f729fbc68cd.ovf |xmllint --format - |egrep -B1 "Memory Size|Number of virtual CPU"
        <rasd:Caption>4 virtual cpu</rasd:Caption>
        <rasd:Description>Number of virtual CPU</rasd:Description>
--
        <rasd:Caption>4096 MB of memory</rasd:Caption>
        <rasd:Description>Memory Size</rasd:Description>

10. But both unregister_ovf_of_entities and vm_ovf_generations has old values.


/usr/share/ovirt-engine/dbscripts/engine-psql.sh -c "select ovf_data from unregistered_ovf_of_entities where entity_guid = 'cbc144ab-50ae-4dc6-bd10-0f729fbc68cd';" | cut -c 2-|grep xml|xmllint --format - |egrep -B1 "Memory Size|Number of virtual CPU"



        <rasd:Caption>2 virtual cpu</rasd:Caption>
        <rasd:Description>Number of virtual CPU</rasd:Description>
--
        <rasd:Caption>2048 MB of memory</rasd:Caption>
        <rasd:Description>Memory Size</rasd:Description>


/usr/share/ovirt-engine/dbscripts/engine-psql.sh -c "select ovf_data from vm_ovf_generations where vm_guid='cbc144ab-50ae-4dc6-bd10-0f729fbc68cd'" | cut -c 2-|grep xml|xmllint --format - |egrep -B1 "Memory Size|Number of virtual CPU"
        <rasd:Caption>2 virtual cpu</rasd:Caption>
        <rasd:Description>Number of virtual CPU</rasd:Description>
--
        <rasd:Caption>2048 MB of memory</rasd:Caption>
        <rasd:Description>Memory Size</rasd:Description>

11. Import the VM. The VM is imported with an old configuration.

===
engine=# select mem_size_mb,cpu_per_socket from vm_static where vm_guid='cbc144ab-50ae-4dc6-bd10-0f729fbc68cd'  ;
 mem_size_mb | cpu_per_socket 
-------------+----------------
        2048 |              1
(1 row)
===



Actual results:

vm_ovf_generations is not cleared while detaching the storage domain causing VM import with old stale configuration

Expected results:

Because of this bug, the VMs are imported with old stale configuration during Active Passive Disater Recovery.


Additional info:

Comment 1 Sandro Bonazzola 2021-04-15 07:07:53 UTC
All referenced patches are merged, can you please update on this bug status?

Comment 3 Pavel Bar 2021-04-20 12:25:33 UTC
Steps to Reproduce:
------------------
0. Environment (2 engines):
	a. Engine 1
		Data Center "DC1", Cluster, Host, Storage Domain "NFS_SD1", 1 disk.
	b. Engine 2
		Data Center "DC2", Cluster, Host.

Note: at the following steps where VM is mentioned, do *exactly the same* for VM template (can be tested in parallel, at the same flow).

1. Create a "VM1" VM on Engine 1 with 1 GB memory and 1 CPU. Click on the VM and attach a disk to it.

2. Increase VM memory to 2 GB and CPU to 2.

3. Put the Storage Domain in maintenance mode ("Storage" → "Storage Domains" → click on "NFS_SD1" → "Data Center" tab → select the "DC1" Data Center → Press the "Maintenance" button).

4. Run "engine-psql.sh" from the CLI on Engine 1 (use the correct vm_guid for VM / VM template):
pbar@engine1$ ~/dev/deploy/share/ovirt-engine/dbscripts/engine-psql.sh -c "select ovf_data from vm_ovf_generations where vm_guid='e397e809-1237-46fa-87de-fb669b1c9ec0'" | cut -c 2- | grep xml | xmllint --format - | egrep -B1 "Memory Size|Number of virtual CPU"
        <rasd:Caption>2 virtual cpu</rasd:Caption>
        <rasd:Description>Number of virtual CPU</rasd:Description>
--
        <rasd:Caption>2048 MB of memory</rasd:Caption>
        <rasd:Description>Memory Size</rasd:Description>

5. Detach the Storage Domain ("Storage" → "Storage Domains" → click on "NFS_SD1" → "Data Center" tab → select the "DC1" Data Center → Press the "Detach" button).

6. Run "engine-psql.sh" from the CLI on Engine 1:
~/dev/deploy/share/ovirt-engine/dbscripts/engine-psql.sh -c "select vm_guid,ovf_generation from vm_ovf_generations;"
               vm_guid                | ovf_generation 
--------------------------------------+----------------
 e397e809-1237-46fa-87de-fb669b1c9ec0 |              5
(1 row)

7. Import the "NFS_SD1" Storage Domain on Engine 2 ("Storage" → "Storage Domains" → "Import Domain" button → fill "Name" and "Export Path" the same as you used when initially adding "NFS_SD1" to Engine 1).

8. Attach the "NFS_SD1" Storage Domain to "DC2" Data Center on Engine 2 ("Compute" → "Data Centers" → click on "DC2" → "Attach Data" button → Choose "NFS_SD1").

9. Import the "VM1" VM to the "NFS_SD1" Storage Domain on Engine 2 ("Storage" → "Storage Domains" → click on "NFS_SD1" → "VM Import" tab → Select "VM1" to import → Press "Import" button). The "VM1" was imported with the correct configuration.

10. Increase the "VM1" memory and CPU to 4 GB and 4 CPU on Engine 2.

11. Put the "NFS_SD1" Storage Domain in maintenance mode on Engine 2 ("Storage" → "Storage Domains" → click on "NFS_SD1" → "Data Center" tab → select the "DC2" Data Center → Press the "Maintenance" button).

12. Detach the Storage Domain from Engine 2 ("Storage" → "Storage Domains" → click on "NFS_SD1" → "Data Center" tab → select the "DC2" Data Center → Press the "Detach" button).

13. Attach the "NFS_SD1" Storage Domain to "DC1" Data Center back to Engine 1 ("Compute" → "Data Centers" → click on "DC1" → "Attach Data" button → Choose "NFS_SD1"). At the end of the process, "DC1" Data Center is up and running.
!!! The bug actually happens at this step - "unregistered_ovf_of_entities" DB table still contains wrong XML (contains 2 CPUs and 2 GB memory) !!!

14. Confirm that the OVF_STORE is updated with the new value ("Storage" → "Storage Domains" → select "NFS_SD1" → "..." menu → "Update OVFs").

15. But both "unregistered_ovf_of_entities" and "vm_ovf_generations" DB tables still contain the old values.
!!! "vm_ovf_generations" DB table is OK, but the "unregistered_ovf_of_entities" DB table should have a new XML, you can test that at the old environment it's 2 CPUs and 2048 MB, while at the fixed one 4 CPUs and 4096 MB !!!
	a. "unregistered_ovf_of_entities" DB table:
~/dev/deploy/share/ovirt-engine/dbscripts/engine-psql.sh -c "select ovf_data from unregistered_ovf_of_entities where entity_guid = 'e397e809-1237-46fa-87de-fb669b1c9ec0';" | cut -c 2- | grep xml | xmllint --format - | egrep -B1 "Memory Size|Number of virtual CPU"
		<rasd:Caption>2 virtual cpu</rasd:Caption>
		<rasd:Description>Number of virtual CPU</rasd:Description>
--
		<rasd:Caption>2048 MB of memory</rasd:Caption>
		<rasd:Description>Memory Size</rasd:Description>

	b. "vm_ovf_generations" DB table:
~/dev/deploy/share/ovirt-engine/dbscripts/engine-psql.sh -c "select ovf_data from vm_ovf_generations where vm_guid='e397e809-1237-46fa-87de-fb669b1c9ec0'" | cut -c 2- | grep xml | xmllint --format - | egrep -B1 "Memory Size|Number of virtual CPU"
		<rasd:Caption>2 virtual cpu</rasd:Caption>
		<rasd:Description>Number of virtual CPU</rasd:Description>
--
		<rasd:Caption>2048 MB of memory</rasd:Caption>
		<rasd:Description>Memory Size</rasd:Description>

16. Import "VM1" to the "NFS_SD1" Storage Domain on Engine 1 ("Storage" → "Storage Domains" → click on "NFS_SD1" → "VM Import" tab → Select "VM1" to import → Press "Import" button).
!!! The VM is imported with an old configuration, the problem is a stale data at "unregistered_ovf_of_entities" DB table, see the unlucky step 13 !!!

17. Run "engine-psql.sh" from the CLI on Engine 1 (at the fixed version "mem_size_mb" should be 4096 MB):
~/dev/deploy/share/ovirt-engine/dbscripts/engine-psql.sh -c "select vm_name, vm_guid, mem_size_mb, cpu_per_socket, min_allocated_mem, max_memory_size_mb from vm_static where vm_guid='e397e809-1237-46fa-87de-fb669b1c9ec0';"

 vm_name |               vm_guid                | mem_size_mb | cpu_per_socket | min_allocated_mem | max_memory_size_mb 
---------+--------------------------------------+-------------+----------------+-------------------+--------------------
 VM1     | e397e809-1237-46fa-87de-fb669b1c9ec0 |        2048 |              1 |              2048 |               8192
(1 row)

Comment 4 Evelina Shames 2021-04-20 15:31:44 UTC
(In reply to Pavel Bar from comment #3)
> Steps to Reproduce:
> ------------------
> 0. Environment (2 engines):
> 	a. Engine 1
> 		Data Center "DC1", Cluster, Host, Storage Domain "NFS_SD1", 1 disk.
> 	b. Engine 2
> 		Data Center "DC2", Cluster, Host.
> 
> Note: at the following steps where VM is mentioned, do *exactly the same*
> for VM template (can be tested in parallel, at the same flow).
> 
> 1. Create a "VM1" VM on Engine 1 with 1 GB memory and 1 CPU. Click on the VM
> and attach a disk to it.
> 
> 2. Increase VM memory to 2 GB and CPU to 2.
> 
> 3. Put the Storage Domain in maintenance mode ("Storage" → "Storage Domains"
> → click on "NFS_SD1" → "Data Center" tab → select the "DC1" Data Center →
> Press the "Maintenance" button).
> 
> 4. Run "engine-psql.sh" from the CLI on Engine 1 (use the correct vm_guid
> for VM / VM template):
> pbar@engine1$ ~/dev/deploy/share/ovirt-engine/dbscripts/engine-psql.sh -c
> "select ovf_data from vm_ovf_generations where
> vm_guid='e397e809-1237-46fa-87de-fb669b1c9ec0'" | cut -c 2- | grep xml |
> xmllint --format - | egrep -B1 "Memory Size|Number of virtual CPU"
>         <rasd:Caption>2 virtual cpu</rasd:Caption>
>         <rasd:Description>Number of virtual CPU</rasd:Description>
> --
>         <rasd:Caption>2048 MB of memory</rasd:Caption>
>         <rasd:Description>Memory Size</rasd:Description>
> 
> 5. Detach the Storage Domain ("Storage" → "Storage Domains" → click on
> "NFS_SD1" → "Data Center" tab → select the "DC1" Data Center → Press the
> "Detach" button).
> 
> 6. Run "engine-psql.sh" from the CLI on Engine 1:
> ~/dev/deploy/share/ovirt-engine/dbscripts/engine-psql.sh -c "select
> vm_guid,ovf_generation from vm_ovf_generations;"
>                vm_guid                | ovf_generation 
> --------------------------------------+----------------
>  e397e809-1237-46fa-87de-fb669b1c9ec0 |              5
> (1 row)
> 
> 7. Import the "NFS_SD1" Storage Domain on Engine 2 ("Storage" → "Storage
> Domains" → "Import Domain" button → fill "Name" and "Export Path" the same
> as you used when initially adding "NFS_SD1" to Engine 1).
> 
> 8. Attach the "NFS_SD1" Storage Domain to "DC2" Data Center on Engine 2
> ("Compute" → "Data Centers" → click on "DC2" → "Attach Data" button → Choose
> "NFS_SD1").
> 
> 9. Import the "VM1" VM to the "NFS_SD1" Storage Domain on Engine 2
> ("Storage" → "Storage Domains" → click on "NFS_SD1" → "VM Import" tab →
> Select "VM1" to import → Press "Import" button). The "VM1" was imported with
> the correct configuration.
> 
> 10. Increase the "VM1" memory and CPU to 4 GB and 4 CPU on Engine 2.
> 
> 11. Put the "NFS_SD1" Storage Domain in maintenance mode on Engine 2
> ("Storage" → "Storage Domains" → click on "NFS_SD1" → "Data Center" tab →
> select the "DC2" Data Center → Press the "Maintenance" button).
> 
> 12. Detach the Storage Domain from Engine 2 ("Storage" → "Storage Domains" →
> click on "NFS_SD1" → "Data Center" tab → select the "DC2" Data Center →
> Press the "Detach" button).
> 
> 13. Attach the "NFS_SD1" Storage Domain to "DC1" Data Center back to Engine
> 1 ("Compute" → "Data Centers" → click on "DC1" → "Attach Data" button →
> Choose "NFS_SD1"). At the end of the process, "DC1" Data Center is up and
> running.
> !!! The bug actually happens at this step - "unregistered_ovf_of_entities"
> DB table still contains wrong XML (contains 2 CPUs and 2 GB memory) !!!
> 
> 14. Confirm that the OVF_STORE is updated with the new value ("Storage" →
> "Storage Domains" → select "NFS_SD1" → "..." menu → "Update OVFs").
> 
> 15. But both "unregistered_ovf_of_entities" and "vm_ovf_generations" DB
> tables still contain the old values.
> !!! "vm_ovf_generations" DB table is OK, but the
> "unregistered_ovf_of_entities" DB table should have a new XML, you can test
> that at the old environment it's 2 CPUs and 2048 MB, while at the fixed one
> 4 CPUs and 4096 MB !!!
> 	a. "unregistered_ovf_of_entities" DB table:
> ~/dev/deploy/share/ovirt-engine/dbscripts/engine-psql.sh -c "select ovf_data
> from unregistered_ovf_of_entities where entity_guid =
> 'e397e809-1237-46fa-87de-fb669b1c9ec0';" | cut -c 2- | grep xml | xmllint
> --format - | egrep -B1 "Memory Size|Number of virtual CPU"
> 		<rasd:Caption>2 virtual cpu</rasd:Caption>
> 		<rasd:Description>Number of virtual CPU</rasd:Description>
> --
> 		<rasd:Caption>2048 MB of memory</rasd:Caption>
> 		<rasd:Description>Memory Size</rasd:Description>
> 
> 	b. "vm_ovf_generations" DB table:
> ~/dev/deploy/share/ovirt-engine/dbscripts/engine-psql.sh -c "select ovf_data
> from vm_ovf_generations where
> vm_guid='e397e809-1237-46fa-87de-fb669b1c9ec0'" | cut -c 2- | grep xml |
> xmllint --format - | egrep -B1 "Memory Size|Number of virtual CPU"
> 		<rasd:Caption>2 virtual cpu</rasd:Caption>
> 		<rasd:Description>Number of virtual CPU</rasd:Description>
> --
> 		<rasd:Caption>2048 MB of memory</rasd:Caption>
> 		<rasd:Description>Memory Size</rasd:Description>
> 
> 16. Import "VM1" to the "NFS_SD1" Storage Domain on Engine 1 ("Storage" →
> "Storage Domains" → click on "NFS_SD1" → "VM Import" tab → Select "VM1" to
> import → Press "Import" button).
> !!! The VM is imported with an old configuration, the problem is a stale
> data at "unregistered_ovf_of_entities" DB table, see the unlucky step 13 !!!
> 
> 17. Run "engine-psql.sh" from the CLI on Engine 1 (at the fixed version
> "mem_size_mb" should be 4096 MB):
> ~/dev/deploy/share/ovirt-engine/dbscripts/engine-psql.sh -c "select vm_name,
> vm_guid, mem_size_mb, cpu_per_socket, min_allocated_mem, max_memory_size_mb
> from vm_static where vm_guid='e397e809-1237-46fa-87de-fb669b1c9ec0';"
> 
>  vm_name |               vm_guid                | mem_size_mb |
> cpu_per_socket | min_allocated_mem | max_memory_size_mb 
> ---------+--------------------------------------+-------------+--------------
> --+-------------------+--------------------
>  VM1     | e397e809-1237-46fa-87de-fb669b1c9ec0 |        2048 |             
> 1 |              2048 |               8192
> (1 row)

Reproduced on ovirt-engine-4.4.6.3-0.8.el8ev.noarch
Verified with the above steps on ovirt-engine-4.4.6.5-447.gd80dda7.9.el8ev.noarch.

Moving to 'Verified'

Comment 8 errata-xmlrpc 2021-06-01 13:22:11 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: RHV Manager security update (ovirt-engine) [ovirt-4.4.6]), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:2179

Comment 9 meital avital 2022-08-08 20:11:08 UTC
Due to QE capacity, we are not going to cover this issue in our automation


Note You need to log in before you can comment on or make changes to this bug.