Bug 1910858
| Summary: | vm_ovf_generations is not cleared while detaching the storage domain causing VM import with old stale configuration | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Virtualization Manager | Reporter: | nijin ashok <nashok> |
| Component: | ovirt-engine | Assignee: | Pavel Bar <pbar> |
| Status: | CLOSED ERRATA | QA Contact: | Evelina Shames <eshames> |
| Severity: | high | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 4.3.4 | CC: | dfodor, eshenitz, mavital, pbar, sfishbai, tnisan |
| Target Milestone: | ovirt-4.4.6 | Keywords: | ZStream |
| Target Release: | 4.4.6 | ||
| Hardware: | All | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | ovirt-engine-4.4.6.4 | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2021-06-01 13:22:11 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | Storage | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
All referenced patches are merged, can you please update on this bug status? Steps to Reproduce:
------------------
0. Environment (2 engines):
a. Engine 1
Data Center "DC1", Cluster, Host, Storage Domain "NFS_SD1", 1 disk.
b. Engine 2
Data Center "DC2", Cluster, Host.
Note: at the following steps where VM is mentioned, do *exactly the same* for VM template (can be tested in parallel, at the same flow).
1. Create a "VM1" VM on Engine 1 with 1 GB memory and 1 CPU. Click on the VM and attach a disk to it.
2. Increase VM memory to 2 GB and CPU to 2.
3. Put the Storage Domain in maintenance mode ("Storage" → "Storage Domains" → click on "NFS_SD1" → "Data Center" tab → select the "DC1" Data Center → Press the "Maintenance" button).
4. Run "engine-psql.sh" from the CLI on Engine 1 (use the correct vm_guid for VM / VM template):
pbar@engine1$ ~/dev/deploy/share/ovirt-engine/dbscripts/engine-psql.sh -c "select ovf_data from vm_ovf_generations where vm_guid='e397e809-1237-46fa-87de-fb669b1c9ec0'" | cut -c 2- | grep xml | xmllint --format - | egrep -B1 "Memory Size|Number of virtual CPU"
<rasd:Caption>2 virtual cpu</rasd:Caption>
<rasd:Description>Number of virtual CPU</rasd:Description>
--
<rasd:Caption>2048 MB of memory</rasd:Caption>
<rasd:Description>Memory Size</rasd:Description>
5. Detach the Storage Domain ("Storage" → "Storage Domains" → click on "NFS_SD1" → "Data Center" tab → select the "DC1" Data Center → Press the "Detach" button).
6. Run "engine-psql.sh" from the CLI on Engine 1:
~/dev/deploy/share/ovirt-engine/dbscripts/engine-psql.sh -c "select vm_guid,ovf_generation from vm_ovf_generations;"
vm_guid | ovf_generation
--------------------------------------+----------------
e397e809-1237-46fa-87de-fb669b1c9ec0 | 5
(1 row)
7. Import the "NFS_SD1" Storage Domain on Engine 2 ("Storage" → "Storage Domains" → "Import Domain" button → fill "Name" and "Export Path" the same as you used when initially adding "NFS_SD1" to Engine 1).
8. Attach the "NFS_SD1" Storage Domain to "DC2" Data Center on Engine 2 ("Compute" → "Data Centers" → click on "DC2" → "Attach Data" button → Choose "NFS_SD1").
9. Import the "VM1" VM to the "NFS_SD1" Storage Domain on Engine 2 ("Storage" → "Storage Domains" → click on "NFS_SD1" → "VM Import" tab → Select "VM1" to import → Press "Import" button). The "VM1" was imported with the correct configuration.
10. Increase the "VM1" memory and CPU to 4 GB and 4 CPU on Engine 2.
11. Put the "NFS_SD1" Storage Domain in maintenance mode on Engine 2 ("Storage" → "Storage Domains" → click on "NFS_SD1" → "Data Center" tab → select the "DC2" Data Center → Press the "Maintenance" button).
12. Detach the Storage Domain from Engine 2 ("Storage" → "Storage Domains" → click on "NFS_SD1" → "Data Center" tab → select the "DC2" Data Center → Press the "Detach" button).
13. Attach the "NFS_SD1" Storage Domain to "DC1" Data Center back to Engine 1 ("Compute" → "Data Centers" → click on "DC1" → "Attach Data" button → Choose "NFS_SD1"). At the end of the process, "DC1" Data Center is up and running.
!!! The bug actually happens at this step - "unregistered_ovf_of_entities" DB table still contains wrong XML (contains 2 CPUs and 2 GB memory) !!!
14. Confirm that the OVF_STORE is updated with the new value ("Storage" → "Storage Domains" → select "NFS_SD1" → "..." menu → "Update OVFs").
15. But both "unregistered_ovf_of_entities" and "vm_ovf_generations" DB tables still contain the old values.
!!! "vm_ovf_generations" DB table is OK, but the "unregistered_ovf_of_entities" DB table should have a new XML, you can test that at the old environment it's 2 CPUs and 2048 MB, while at the fixed one 4 CPUs and 4096 MB !!!
a. "unregistered_ovf_of_entities" DB table:
~/dev/deploy/share/ovirt-engine/dbscripts/engine-psql.sh -c "select ovf_data from unregistered_ovf_of_entities where entity_guid = 'e397e809-1237-46fa-87de-fb669b1c9ec0';" | cut -c 2- | grep xml | xmllint --format - | egrep -B1 "Memory Size|Number of virtual CPU"
<rasd:Caption>2 virtual cpu</rasd:Caption>
<rasd:Description>Number of virtual CPU</rasd:Description>
--
<rasd:Caption>2048 MB of memory</rasd:Caption>
<rasd:Description>Memory Size</rasd:Description>
b. "vm_ovf_generations" DB table:
~/dev/deploy/share/ovirt-engine/dbscripts/engine-psql.sh -c "select ovf_data from vm_ovf_generations where vm_guid='e397e809-1237-46fa-87de-fb669b1c9ec0'" | cut -c 2- | grep xml | xmllint --format - | egrep -B1 "Memory Size|Number of virtual CPU"
<rasd:Caption>2 virtual cpu</rasd:Caption>
<rasd:Description>Number of virtual CPU</rasd:Description>
--
<rasd:Caption>2048 MB of memory</rasd:Caption>
<rasd:Description>Memory Size</rasd:Description>
16. Import "VM1" to the "NFS_SD1" Storage Domain on Engine 1 ("Storage" → "Storage Domains" → click on "NFS_SD1" → "VM Import" tab → Select "VM1" to import → Press "Import" button).
!!! The VM is imported with an old configuration, the problem is a stale data at "unregistered_ovf_of_entities" DB table, see the unlucky step 13 !!!
17. Run "engine-psql.sh" from the CLI on Engine 1 (at the fixed version "mem_size_mb" should be 4096 MB):
~/dev/deploy/share/ovirt-engine/dbscripts/engine-psql.sh -c "select vm_name, vm_guid, mem_size_mb, cpu_per_socket, min_allocated_mem, max_memory_size_mb from vm_static where vm_guid='e397e809-1237-46fa-87de-fb669b1c9ec0';"
vm_name | vm_guid | mem_size_mb | cpu_per_socket | min_allocated_mem | max_memory_size_mb
---------+--------------------------------------+-------------+----------------+-------------------+--------------------
VM1 | e397e809-1237-46fa-87de-fb669b1c9ec0 | 2048 | 1 | 2048 | 8192
(1 row)
(In reply to Pavel Bar from comment #3) > Steps to Reproduce: > ------------------ > 0. Environment (2 engines): > a. Engine 1 > Data Center "DC1", Cluster, Host, Storage Domain "NFS_SD1", 1 disk. > b. Engine 2 > Data Center "DC2", Cluster, Host. > > Note: at the following steps where VM is mentioned, do *exactly the same* > for VM template (can be tested in parallel, at the same flow). > > 1. Create a "VM1" VM on Engine 1 with 1 GB memory and 1 CPU. Click on the VM > and attach a disk to it. > > 2. Increase VM memory to 2 GB and CPU to 2. > > 3. Put the Storage Domain in maintenance mode ("Storage" → "Storage Domains" > → click on "NFS_SD1" → "Data Center" tab → select the "DC1" Data Center → > Press the "Maintenance" button). > > 4. Run "engine-psql.sh" from the CLI on Engine 1 (use the correct vm_guid > for VM / VM template): > pbar@engine1$ ~/dev/deploy/share/ovirt-engine/dbscripts/engine-psql.sh -c > "select ovf_data from vm_ovf_generations where > vm_guid='e397e809-1237-46fa-87de-fb669b1c9ec0'" | cut -c 2- | grep xml | > xmllint --format - | egrep -B1 "Memory Size|Number of virtual CPU" > <rasd:Caption>2 virtual cpu</rasd:Caption> > <rasd:Description>Number of virtual CPU</rasd:Description> > -- > <rasd:Caption>2048 MB of memory</rasd:Caption> > <rasd:Description>Memory Size</rasd:Description> > > 5. Detach the Storage Domain ("Storage" → "Storage Domains" → click on > "NFS_SD1" → "Data Center" tab → select the "DC1" Data Center → Press the > "Detach" button). > > 6. Run "engine-psql.sh" from the CLI on Engine 1: > ~/dev/deploy/share/ovirt-engine/dbscripts/engine-psql.sh -c "select > vm_guid,ovf_generation from vm_ovf_generations;" > vm_guid | ovf_generation > --------------------------------------+---------------- > e397e809-1237-46fa-87de-fb669b1c9ec0 | 5 > (1 row) > > 7. Import the "NFS_SD1" Storage Domain on Engine 2 ("Storage" → "Storage > Domains" → "Import Domain" button → fill "Name" and "Export Path" the same > as you used when initially adding "NFS_SD1" to Engine 1). > > 8. Attach the "NFS_SD1" Storage Domain to "DC2" Data Center on Engine 2 > ("Compute" → "Data Centers" → click on "DC2" → "Attach Data" button → Choose > "NFS_SD1"). > > 9. Import the "VM1" VM to the "NFS_SD1" Storage Domain on Engine 2 > ("Storage" → "Storage Domains" → click on "NFS_SD1" → "VM Import" tab → > Select "VM1" to import → Press "Import" button). The "VM1" was imported with > the correct configuration. > > 10. Increase the "VM1" memory and CPU to 4 GB and 4 CPU on Engine 2. > > 11. Put the "NFS_SD1" Storage Domain in maintenance mode on Engine 2 > ("Storage" → "Storage Domains" → click on "NFS_SD1" → "Data Center" tab → > select the "DC2" Data Center → Press the "Maintenance" button). > > 12. Detach the Storage Domain from Engine 2 ("Storage" → "Storage Domains" → > click on "NFS_SD1" → "Data Center" tab → select the "DC2" Data Center → > Press the "Detach" button). > > 13. Attach the "NFS_SD1" Storage Domain to "DC1" Data Center back to Engine > 1 ("Compute" → "Data Centers" → click on "DC1" → "Attach Data" button → > Choose "NFS_SD1"). At the end of the process, "DC1" Data Center is up and > running. > !!! The bug actually happens at this step - "unregistered_ovf_of_entities" > DB table still contains wrong XML (contains 2 CPUs and 2 GB memory) !!! > > 14. Confirm that the OVF_STORE is updated with the new value ("Storage" → > "Storage Domains" → select "NFS_SD1" → "..." menu → "Update OVFs"). > > 15. But both "unregistered_ovf_of_entities" and "vm_ovf_generations" DB > tables still contain the old values. > !!! "vm_ovf_generations" DB table is OK, but the > "unregistered_ovf_of_entities" DB table should have a new XML, you can test > that at the old environment it's 2 CPUs and 2048 MB, while at the fixed one > 4 CPUs and 4096 MB !!! > a. "unregistered_ovf_of_entities" DB table: > ~/dev/deploy/share/ovirt-engine/dbscripts/engine-psql.sh -c "select ovf_data > from unregistered_ovf_of_entities where entity_guid = > 'e397e809-1237-46fa-87de-fb669b1c9ec0';" | cut -c 2- | grep xml | xmllint > --format - | egrep -B1 "Memory Size|Number of virtual CPU" > <rasd:Caption>2 virtual cpu</rasd:Caption> > <rasd:Description>Number of virtual CPU</rasd:Description> > -- > <rasd:Caption>2048 MB of memory</rasd:Caption> > <rasd:Description>Memory Size</rasd:Description> > > b. "vm_ovf_generations" DB table: > ~/dev/deploy/share/ovirt-engine/dbscripts/engine-psql.sh -c "select ovf_data > from vm_ovf_generations where > vm_guid='e397e809-1237-46fa-87de-fb669b1c9ec0'" | cut -c 2- | grep xml | > xmllint --format - | egrep -B1 "Memory Size|Number of virtual CPU" > <rasd:Caption>2 virtual cpu</rasd:Caption> > <rasd:Description>Number of virtual CPU</rasd:Description> > -- > <rasd:Caption>2048 MB of memory</rasd:Caption> > <rasd:Description>Memory Size</rasd:Description> > > 16. Import "VM1" to the "NFS_SD1" Storage Domain on Engine 1 ("Storage" → > "Storage Domains" → click on "NFS_SD1" → "VM Import" tab → Select "VM1" to > import → Press "Import" button). > !!! The VM is imported with an old configuration, the problem is a stale > data at "unregistered_ovf_of_entities" DB table, see the unlucky step 13 !!! > > 17. Run "engine-psql.sh" from the CLI on Engine 1 (at the fixed version > "mem_size_mb" should be 4096 MB): > ~/dev/deploy/share/ovirt-engine/dbscripts/engine-psql.sh -c "select vm_name, > vm_guid, mem_size_mb, cpu_per_socket, min_allocated_mem, max_memory_size_mb > from vm_static where vm_guid='e397e809-1237-46fa-87de-fb669b1c9ec0';" > > vm_name | vm_guid | mem_size_mb | > cpu_per_socket | min_allocated_mem | max_memory_size_mb > ---------+--------------------------------------+-------------+-------------- > --+-------------------+-------------------- > VM1 | e397e809-1237-46fa-87de-fb669b1c9ec0 | 2048 | > 1 | 2048 | 8192 > (1 row) Reproduced on ovirt-engine-4.4.6.3-0.8.el8ev.noarch Verified with the above steps on ovirt-engine-4.4.6.5-447.gd80dda7.9.el8ev.noarch. Moving to 'Verified' Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: RHV Manager security update (ovirt-engine) [ovirt-4.4.6]), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2021:2179 Due to QE capacity, we are not going to cover this issue in our automation |
Description of problem: While detaching the storage domain, the VM information from vm_ovf_generations is not cleared causing VM import with old stale configuration. A storage domain was detached from environment A. This storage domain was then attached to a new environment B. Changed the configuration of the VMs from B. Detached the storage domain from B. Attached it back to the old environment A. Imported the VMs The VMs are imported without the configuration changes done on the new environment B. Version-Release number of selected component (if applicable): vdsm-client-4.40.35.1-1.el8ev.noarch rhvm-4.4.3.12-0.1.el8ev.noarch How reproducible: 100% Steps to Reproduce: 1. Create a VM in environment A with 1 GB memory and 1 CPU. 2. Increase Memory to 2 GB and CPU to 2. 3. Put the storage domain in maintenance mode. The vm_ovf_generation is updated with the new configuration. === # /usr/share/ovirt-engine/dbscripts/engine-psql.sh -c "select ovf_data from vm_ovf_generations where vm_guid='cbc144ab-50ae-4dc6-bd10-0f729fbc68cd'" | cut -c 2-|grep xml|xmllint --format - |egrep -B1 "Memory Size|Number of virtual CPU" <rasd:Caption>2 virtual cpu</rasd:Caption> <rasd:Description>Number of virtual CPU</rasd:Description> -- <rasd:Caption>2048 MB of memory</rasd:Caption> <rasd:Description>Memory Size</rasd:Description> === 4. Detach the storage domain. vm_ovf_generations still contains the VMs OVF data and was not cleaned. VMs entry was removed from all other tables. === engine=# select vm_guid,ovf_generation from vm_ovf_generations ; vm_guid | ovf_generation --------------------------------------+---------------- cbc144ab-50ae-4dc6-bd10-0f729fbc68cd | 5 (1 row) === 5. Attach the storage domain in environment B and import the VM. 6. The VM was imported with the correct configuration. Increase the memory and CPU to 4 GB and 4 CPU from environment B. 7. Detach the storage domain from environment B. 8. Attach the storage domain back to environment A. 9. Confirm that the OVF_STORE is updated with the new value. # dd if=/dev/1ca21662-7c9b-42f6-a001-faf56d3114d7/82b1413b-8068-4a5c-bf58-f01d5037e157 bs=1M |tar -xvf - info.json cbc144ab-50ae-4dc6-bd10-0f729fbc68cd.ovf metadata.json 472aa645-c290-4584-9d4f-7272c2cf0a76.ovf # cat cbc144ab-50ae-4dc6-bd10-0f729fbc68cd.ovf |xmllint --format - |egrep -B1 "Memory Size|Number of virtual CPU" <rasd:Caption>4 virtual cpu</rasd:Caption> <rasd:Description>Number of virtual CPU</rasd:Description> -- <rasd:Caption>4096 MB of memory</rasd:Caption> <rasd:Description>Memory Size</rasd:Description> 10. But both unregister_ovf_of_entities and vm_ovf_generations has old values. /usr/share/ovirt-engine/dbscripts/engine-psql.sh -c "select ovf_data from unregistered_ovf_of_entities where entity_guid = 'cbc144ab-50ae-4dc6-bd10-0f729fbc68cd';" | cut -c 2-|grep xml|xmllint --format - |egrep -B1 "Memory Size|Number of virtual CPU" <rasd:Caption>2 virtual cpu</rasd:Caption> <rasd:Description>Number of virtual CPU</rasd:Description> -- <rasd:Caption>2048 MB of memory</rasd:Caption> <rasd:Description>Memory Size</rasd:Description> /usr/share/ovirt-engine/dbscripts/engine-psql.sh -c "select ovf_data from vm_ovf_generations where vm_guid='cbc144ab-50ae-4dc6-bd10-0f729fbc68cd'" | cut -c 2-|grep xml|xmllint --format - |egrep -B1 "Memory Size|Number of virtual CPU" <rasd:Caption>2 virtual cpu</rasd:Caption> <rasd:Description>Number of virtual CPU</rasd:Description> -- <rasd:Caption>2048 MB of memory</rasd:Caption> <rasd:Description>Memory Size</rasd:Description> 11. Import the VM. The VM is imported with an old configuration. === engine=# select mem_size_mb,cpu_per_socket from vm_static where vm_guid='cbc144ab-50ae-4dc6-bd10-0f729fbc68cd' ; mem_size_mb | cpu_per_socket -------------+---------------- 2048 | 1 (1 row) === Actual results: vm_ovf_generations is not cleared while detaching the storage domain causing VM import with old stale configuration Expected results: Because of this bug, the VMs are imported with old stale configuration during Active Passive Disater Recovery. Additional info: