Bug 1524455 - Fails to import lease for HA VM with lease while export/import from 4.1 to 4.2 procedure
Summary: Fails to import lease for HA VM with lease while export/import from 4.1 to 4...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: ovirt-engine
Classification: oVirt
Component: BLL.Virt
Version: 4.2.0
Hardware: x86_64
OS: Linux
medium
high
Target Milestone: ovirt-4.2.2
: 4.2.2.3
Assignee: Shmuel Melamud
QA Contact: Polina
URL:
Whiteboard:
: 1547968 (view as bug list)
Depends On:
Blocks: 1547991
TreeView+ depends on / blocked
 
Reported: 2017-12-11 14:49 UTC by Polina
Modified: 2018-04-18 12:27 UTC (History)
8 users (show)

Fixed In Version: ovirt-engine-4.2.2.3
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-04-18 12:27:12 UTC
oVirt Team: Virt
Embargoed:
rule-engine: ovirt-4.2+
mtessun: planning_ack+
rule-engine: devel_ack+
rule-engine: testing_ack+


Attachments (Terms of Use)
engine.log (10.10 MB, text/plain)
2017-12-11 14:49 UTC, Polina
no flags Details
4.2engine.log (2.08 MB, text/plain)
2018-03-29 10:38 UTC, Polina
no flags Details


Links
System ID Private Priority Status Summary Last Updated
oVirt gerrit 88314 0 master MERGED core: Default SD for imported VM with lease 2018-03-01 13:55:10 UTC
oVirt gerrit 88411 0 ovirt-engine-4.2 MERGED core: Default SD for imported VM with lease 2018-03-04 08:05:39 UTC

Description Polina 2017-12-11 14:49:44 UTC
Created attachment 1366031 [details]
engine.log

Description of problem:HA VM with lease that was exported in 4.1 has been imported in 4.2 without lease


Version-Release number of selected component (if applicable):
rhvm-4.2.0-0.6.el7.noarch
rhv-release-4.2.0-8-001.noarch

How reproducible:100%


Steps to Reproduce:
1. In 4.1.8.2-0.1.el7create HA VM with lease (nfs_2). check that the lease exists by running on the host command <sanlock client status>
2. Export the VM
3. in 4.2 import VM 

Actual results: VM is imported without lease (HA no lease)


Expected results: must be the same HA with lease


Additional info: engine.log attached

Comment 1 Tomas Jelinek 2017-12-12 07:59:04 UTC
It does not reproduce to me. I will need some more info:
1: please provide the OVF of the VM after exported
2: does the storage domain with same ID still exist in the new setup?

Comment 2 Tomas Jelinek 2017-12-13 12:26:13 UTC
so, after an offline discussion with Polina and Arik, this is the conclusion:
- currently the export/import VM with lease will preserve the lease only if the lease domain ID did not change. This means it works only if exported/imported in the same setup.
- the fix should be like this:
  - when importing, try to use the lease from OVF if possible
  - if not possible, pick any active data domain from the DC where the VM is being imported
  - only if there is none, null the lease ID out and issue a warning to audit log

The fix should be fairly easy and safe, targeting 4.2.2

Comment 3 Arik 2017-12-28 15:54:48 UTC
Tal, back in the early days of VM leases we were concerned about where to create the lease, in which storage domain. We even thought about proposing users to have a dedicated storage domain for that. But now that we have the ability to hot-(un)plug a lease, we can easily move it between storage domains and so it doesn't matter to you if as part of the flow described in comment #2 (where the original storage domain is not part of the data center we import the VM to) I'll pick an arbitrary storage domain that is active, right?

Comment 4 Tal Nisan 2017-12-31 09:24:02 UTC
I don't thing choosing an arbitrary domain for the lease is the right approach, if you import a VM with a lease and the original SD of the lease does not exist you should probably null the lease (as it is now) and issue an audit log saying that the lease could not be imported and advise the user to create a lease again on the SD of his choice

Comment 5 Arik 2017-12-31 09:41:55 UTC
(In reply to Tal Nisan from comment #4)
But look at it from the virt-perspective - if you export and import a VM, we do our best to preserve the VM configuration. Advising the user assumes the operation is done via our webadmin (otherwise you need to have an indication on the VM - that sounds like an over-spec in this case) and is neither suitable for operations that are triggered from cloud-forms nor from sdk. Moreover, even if we alert the user that we broke our promise to import the VM as-is and the VM is missing a lease, imagine a user that does it for 20 VMs that wants to migrate from one data-center to another - recreating a lease for those 20 VMs in the new data center using our webadmin is not a trivial task.

Comment 6 Michal Skrivanek 2018-02-22 11:40:52 UTC
why is this urgent?

Comment 7 Martin Tessun 2018-02-25 15:22:43 UTC
(In reply to Michal Skrivanek from comment #6)
> why is this urgent?

It isn't:

I simply had the wrong field selected in the previous update:
 Martin Tessun 2018-02-07 07:39:00 EST
Priority: unspecified → urgent
Flags: planning_ack? → planning_ack+

which I changed a minute later:
Martin Tessun 2018-02-07 07:41:37 EST
Priority: urgent → medium

Comment 8 Polina 2018-03-12 09:38:03 UTC
tested on rhv-release-4.2.2-4-001.noarch . export is done in rhv-release-4.1.10-5-001.noarch. 

created HA VM in v4.1 with lease nfs_0. Export.
After import in 4.2 DC - the VM has no lease

Comment 9 Eyal Shenitzky 2018-03-13 06:33:10 UTC
*** Bug 1547991 has been marked as a duplicate of this bug. ***

Comment 10 Eyal Shenitzky 2018-03-13 08:12:35 UTC
*** Bug 1547968 has been marked as a duplicate of this bug. ***

Comment 11 Yaniv Kaul 2018-03-15 13:57:31 UTC
Is anyone looking at this?
Is this going into 4.2.2? If not, please defer.

Comment 12 Shmuel Melamud 2018-03-15 14:59:22 UTC
I have a possible fix for this, I'll try to prepare a patch today.

Comment 13 Yaniv Kaul 2018-03-19 08:42:46 UTC
(In reply to Shmuel Melamud from comment #12)
> I have a possible fix for this, I'll try to prepare a patch today.

Any updates? Is it going to make it to 4.2.2?

Comment 14 Shmuel Melamud 2018-03-19 11:28:17 UTC
The fix (https://gerrit.ovirt.org/#/c/89148/) was merged. Backporting now.

Comment 15 Polina 2018-03-29 10:38:09 UTC
Hi, I'm verifying it on rhv-release-4.2.2-8-001.noarch.

Here are two issues I would like to raise:

issue1: 
1. create in 4.1 VM1 with lease nfs_0 (Master in 4.1) and VM2 with lease iscsi_0. while 4.2 environment has these domains as well.
2. After 4.1import/4.2 export I see these VMs in 4.2, both with the same lease nfs_0 (nfs_0 is master in 4.2 ). 

is it expected? I expected that the VMs must have the same lease in 4.2

issue2:
import process of two VMs took 2.5 hr (engine log attached). I tried in two different environments. what is expected?

4.2engine.log attached

Comment 16 Polina 2018-03-29 10:38:55 UTC
Created attachment 1414717 [details]
4.2engine.log

Comment 17 Shmuel Melamud 2018-04-09 20:41:08 UTC
(In reply to Polina from comment #15)
1. 4.2 environment has the same storage domains as 4.1 environment, but do they have the same IDs in both environments?

2. Import may take a lot of time depending on disk sizes, network and storage speed.

Comment 18 Polina 2018-04-10 05:52:32 UTC
(In reply to Shmuel Melamud from comment #17)

> 1. 4.2 environment has the same storage domains as 4.1 environment, but do
> they have the same IDs in both environments?

the 4.1 env has nfs_0, nfs_1, nfs_2, iscsi_0, iscsi_1, iscsi_2.
the 4.2 env has nfs_0, nfs_1, nfs_2, iscsi_0.

since I exported VMs with the lease IDs existing in both environments (VM1 with lease nfs_0, VM2 with lease iscsi_0 ), I expected that they are imported with the same leases

Comment 19 Shmuel Melamud 2018-04-10 19:35:31 UTC
(In reply to Polina from comment #18)

Can you please run the following SQL query:

select id, storage_name from storage_domains;

in both 4.1 and 4.2 environments? What result it returns?

Comment 20 Polina 2018-04-12 05:35:00 UTC
4.2:

engine=# select id, storage_name from storage_domains;
                  id                  |     storage_name      
--------------------------------------+-----------------------
 235d7ccc-900f-4ce6-824f-0ff0f6b30550 | iscsi_0
 02cf884d-cd3c-48d7-add3-3bb694fc2bec | test_gluster_1
 d31037e5-0d45-4306-975b-73eea845ae86 | netappISO
 be038777-9f51-4cc7-9861-1e21f94bf4e1 | nfs_0
 5e92762f-9af9-406c-98c7-302ecc73fcc5 | test_gluster_3
 c1d2c3db-3cef-4530-bf29-5ab6cdbecb41 | export_domain
 11fe39f3-5f3b-4929-a286-4e3315cece5d | test_gluster_2
 be2ee14d-d4da-4a75-8622-bc015c55fe0a | nfs_2
 4348e7d6-473d-4aa1-97d1-b110c5b66de8 | rhevm-qe-infra-glance
 c0a2ce35-2fc3-4c44-9a68-aa939d9270cd | nfs_1
(10 rows)

4.1:

engine=# select id, storage_name from storage_domains;
                  id                  |      storage_name      
--------------------------------------+------------------------
 683cde84-6e24-4470-88fe-8d21c0edd542 | exp_domain
 d8fca7bc-c9a1-479a-8bbd-a147d2620f86 | export_domain
 5ed87b75-33d3-4571-a363-820e6a9e7cb4 | test_gluster_0
 f508f9bd-2723-43af-9f39-9f547354a6ca | ovirt-image-repository
 fd5eeac5-8d6b-482d-80a4-f01d9241473b | iscsi_0
 6224dd67-36d6-4c48-9475-b1e1ff3db779 | iscsi_1
 f268a1fb-0c18-406c-90b5-e79083587d09 | test_gluster_1
 49b950d4-9614-4a70-b6c8-b9a503ffbe76 | nfs_1
 d31037e5-0d45-4306-975b-73eea845ae86 | netappISO
 139c5bc2-514b-45c9-a3e6-4c1a3e79f6bb | iscsi_2
 e51cc388-8023-4656-96d3-2ddda109271b | nfs_2
 6fa2949f-2102-4a81-bc0a-ec7fcd662e05 | rhevm-qe-infra-glance
 bb4a04a6-4b6f-4798-9f33-42314b82f3c1 | test_gluster_2
 7098fb39-a635-4e5f-9434-f2c9bb4344df | nfs_0
(14 rows)

engine=#

Comment 21 Shmuel Melamud 2018-04-12 15:28:55 UTC
So, you can see that, for example, iscsi_0 storage domain has ID 235d7ccc-900f-4ce6-824f-0ff0f6b30550 in 4.2 environment, but in 4.1 environment it has ID fd5eeac5-8d6b-482d-80a4-f01d9241473b. Thus, from the Engine point of view these are different storage domains. When a VM with a lease is exported, the ID of the storage domain where the lease was located is written in the OVF. When the VM is imported, Engine looks for the storage domain with the same ID. If it is not found, the lease is created on default storage domain.

Comment 22 Polina 2018-04-16 13:23:40 UTC
Thank you, Shmuel. I so, I can verify the bug

The version for verification is rhv-release-4.2.2-8-001.noarch

Comment 23 Sandro Bonazzola 2018-04-18 12:27:12 UTC
This bugzilla is included in oVirt 4.2.2 release, published on March 28th 2018.

Since the problem described in this bug report should be
resolved in oVirt 4.2.2 release, it has been closed with a resolution of CURRENT RELEASE.

If the solution does not work for you, please open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.