Bug 1298134 - v2v: No import rollback when deleting source VM during import.
v2v: No import rollback when deleting source VM during import.
Status: CLOSED CURRENTRELEASE
Product: ovirt-engine
Classification: oVirt
Component: BLL.Virt (Show other bugs)
3.6.1
Unspecified Unspecified
low Severity medium (vote)
: ovirt-3.6.6
: 3.6.6
Assigned To: Shahar Havivi
Nisim Simsolo
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2016-01-13 05:24 EST by Nisim Simsolo
Modified: 2016-05-30 06:53 EDT (History)
7 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-05-30 06:53:34 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: Virt
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
rule-engine: ovirt‑3.6.z+
mgoldboi: planning_ack+
rule-engine: devel_ack+
rule-engine: testing_ack+


Attachments (Terms of Use)
engine log (24.91 KB, application/x-gzip)
2016-01-13 05:27 EST, Nisim Simsolo
no flags Details
vdsm log (772.63 KB, application/x-gzip)
2016-01-13 05:27 EST, Nisim Simsolo
no flags Details
VMware VMs (2.04 KB, text/plain)
2016-02-02 09:01 EST, Nisim Simsolo
no flags Details
host processes (50.35 KB, text/plain)
2016-02-02 09:03 EST, Nisim Simsolo
no flags Details
getVdsStats (8.80 KB, text/plain)
2016-02-02 09:05 EST, Nisim Simsolo
no flags Details
new vdsm.log (724.42 KB, application/x-gzip)
2016-02-02 09:06 EST, Nisim Simsolo
no flags Details
new engine.log (113.55 KB, application/x-gzip)
2016-02-02 09:07 EST, Nisim Simsolo
no flags Details


External Trackers
Tracker ID Priority Status Summary Last Updated
oVirt gerrit 55477 master MERGED Handle closed stream when virt-v2v fails 2016-03-31 12:16 EDT
oVirt gerrit 55607 ovirt-3.6 MERGED Handle closed stream when virt-v2v fails 2016-04-04 02:57 EDT

  None (edit)
Description Nisim Simsolo 2016-01-13 05:24:24 EST
Description of problem:
- When deleting source VM during import, the import progress get stuck in webadmin.
- Observing VdsStats shows that v2v job is still running.
- Observing host processes shows that there are no v2v processes.

Version-Release number of selected component (if applicable):
rhevm-3.6.2-0.1.el6
sanlock-3.2.4-1.el7.x86_64
libvirt-client-1.2.17-13.el7_2.2.x86_64
qemu-kvm-rhev-2.3.0-31.el7_2.4.x86_64
vdsm-4.17.15-0.el7ev.noarch

How reproducible:
Consistently. not dependent in import progress phase (initializing, copying etc.). 

Steps to Reproduce:
1. Import VM from VMware environment.
2. During import, delete VMware VM.
3.

Actual results:
Import process get stuck without rolling back.

Expected results:
VM conversion should be canceled with rollback. 

Additional info:
vdsm and engine log attached.
Import started at 2016-01-13 09:02:10,873 (engine.log)
Comment 1 Nisim Simsolo 2016-01-13 05:27 EST
Created attachment 1114355 [details]
engine log
Comment 2 Nisim Simsolo 2016-01-13 05:27 EST
Created attachment 1114356 [details]
vdsm log
Comment 3 Michal Skrivanek 2016-01-29 08:57:25 EST
Please add logs/proof that on host v2v process was no longer running; or v2v logs.

Looking at vdsm logs there is no indication about anything - Shahar, are you sure when v2v dies it is reported back to vdsm correctly?
Comment 4 Shahar Havivi 2016-01-31 03:03:08 EST
(In reply to Michal Skrivanek from comment #3)
> Please add logs/proof that on host v2v process was no longer running; or v2v
> logs.
> 
> Looking at vdsm logs there is no indication about anything - Shahar, are you
> sure when v2v dies it is reported back to vdsm correctly?
Yes,
as a test I started to import and stop the engine (for not rolling back and clear the job) killed virt-v2v and the stats show:
        v2vJobs = {'423c38b0-37f6-4d1b-da17-bd1d55fe296c': {'description': "Job '423c38b0-37f6-4d1b-da17-bd1d55fe296c' process failed exit-code: -9, stderr: ",
                                                            'progress': 0,
                                                            'status': 'error'}}
Comment 5 Michal Skrivanek 2016-02-02 06:16:33 EST
so, that would be an indication that v2v didn't die, right
(adding back needinfo on nisim)
Comment 6 Nisim Simsolo 2016-02-02 08:51:43 EST
I reproduced it again.. 
when deleting source VM during rhevm "copying disk" phase, the import get stuck in this phase forever.
as for the jobs and processes:
v2v job is still running (see getVdsStats.txt attached)
v2v process is also running [virt-v2v] <defunct> (see host_process.txt attached)
Also, see vmware_vms.txt for existing vmware VMs. the source VM deleted name is RHEL7_test.
vdsm and engine log also attached.

Another thing important to mention, when deleting source VM during initialize phase, import progress is rolled back as expected.
Comment 7 Nisim Simsolo 2016-02-02 09:01 EST
Created attachment 1120441 [details]
VMware VMs
Comment 8 Nisim Simsolo 2016-02-02 09:03 EST
Created attachment 1120442 [details]
host processes
Comment 9 Nisim Simsolo 2016-02-02 09:05 EST
Created attachment 1120443 [details]
getVdsStats
Comment 10 Nisim Simsolo 2016-02-02 09:06 EST
Created attachment 1120444 [details]
new vdsm.log
Comment 11 Nisim Simsolo 2016-02-02 09:07 EST
Created attachment 1120445 [details]
new engine.log
Comment 12 Nisim Simsolo 2016-05-10 09:01:34 EDT
Verified. 
rhevm-3.6.6.2-0.1.el6
qemu-kvm-rhev-2.3.0-31.el7_2.10.x86_64
vdsm-4.17.28-0.el7ev.noarch
sanlock-3.2.4-2.el7_2.x86_64
libvirt-client-1.2.17-13.el7_2.4.x86_64

Verification scenario:
1. Import VM from VMware.
2. During "initializing" phase, delete source VM and verify ConvertVmCallback started, VM deleted from webadmin and its disks were deleted also.
3. Import another VM from VMware.
4. During "copying" phase, delete source VM and verify ConvertVmCallback started, VM deleted from webadmin and its disks were deleted also.

Note You need to log in before you can comment on or make changes to this bug.