Bug 1052097 - VM migration back to original host fail on "file exists"
Summary: VM migration back to original host fail on "file exists"
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: vdsm
Version: 3.3.0
Hardware: Unspecified
OS: Unspecified
Target Milestone: ---
: 3.4.0
Assignee: Vinzenz Feenstra [evilissimo]
QA Contact: Ilanit Stein
Whiteboard: virt
: 1085801 (view as bug list)
Depends On: 1082986
Blocks: 1088906
TreeView+ depends on / blocked
Reported: 2014-01-13 10:20 UTC by Ilanit Stein
Modified: 2019-04-28 09:17 UTC (History)
13 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Previously, when links were left behind on a host, migration of a virtual machine to that host failed and the virtual machine was shut down on the host. This happened in cases in which the version of VDSM supported by Red Hat Enterprise Virtualization 3.2 (4.10.2) was present on the host. This was caused by a failure to remove guest socket files. Only symlinks to the guest socket files were removed. Links are now cleaned when virtual machines are migrated from hosts (the guest socket files linked to are now removed prior to the removal of the symlinks). Virtual machines no longer shut down when they are migrated back to those hosts from other hosts.
Clone Of:
: 1082986 1088906 (view as bug list)
Last Closed: 2014-06-09 13:27:35 UTC
oVirt Team: ---
Target Upstream Version:

Attachments (Terms of Use)
engine.log migration start & fail @ 2014-Jan-12, 16:21 (810.71 KB, application/x-gzip)
2014-01-13 10:21 UTC, Ilanit Stein
no flags Details
host1 logs (576.11 KB, application/zip)
2014-01-13 10:22 UTC, Ilanit Stein
no flags Details
host2 logs (848.21 KB, application/zip)
2014-01-13 10:23 UTC, Ilanit Stein
no flags Details

System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2014:0504 0 normal SHIPPED_LIVE vdsm 3.4.0 bug fix and enhancement update 2014-06-09 17:21:35 UTC
oVirt gerrit 26278 0 None None None Never
oVirt gerrit 26288 0 None MERGED virt: Remove all guest sockets on cleanup Never
oVirt gerrit 26882 0 None MERGED vm: Fix guest channel symlink creation handling Never
oVirt gerrit 27183 0 None MERGED vm: Fix guest channel symlink creation handling Never

Description Ilanit Stein 2014-01-13 10:20:41 UTC
Description of problem:

On rhevm 3.3 (is30), Have 2 3.3 hosts in 3.2 cluster.
Downgrade vdsm of host1 (remove the 3.3 vdsm, and install 3.2 vdsm).
Run a VM on host1. 
Migrate the VM to host2.
Upgrade host1 vdsm to 3.3.
Migrate VM back to original host, host1
Migration failed on "file exists" error.

Version-Release number of selected component (if applicable):
vdsm 3.3: (is30  ) vdsm-4.13.2-0.5.el6ev.x86_64.rpm
vdsm 3.2: (sf22.4) vdsm-4.10.2-30.1.el6ev.x86_64.rpm

Additional info:
event error on rhevm: VM mig_ver is down. Exit message: [Errno 17] File exists.

vdsm error on destination host (host1):
Thread-38::ERROR::2014-01-12 16:21:09,752::vm::2162::vm.Vm::(_startUnderlyingVm) vmId=`53396dbd-388f-47be-a936-4a92a5dc6a6a`::The vm start process failed
Traceback (most recent call last):
  File "/usr/share/vdsm/vm.py", line 2140, in _startUnderlyingVm
  File "/usr/share/vdsm/vm.py", line 3575, in _waitForIncomingMigrationFinish
  File "/usr/share/vdsm/vm.py", line 2926, in _domDependentInit
  File "/usr/share/vdsm/vm.py", line 2912, in _updateAgentChannels
    os.symlink(path, socketPath)
OSError: [Errno 17] File exists
Thread-38::DEBUG::2014-01-12 16:21:09,766::vm::2615::vm.Vm::(setDownStatus) vmId=`53396dbd-388f-47be-a936-4a92a5dc6a6a`::Changed state to Down: [Errno 17] File exists

Comment 1 Ilanit Stein 2014-01-13 10:21:47 UTC
Created attachment 849248 [details]
engine.log migration start & fail @ 2014-Jan-12, 16:21

Comment 2 Ilanit Stein 2014-01-13 10:22:52 UTC
Created attachment 849249 [details]
host1 logs

Comment 3 Ilanit Stein 2014-01-13 10:23:30 UTC
Created attachment 849250 [details]
host2 logs

Comment 4 Michal Skrivanek 2014-02-03 15:45:29 UTC
not sure if 3.3.z needed, let's review

Comment 5 Vinzenz Feenstra [evilissimo] 2014-02-04 11:36:32 UTC
*** Bug 1052841 has been marked as a duplicate of this bug. ***

Comment 7 Vinzenz Feenstra [evilissimo] 2014-04-01 09:18:18 UTC
Merged to u/s master as http://gerrit.ovirt.org/gitweb?p=vdsm.git;a=commit;h=4a7af0277aa441478b040163190ded27a9d7dcc2

Comment 8 Vinzenz Feenstra [evilissimo] 2014-04-01 13:46:02 UTC
Merged u/s to ovirt-3.4 branch as http://gerrit.ovirt.org/gitweb?p=vdsm.git;a=commit;h=be6302a938c1a1f1dac1e03b65ce10c7fecceb92

Comment 10 Michal Skrivanek 2014-04-16 11:30:51 UTC
discovered an issue described in bug 1085801. We need an additional fix and possibly a backport to 3.3 as more and more people are hitting this

Comment 11 Michal Skrivanek 2014-04-16 11:31:42 UTC
*** Bug 1085801 has been marked as a duplicate of this bug. ***

Comment 14 Ilanit Stein 2014-05-01 21:35:37 UTC
Verified on rhevm3.4 av8.1:

Had 3.2 cluster.
host_1 3.3 vdsm vdsm-4.13.2-0.14.el6ev (from is36)
host_2 3.4 vdsm vdsm-4.14.7-0.2.rc.el6ev.x86_64
Run a VM on host_1
Migrate VM to host_2
Upgrade vdsm on host_1 from 3.3 to 3.4
Migrate VM back to host_1

NO migration failures.

Comment 15 errata-xmlrpc 2014-06-09 13:27:35 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.